Re: Renesas SH4 support in emdebian
Neil Williams <codehelp@debian.org> writes:
> On Wed, 27 Jan 2010 21:42:55 +0100
> Goswin von Brederlow <goswin-v-b@web.de> wrote:
>
>> Neil Williams <codehelp@debian.org> writes:
>> > You're misunderstanding the problem - you're thinking of it
>> > downstream (apt), the problem exists upstream (repository
>> > creation).
>
> ... and you're STILL missing that precise point.
>
>> > The issue arises *before* the Packages file is created
>> > in the local repository, i.e. when trying to automate the creation
>> > of a new archive to use these port archives as a base for
>> > subsequent modification of the packages and inclusion into another
>> > repository. It isn't a case of just rsync because only a part of
>> > the archive is needed (one package at a time). There is no easy way
>> > to reconstruct the archive to what it would be on ftp.debian.org if
>> > the sh4 arch was to be accepted as a release architecture for
>> > Debian main.
>>
>> Sure there is: reprepro. :)
>
> reprepro needs all the conf/ files to be created using only the manpage
> as a guide before anything can happen - and errors in the first
> conf/distributions can cause massive problems later down the road as I
> found to my cost with Crush 1.0.
>
>> > When there is more than one pool*/ directory, there has to be a
>> > manual step to work out how to create one pool from the many. In
>> > this case it means multiple update rules per suite.
>>
>> Not really, fetch Packages file, iterate over all packages therein,
>> convert and dump them into the local archive.
>
> and what creates the local archive? Once something or someone has
> created the conf files, reprepro can do all this. You're one stage
> ahead of the problem - the creation of the files in conf/ for three
> repositories.
>
>> As said, a tool that doesn't use the Packages file to get the url for
>> debs but just assume files are in certain places is just buggy.
>
> No. It's not about the path, it is about how to collate two different
> repositories back into one.
>
>> > Grip uses three repositories:
>> >
>> > filter/ - a selective mirror of a standard Debian mirror with one
>> > pool.
>>
>> Which only has one pool if you use reprepro to fetch the packages. So
>> there is no problem there even with the grip scripts assuming a fixed
>> pool/main/<hash>/<package/ path.
>
> reprepro can't get anything until the conf files are created! It does
> NOT create them for you. Yes, filter/ uses a single pool - the problem
> is CREATING the conf/ files to allow reprepro to fetch the packages
> themselves into that single pool/. You're looking at the problem long
> after the real issue - and yes, you're right there is no problem
> updating filter/ ONCE IT EXISTS.
>
>> > grip/ - the processed package repository, again in standard format
>> > locale/ - the TDeb mirror, packages built during the grip
>> > processing.
>> >
>> > The problem with pool*/ vs pool/ affects the initial setup of the
>> > filter repository - this stage can be automated *if* the mirror from
>> > which we update is a typical Debian mirror with a single pool/.
>> > reprepro can work with the pool*/ - it just needs manual tweaking of
>> > the update rules in the first place.
>>
>> Don't see where:
>>
>> conf/distributions:
>> Origin: sh4
>> Label: sh4
>> Suite: sid
>> Codename: sid
>
> WRONG!
Couldn't think of a suitable codename and it really doesn't matter what
it is for the filter repository. That is only used as input for the
conversion, right?
>> Version: 666
>> Architectures: sh4
>
> Must have source.
Then add it. It is just an example. Didn't want to download everything.
>> Components: main
>> Description: Debian sh4
>> Update: - sh4
>> SignWith: yes
>
> that won't work - you need a real key there.
That will use the users default key. Left over from when you couldn't
put a key id there.
>> DscIndices: Sources Release . .gz
>> DebIndices: Packages Release . .gz
>
> and where did all that come from? You had to write it and immediately
> fell into the problem of confusing Suite and Codename.
Cut&paste.
> A mirror created with such a conf/distributions will fail. Suite is the
> symlink (unstable, testing, stable, oldstable, experimental etc.) and
> Codename is the release name (etch, lenny, squeeze, sid).
Mirrors fine as mirroring does not run reprepro createsymlinks. At least
not here.
> This problem *has* been highlighted in the manpage for reprepro after a
> bug report (from me) about how easily Suite and Codename can be
> confused and the chaos that results if you get it wrong - as you have.
>
> Suite and Codename cannot be the same - the symlink cannot be created.
The files above are just to show that it will mirror everything into a
single pool. Adjust them to suite you. None of your complains has
anything to do with upstream having multiple pools. That fact just does
not enter into things. That was my point.
> $ man reprepro
>
> Codename
> This required field is the unique identifier of a distribution and
> used as directory name within dists/ It is also copied into the Release
> files.
>
> Note that this name is not supposed to change. You most likely
> never ever want a name like testing or stable here (those are suite
> names and supposed to point to another distribution later).
>
> Suite
> This optional field is simply copied into the Release files. In
> Debian it contains names like stable, testing or unstable. To create
> symlinks from the Suite to the Codename, use the createsymlinks
> command of reprepro.
>
> Codename: sid is ALWAYS wrong, just as it would be for testing or
> stable.
>
> Plus, this repository is going to need to have unstable, testing,
> stable, stable-proposed-updates and testing-proposed-updates - for all
> three repositories - maybe even experimental. Creating all those conf/
> files yourself isn't looking quite so appealing now, is it?
>
>> conf/updates:
>> Name: sh4
>> Method: http://ftp.ch.debian.org/debian-ports
>
> And you get no source packages because you've only got one update rule.
> You do need the source package for any public repository (licensing
> reasons) and in order to generate all the TDebs.
Now that is the first valid point you made and still somewhat wrong.
The problem is that upstream has no source in the first palce. There
should be a
http://ftp.ch.debian.org/debian-ports/dists/sid/main/source/Sources.gz
As such ftp.ch.debian.org is in violation of the GPL and other licenses.
But again not a problem of multiple pools.
>> pool/main/3/3dchess/3dchess_0.8.1-16_sh4.deb
>> pool/main/4/4g8/4g8_1.0-3_sh4.deb
>> pool/main/6/6tunnel/6tunnel_0.11rc2-2_sh4.deb
>
> and no .dsc - that's just not going to work.
>
>> You do filter by package name and not by pool/main/<hash>/<name>,
>> right?
>
> Why? There are no packages and no Packages file at the point where you
> need to decide how to create the files in conf/ and how to recombine
> the source (which is in pool/ in the debian/ repository) with the
> ported sh4 .debs which are in pool-sh4/.
No, the source in debian/pool/ is not the source for
debian-ports/pool-sh4/. Nothing will garanty that the version of the
source debian/pool has is the same as the binaries the sh4 port has and it
will have mismatches on a daily and continuing basis.
If it where you would only need a second entry in updates though.
> I don't have to parse path: from the Packages file at all, I never use
> it. This is about creating the conf/ files that determine the path
> during the creation of the three repositories used to build a Grip
> archive.
>
>> PS: sorry to be so adversarial but I think there just is no problem
>> here.
>
> Please understand that you've got completely the wrong end of the
> problem!
>
> You created the conf files yourself (and got them wrong) - it is THAT
> stage that needs to be automated.
>
> You made two errors, the first is EXACTLY the same bug as I encountered
> originally (and which the manpage now clearly indicates is wrong) and
> the second is the same bug as the current automation would cause!
>
> You proved the problem exists in your own example!
You proved I'm bad writing perfect examples. You haven't shown any
problem with the split pool directory. You also shown that
ftp.ch.debian.org is violating the GPL by having binaries without their
sources and that it will be hard to fix that when mirroring them.
So in conclusion the pool (arch all) + pool-sh4 (arch sh4) split makes
no difference. But adding sh4 support is still hard.
MfG
Goswin
Reply to: