Re: next generation apt/dpkg-cross
Neil Williams <codehelp@debian.org> writes:
> On Wed, 20 Jan 2010 15:43:40 +0100
> Goswin von Brederlow <goswin-v-b@web.de> wrote:
>> > We must avoid having -cross packages depending on non-multiarched
>> > native packages - even the dummy -cross ones. Arch:all is fine.
>>
>> I don't see why. Since I'm populating the apt database with the
>> native, the foreign multiarch and the -cross packages the update
>> process can use either one or any combination of them. So when a new
>> version of a package comes out the -cross package will be upgraded
>> just like a native or foreign multiarch package.
>
> How are you populating the apt cache? Trying to fool apt into thinking
> that -cross packages exist in a repository is going to break when
> dependency chains get long.
But they do exist. Apt-get will download them and tell dpkg to install
them. The dependency chanins for the -cross packages are at most one
item longer than the non -cross packages because the only time the
chain gets longer is when a dummy -cross package depends on a
multiarch package. If apt can not handle chains one longer than now
then there is something seriously wrong in apt.
>From experience I can say I never had a problem of apt not being able
to resolve dependencies with -cross packages for me and I used it to
install a wine-i386-cross for example (which has quite a big
dependency chain).
>> When the -cross
>> package is no longer needed (no reverse depends on it) the package
>> will show up in the list of automatic removals (unless set to manual)
>> in apt or aptitude.
>
> With apt-cross and package renaming, this caused horrendous problems in
> real situations - apt-get simply failed to work or would force the
> removal of the entire set of -cross packages when any depended on
> native packages.
Which must be an artifact of the mess the existing apt-cross does. I
have never seen any apt problem that I didn't get without -cross
packages also so far.
>> Since you do not have suite informations on the dpkg-cross level I
>> would stick with the -cross name for a dependency as long as any used
>> suite has that package non-multiarch.
>
> How can dpkg-cross know that when it knows nothing about suites? If we
> take that as global, it includes old-stable. If we try to pin that to
> one installation, deciding which suites are actually used is very
> complex. apt-cross simply cannot handle it.
>
> However, the support to help apt-cross is just in the apt-cross package
> itself - if we replace apt-cross, we don't have to have suite-specific
> lists of multiarched packages.
The suites being used are configured in sources.list and updated via
apt-get update. A Apt::Update::Post-Invoke script can go through all
*_Packages files and filter out a list of packages that are Multi-Arch
in all used suits.
You already mentioned apt updating the list of multiarch packages
automatically. This would do the same but give a conservative and suit
indepedent list.
> Current apt-cross will continue to operate *within* any suite which
> currently contains it *as long as* builds on that system target Lenny.
> The next version of apt-cross will be able to cope with some level of
> multiarched packages but there'll come a point where the pain could
> become too great and apt-cross would have to deprecated for all except
> building to target Lenny.
>
>> Only when all used sites have a
>> package as multiarch should the dependency use the multiarch package
>> directly. For any chroot with just one suite that would mean it would
>> always use the multiarch packages as soon as available, which is what
>> you (and me) want. On the other hand for setups with multiple suites
>> it would remain on the safe side and rather use -cross when in doubt.
>>
>> Would that be a acceptable middle ground?
>
> With apt-cross, multiple suites just don't work - it requires a lot of
> wrapping to ensure apt-cross is always called with the right --suite
> option.
>
>> > What's been missing is how to get apt to do that *after* we've
>> > mangled the package names by adding '$arch-cross'. Once we stop
>> > doing that, apt is just fine with the dependency resolution in
>> > download mode. (See multistrap.) Once we fix that and get dpkg to
>> > be able to install the downloaded .debs, apt can take over from
>> > apt-cross.
>>
>> I never had a problem with that in my implementation.
>
> If it can really do that with complex dependency chains like installing
> a full cross-building environment for building GTK+ applications in a
> clean chroot (see #502433) then apt-cross can be replaced.
I believe it can handle that trivially. I will put #502433 down as
test case.
> If apt-cross is replaced, those multiarch files read by dpkg-cross can
> remain empty, as you wanted. The helper is part of apt-cross and
> doesn't have to be run.
>
> However, the two methods are incompatible. apt-cross cannot do what
> your scripts can do and your scripts rely on functionality that is not
> in older suites.
>
> Can we get your scripts (under a suitable name) into Debian so that
> users and packages can migrate to multiarch?
>
> I think we can keep apt-cross for Lenny and before, demarcate
> Squeeze/Sid as the painful transition from apt-cross to apt-multiarch
> and from there on to full multiarch.
Do we need an apt-cross in squeze just to build lenny packages? Can't
lenny builds done from a lenny base?
What my scripts need is a new enough apt (hopefully squeeze). If you
have apt you can still install lenny debs with it though. I don't rely
on anything apart from apt that would require squeeze.
>> > Good! (except that is going to break apt-cross really, really
>> > badly.) :-|
>>
>> *BOOOM* There really is no way around that with multiarch. The current
>> names plainly don't allow for multiple architectures.
>
> So let's get apt-cross out of the way and start the work on the
> replacement - under a new name, please.
>
>> > It is because apt thinks it is a library. apt-armel-cross doesn't
>> > contain apt itself, just the libapt-pkg library.
>>
>> Sure. I gathered that. But what depends on apt-armel-cross that you
>> need it? Is it just so you can cross compile aptitude and synaptic?
>
> There are other reverse dependencies but yes, those are two of them.
> Witness the list of binNMU's done after each upload of apt.
>
>> > The list tells you if libbar is multiarched or not. Reliably.
>> > dpkg-cross will then use that information when considering the
>> > Depends: line of libfoo.
>>
>> You say suite-specific but when installing libfoo_1.2-3_arm.deb what
>> suite does it belong too?
>
> Whichever suite the Packages files says it is - according to the
> version. The determination of suite precedes the download stage, so we
> only get 1.2-3 if 1.2-2 is in a different suite. At least, that's how
> apt-cross needs to do it.
I ment if you install it with dpkg-cross directly instead of through
apt-cross. The package might even be locally build and not be in any
suite.
>> But lets ignore this for now. If dpkg-cross
>> can reliable say when -cross is needed and when not then I'm happy to
>> use that same mechanism. Our only disagreement is wether dpkg-cross
>> can reliable say so and that we will see when it is used more.
>
> If we can replace apt-cross for everything except Lenny (or older)
> with a script that can cope with -cross packages for all, then the
> problem goes away. The issue is whether having those local packages
> around complicates long, long dependency chains.
A -cross package can depend on a arch:all package, a binary package
(arch: native), a multiarch library or a -cross package. And only a
-cross package can depend on a -cross package.
That means that in any dependency graph any non circular path can have
at most one transition from -cross to non-cross. And only a -cross
package depending on a multiarch library (libfoo-arm-cross depends on
libfoo:arm) makes the path longer than without -cross packages. So any
dependency chain is at most one longer than without -cross. The
overall graph though can be a tad bit larger. Theoretically the graph
is at most 3 times larger (every single package native, multiarch
foreign and -cross) but that will never happen.
Note: Having amd64 + arm + arm -cross packages is still less than
amd64 + i386 + kfreebsd-amd64 + kfreebsd-i386, wich is a real use case
of multiarch. So apt has to cope with enlarged dependency graphs in
the future. Poor us just has to find and fix any bugs that causes.
> e.g. You've got a full GNOME install on amd64. You need to cross-build
> a GPE package which depends on libgtk+2.0-0 which you also have
> installed for amd64. If, with apt-cross, libgpewidget1-armel-cross
> depends on libgtk+2.0-0{amd64} because gtk is not yet multiarched, then
> trying to upgrade amd64 gtk will cause the removal of
> lbgpewidget1-armel-cross AND all it's reverse dependencies because they
> are local packages and not in a repository. Now scale that up to one of
> the X11 libraries or something like directFB.
Worked for me with having gnome and a 32bit wine installed (which
needs gtk and a few more). So I'm not worried (yet :) about that. I've
never seen the kind of problems you describe.
> When we did have a cross-repository, those upgrade errors still took
> place for uncertain reasons, possibly related to how toolchains were
> built at the time.
>
> THAT is why I insist on not having -cross packages depending on
> non-multiarch packages that are not -cross - apt-cross just cannot
> help apt sort out the mess.
You really just have to try&see. This is a problem one can only
understand by having an example case that fails (or not :).
>> > I'm trying to see a clear dividing line between -cross - OLD, BAD
>> > and multiarch - NEW, GOOD! A dividing line that is consistent
>> > across package names, script names, paths, behaviour, the lot.
>> >
>> > It's going to confuse the hell out of me if people keep talking
>> > about -cross when talking about the transition away from -cross.
>> > -cross must die! (eventually)
>>
>> This is still all about the OLD, BAD -cross. It will go away with
>> multiarch.
>
> Yes, but it depends on parts of multiarch, so it isn't as BAD as
> apt-cross.
> :-)
Yeah. I've done the "apt-get -o Apt::Architecture=arm update" and then
move the downloaded files around dance in earlier versions. That is an
ugly mess and eats >300MB disk space as well.
>> > What is /usr/share/apt-cross/package.data about?
>>
>> It is an executable that can alter the package in any way neccessary
>> to make it functional as -cross package.
>
> shell or perl, presumably, being in /usr/share/ ?
I only done hooks as shell scripts. But they could be anything. They
could be provided by the packages themself as well, although by now I
would rather habe mai ntainers multiarchify their packages then add a
hook.
Should there be support for non script hooks? I would think shell is
enough with the option of perl if someone really needs to. If non
script hooks are to be allowed then /usr/lib/... would have to be
used.
>> It just replaces apt-cross until the multiarch transition is
>> finished. I expect that to still be a few years so we need a working
>> version.
>
> True, but it does seem to be at least one step *beyond* where apt-cross
> can take us and therefore, it should be a replacement.
It is hard to find a good name. apt-cross is not enough but
apt-multiarch promises too much. It isn't real multiarch as it still
uses -cross packages.
I'm open to suggestions for something inbetween.
MfG
Goswin
Reply to: