[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Bug#616111: Bootstrapping a foreign architecture with multiarch



On Sat, 2 Apr 2011 05:47:47 +0200
Guillem Jover <guillem@debian.org> wrote:

Adding linaro people and debian-embedded to CC:

> Hi!
> 
> On Fri, 2011-04-01 at 22:31:07 +0100, Neil Williams wrote:
> > dpkg cannot be executed inside the chroot because it has not
> > necessarily been unpacked at this stage, it certainly has not been
> > configured. (dpkg is running from outside the chroot).
> 
> (An Essential package does not need to be configured to be functional.)

(It's quite possibly the wrong architecture inside the chroot, so
this doesn't apply.) 

> > Yet dpkg-query won't work with --admindir to operate from outside:

I'm checking on this - it could be an artefact of how the chroot was
initially created.

> > Anyways:
> > 
> > > Multi-Arch: same packages will use
> > > /var/lib/dpkg/info/<package>:<arch>.<something>
> > > 
> > > > For Multistrap, every file in /var/lib/dpkg/ has to be created by
> > > > multistrap, based on the data obtained from dpkg -e (basically the
> > > > DEBIAN/ content). This data needs only to be sufficient that dpkg can
> > > > correctly configure the packages once dpkg itself is able to be
> > > > executed inside the new filesystem.
> > > 
> > > It's sufficient, you just need to know the value of the Multi-Arch field.
> > 
> > True. I've got a change now which uses dpkg -f to check the Multi-Arch
> > field of the actual downloaded .deb (independent of arch) and use that
> > to detect Multi-Arch: same, then lookup Architecture in the same way,
> > add that to the filename of the created maintainer script etc.
> > for /path/to/chroot/var/lib/dpkg/info/*
> 
> If you are not going to install multiple instances of “Multi-Arch: same”
> packages, then just keep writting the files as you are currently
> doing, the next dpkg native run on that chroot image will
> automatically upgrade the db layout. Which seems to be the less
> onerous way to handle this. Otherwise you'll have to duplicate the
> db layout handling, including format file, etc, and at that point I'd
> say you are on your own.

Multistrap will need to be able to create chroots containing multiple
instances of Multi-Arch: same packages because it needs to be able to
build cross-building chroots with libc6-dev of the build and host
architectures, as well as other common dependencies like libglib2.0-0
(needed as build arch by pkg-config and as host arch by everything
else).

Multistrap, like debootstrap and other similar scripts, is commonly
used in two roles - to create a chroot of the build architecture for
building stuff in a contained environment and creating a chroot of a
foreign architecture to act as a root filesystem.

As with debootstrap, I see no reason to implement two different methods
to creating these chroots inside a single tool, so when creating a build
arch chroot multistrap uses the same processing as when creating a
foreign arch chroot. That processing has to avoid a design flaw in dpkg
where the normal --unpack method would attempt to run the preinst
scripts.

> On Fri, 2011-04-01 at 22:43:35 +0100, Neil Williams wrote:
> > Further to this, --control-path doesn't include 'list' but 
> > $pkg:$arch.list files are created.
> 
> This was done on purpose, .list and .conffiles are internal files, the
> correct interfaces to access the information is dpkg-query.

Internal files maybe, but multistrap still needs to create them to
handle packages of a foreign architecture.

Even once Multi-Arch is fully deployed, this will continue because the
assumption with Multi-Arch is that the foreign architecture packages
are being installed alongside the build architecture. When preparing a
root filesystem for a foreign architecture (e.g. an embedded device),
multistrap cannot put the amd64 binaries into the chroot and then add
the armel, it needs to be able to unpack armel binaries into an armel
chroot without running the preinst scripts or using the armel dpkg
binary inside the chroot. The preinst scripts can't run on the build
arch (amd64) because that would expose the chroot to the external
configuration. i.e. it needs to be possible to create an armel chroot
of Ubuntu natty from an i386 machine running Debian Wheezy and
similarly from Ubuntu natty i386 to create a Debian Wheezy armel
chroot. Currently, dpkg --unpack cannot cope with this and the existing
Multi-Arch changes in dpkg do not address this problem.

This has been covered before in this bug and elsewhere without
resolution, but please reconsider how dpkg - even in a Multi-Arch world
- needs to support unpacking foreign architecture binaries into a
foreign architecture chroot and that this mandates that dpkg *must not*
attempt to execute the preinst scripts inside the foreign chroot. It
can *only* do so if the packages are to be unpacked into the main,
external, system.

When using --admindir and --instdir options to --unpack, dpkg *must*
allow for the specified directories to actually be a foreign
architecture chroot and act accordingly. Maybe this requires the
addition of an --arch option to --unpack but it *does* need to be
addressed. The files being created in /var/lib/dpkg/ are not
architecture-dependent, it's only dpkg which needs to understand that
when creating a foreign architecture chroot, the external dpkg must not
execute the preinst scripts. That is all that is required.

Working out how to configure the resulting system (probably by running
the preinst scripts manually with the install argument prior to
running dpkg --configure -a) and whether to use an emulator or after
booting, is left to the admin of the device. dpkg doesn't need to care,
it simply needs to "do the right thing" when unpacking.

If this doesn't change, I will simply have to continue reimplementing
dpkg behaviour in multistrap and elsewhere, causing this inconsistency
to continue forever. That is not helpful. Let's solve this properly at
this ideal opportunity when Multi-Arch is still in final development.

I don't need to be "on my own" on this one (and I'm not anyway because
a lot of people need to have this behaviour from dpkg or from some
other tool which can hack around dpkg to avoid this design flaw).

This can't be swept under the carpet again, it needs to be addressed.
dpkg --unpack NEEDS to understand that when dealing with a foreign
architecture chroot it CANNOT execute the preinst scripts.

I will escalate this issue further if the dpkg team claim that this
functionality is not possible to implement. Now is the perfect time to
fix this long standing design flaw in dpkg. On the surface, it seems
a simple thing to setup: if an --arch option is used alongside
--admindir and --instdir to the --unpack command, then the preinst
scripts are never executed. It doesn't have to be an option called
--arch (although that is probably the easiest for embedded developers
to find in the dpkg man page), it could be "--no-preinst" or something.

It is just plain wrong that, even with Multi-Arch, dpkg cannot be used
to bootstrap a foreign architecture into a chroot and that people like
me have to invent a set of hacks to duplicate what dpkg should do
itself. I've done this enough times already.

-- 


Neil Williams
=============
http://www.linux.codehelp.co.uk/

Attachment: pgp7fbbF9BO1D.pgp
Description: PGP signature


Reply to: