[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: multistrap rootfs with initramfs



On Mon, 02 Nov 2009 23:59:07 +1100
"Brendan Simon (eTRIX)" <brendan.simon@etrix.com.au> wrote:

> > I would be good for someone to work out what the actual bug is. A
> > bugrep pointing out that there is an issue would be a helpful place to
> > hang such info.
> >   
> I could have a go at a bug report for dpkg.  Do I need to register as a
> Debian developer or have a Debian account ??

No, you should use a Debian machine and run 'reportbug dpkg'. The issue
is that you will need to provide a test case that shows the bug within
a normal Debian setup. I've been unable to reproduce that. If you file
the bug as it stands, it is likely to be dismissed as "unreproducible".
dpkg is a fundamental part of Debian and tends to get a lot of bugs
reported - sometimes triage can be quite aggressive.

The process of identifying the test case could well be sufficient to
show what is actually going on - depends if you can use a debugger on
dpkg on the device.

> For /etc/fstab I have the following (from a vendor SDK).
> They all seem pretty standard -- not sure about the /dev/nfs line though.

/dev/nfs would not be standard AFAICT.
 
> # UNCONFIGURED FSTAB FOR BASE SYSTEM
> none      /proc   proc    defaults 0 0
> none      /sys   sysfs    defaults 0 0

Those are about all I can see as "generic", possibly tmp. There's
nothing to say that all multistrap devices will want /var/* in tmpfs.

The problem there is that having a file at all makes it harder to know
if it is safe to update that file with more specific content later on.
It's a much simpler test to say that no file exists rather than to
try to "patch" an existing file and/or detect whether you need to patch
it at all.

If multistrap puts any files like this into the filesystem, *someone*
is going to need to adapt those files later on and when we tried this
with balloon3-config and grip-config, it became incredibly hard to be
*sure* that you're doing TheRightThing by replacing the existing file
with the new one. An unreliable update is much more problematic than
no update.

As far as /init goes, it doesn't even need an /etc/fstab file - it
simply needs to know how to mount and then to go on and mount what it
needs.

> /dev/nfs        /               nfs     defaults 0 0
> none            /tmp            tmpfs   defaults 0 0
> none            /var/run        tmpfs   defaults 0 0
> none            /var/lock       tmpfs   defaults 0 0
> none            /var/tmp        tmpfs   defaults 0 0
> none            /media          tmpfs   defaults 0 0
> 
> 
> For /dev, I used a control file in the linux config -- as this was
> already available as part of the vendor SDK.

An external file/script is going to be needed for nearly all multistrap
/dev uses.

> How about having some options in the multistrap config file ??
> examples:
> 
> [/etc/fstab] or [fstab]
> <add entries here>
> 
> or something more generic like:
> [file /etc/fstab uid gid perm]
> <some content>
> 
> or include an external file.
> [file /etc/fstab uid gid perm =
> /my/local/fs/home/brendan/emdebian/config/etc/fstab]

As mentioned in the other thread, I'm not convinced that multistrap
should get *that* involved in post-unpacking configuration. About all
it can do is call a hook or (more commonly IMHO) expect the next
process in the loop to do the work (e.g. when called via make).

> Maybe this is too much the multistrap config file, and another
> config/control (or tool) is a better option.
> If you like an "em_ms_secondstage" or "em_ms_setup"

Yes, a second handler is likely. The _ms_ abbreviation is a touch
unfortunate and we already have an emsecondstage for use with Crush and
emsandbox. I don't want to confuse things even more by reusing it
directly. The guts of emsecondstage are more or less what we need but
adapting it to multistrap instead of debootstrap will more than double
the size of the script. Might as well do a new one.

Multistrap needs this kind of post-unpack config even in native mode
(although it's a lot easier native), so there are two requirements:

1. In cross-mode, the handler needs to know to set the environment
variables and call dpkg --configure -a before doing anything else.
multistrap itself can easily be adapted to enable this part of the
setup, using a file in /etc/ or (my preferred option) a file
in /usr/share that is copied into place as /init or /etc/init.d/.

2. In native mode, it won't hurt if dpkg --configure -a is run anyway
but what we really need is a run-on-first-boot-but-never-again init.d
script. Can we have an init.d script that deletes (or disables) itself
upon success?

3. We need to know whether the boot gets as far as running stuff
in /etc/init.d/ or whether this needs to be done at an earlier stage
of /init.

As such, it isn't necessary to retain the em_ prefix - this is just as
likely to be necessary with /usr/sbin/multistrap.

debootstrap copies itself into the filesystem which isn't particularly
clean, but it's an approach that underpins emsecondstage too. Whether
that survives the changes in Crush 3.0 is uncertain.

Does it have to be a script that you call yourself? initramfs-tools
simply takes a script from /usr/share and makes that /init inside the
initramfs.

/usr/share/initramfs-tools/init

...
mkdir -p /var/lock
mount -t sysfs -o nodev,noexec,nosuid none /sys 
mount -t proc -o nodev,noexec,nosuid none /proc 
# perfect place to see if dpkg --configure -a is usable
# set the environment, run dpkg and unset the environment

> An "em_ms_secondstage" or "em_ms_config" script could be part of the
> rootfs, that ran 'dpkg --configure -a', but after it did some sanity
> checks for various filesystems such as /proc, /sys,  ...  It could even
> mount them if they are not already mounted :)

It would have to in order to run dpkg --configure -a - as emsecondstage
does already.

However, /init has to do those checks (and lots of others besides), so
why not take advantage of that?

In effect, the initramfs packaged into the cpio *is not* complete but
the /init script inside the initramfs takes on the role of completing
the installation at boot. The fact that it only has anything
significant to do on the first boot shouldn't matter.

Whatever scripts you have calling multistrap can then do some work on
the files in /etc/ and then put the modified /init into place.

The idea of doing this via a package has been tried several times - the
results (IMHO) are:

1. Resist the temptation to put files into the package itself - these
would be removed if the package is removed (there is no Essential: in
Grip) and cause other problems on package upgrades (by being removed
and then reinstated). Instead, create files in the postinst where you
can check whether something already exists or not.

2. Ensure that the package is idempotent so that it doesn't
clobber existing files. Ensure that created files are not removed in the
prerm or postrm - yes, this breaks Policy but the thing about
multistrap is that these packages do not need to go anywhere near the
main Debian archive.

http://release.debian.org/lenny/goals.txt
# piuparts-clean archive
  Advocate: Marc Brockschmidt and Luk Claes
  Description: All packages should pass the install stable version,
upgrade to unstable version, purge piuparts test.

(The whole point is that the files created by these kind of packages
must fail this test - one way to handle that would be to prepare the
package(s) as a udeb which is close to the truth.) (Wookey: I think
balloon3-config should be a udeb too, thinking about it.)

3. To be sure, see if the package can be removed or disabled once
configured - possibly by having a higher version of the package in
another archive that is configured as the main apt-source for the
device and which has none of the postinst file-creation code. The
"dangerous" version could then be in a "private" repository used only
by multistrap. If a udeb is used, [3] goes away completely.

4. If a package is used, some method is required to ensure that it is
configured before other packages are configured. It can only rely on
a basic POSIX shell, mount and dpkg being available, at least
initially. dpkg --configure config-udeb prior to dpkg --configure -a
would work. Again, this would be a job for /init.

-- 


Neil Williams
=============
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/

Attachment: pgphNpKW8D3Cm.pgp
Description: PGP signature


Reply to: