Re: strap1: an alternative to debootstrap
Dear Neil, lists,
Neil Williams wrote:
On Mon, 22 Jun 2009 17:50:09 +0200
Pierre Pronchery <khorben@hackable1.org> wrote:
I have just written this paper about strap1, a component of hackable:1
(also described below). I think it can be of interest for both the
-embedded and -boot lists, my apologies otherwise.
Have you had a look at multistrap which is already in Debian?
(multistrap supports multiple repositories but does so for foreign
architectures as well as native.)
http://packages.debian.org/sid/multistrap
I didn't know about it, thank you. I knew about emsandbox though, but at
the time I couldn't use sid (and I still don't like to run sid, but
that's something else...)
strap1 addresses this exact problem, generating final, ready to run
software images from a number of Debian repositories.
Does it support foreign architectures? (Very important for Emdebian,
not necessarily for -boot.)
Yes.
strap1 should run as-is on any GNU/Linux distribution, and even on other
Unix systems (like BSD). It depends on these specific packages in
particular:
- binutils (for ar and strip)
then it looks like a native tool, not cross- or foreign (which would
need binutils-multiarch and other support).
Actually no, the actual executable used is redefined via the STRIP
variable within the profile scripts (eg STRIP=arm-linux-gnueabi-strip
for the Freerunner).
There is no official release of strap1 yet.
As above, multistrap is already in Debian. Feel free to compare the two.
I will :)
As packages are simply extracted into the staging directory before
building the actual image, they are configured with their default
settings, or sometimes not at all. Each package can be configured inside
the packages directory, within a shell script of the same name.
i.e. native with no foreign support. You cannot configure foreign
packages prior to creating the image on the build machine without using
an emulator.
Which I don't want to do for a number of reasons... Moreover, whatever
happens the major part of the configuration has to be done from within
strap1 anyway, because of the many different hardware platforms and use
cases to support.
Although multistrap doesn't explicitly do things like device nodes for
you, it does support adding in whatever scripts are needed to fully
customise such support. Clean-up is supported - specifically including
the clean up of the $dir/var/cache/apt/archives and $dir/var/lib/apt/
directories. Image creation (as with debootstrap) is left to other
scripts to achieve.
Making a complete image creation process as easily as possible is the
actual goal of strap1. It therefore also aims at integrating device
nodes definitions, whichever other scripts that might be necessary as
well, and the actual image generation process as well.
If it would use multistrap instead, it would then still consist of these
scripts. In any case, it could be interesting to see if they could be
shared between emdebian and strap1.
(device nodes are too specific to particular machines to be
particularly useful to encode directly within multistrap.)
strap1 currently calls the relevant "generic" target of the MAKEDEV
script when it's installed. It's true that this will typically have to
be decided from within the profile definitions. It can be done from
within the packages configuration scripts though.
Debian packages consist of an ar archive, with a meta-data archive
member, and an optional archive to decompress. The extraction is
performed without the dpkg tool, using the ar, gzip, bzip2 and tar
utilities as required.
That could be problematic. It is better to use the real dpkg, even if
you don't use apt.
As far as I know dpkg doesn't extract the meta-data. I've had trouble
with the output of "tar -xv" versus "dpkg -X", but nothing sed couldn't
fix so far.
4.2 Limitations of debootstrap
------------------------------
- it can use multiple source repositories at once
multistrap already does that. (I think you mean multiple binary
repositories, neither script actually builds binary packages.)
(indeed, good catch)
- it can configure packages cross-platform
I'm assuming you mean cross-distributions and not cross-architecture.
I mean cross-architecture.
Lack of usable --foreign support in debootstrap is a *huge* problem
for Emdebian and one that strap1 doesn't seem to resolve.
I think it does, albeit not in the most elegant way :/
Dependencies are not resolved recursively at the moment (this problem is
also found in debootstrap). The images generated may therefore be
incomplete in some cases.
That is a *major* flaw. multistrap is designed specifically with this
requirement in mind.
I see how I could fix it, but this might push the limitations of shell
scripting a bit far...
It depends on the use case but debootstrap can resolve dependencies
recursively. -boot may well know more about the detail but it is
possible to give debootstrap a fairly long list of packages and it will
find the missing dependencies. What debootstrap does *not* do is
proceed with the rest of the process only to end up with unresolved
dependencies - that would be a complete waste of time and effort.
I used debootstrap a lot, and have also read its code a few times, and
didn't catch this. I'll be interested by more details about how it does it.
Device nodes creation is not portable across Unix systems.
Then that may be better done as a separate stage, rather than claiming
that the script can do something that is not always relevant.
I think supporting "only" GNU/Linux is not so bad :)
It's just sad to be one step away from more portability... However,
integrating a separate stage should solve this indeed (decompression of
a custom archive?).
The stock Debian packages may not always fit in an embedded environment.
More cooperation with the Emdebian project is certainly desirable.
Have a look at emsandbox as well - that does the rootfs role (using
code modified from debootstrap) for the modified packages in Emdebian
Crush. multistrap is primarily for Emdebian Grip.
Ok, I'll try again. I can easily generate virtual machine images now :)
I would also like to add that we (h:1 developers) are cross-compiling
all of hackable:1-specific Debian packages. We have documented how we do
it here:
http://trac.hackable1.org/trac/wiki/CrossCompiler#Cross-buildingadebianpackage
Thanks for your work and comments!
Cheers,
--
khorben
Reply to: