multistrap on ubuntu
I've been making multistrap work on ubuntu for the last couple of days
(in order to use pdebuild-cross).
I found a few issues worth noting, and some that merit a bit of discussion
so here it all is in one big dollop.
1) There were no configs for maverick or ubuntu cross-building-chroots
2) Multistrap does not deal correctly with 'flat' source URLs, as
produced by apt-ftparchive.
3) Multistrap does not try to stop packages running services in
postinsts.
4) Multistrap specifies the wrong path for the sources list when doing
the initial package install if aptsources is specified. This doesn't
actually seem to matter, but is confusing.
5) There is a bug that multistrap tries to install packages listed in
config stanzas which as not referenced in debootstrap=
Discussion and details:
1) Configs
I've created configs for maverick and lucid. The main difference is
that Ubuntu has a different URL for the 'ports' architectures
(armel, sparc, ia64, powerpc), than for the 'normal' achitectures.
Cross-Toolchains for maverick currently come from an aptftparchive
repo which is why I had to fix 2. This will go away soon as a
requirement for cross-build chroots as the packages will be in
maverick proper. But it's useful that it made me fix 2
I'm not sure quite how many configs we should carry and how they
should be cascaded and named. e.g. we could have 'armel.conf' which
changes between debian and ubuntu versions to point to relevant distro
stuff. That would be easiest from the users POV, but it's dull having
an Ubuntu patch for them to carry. We could keep armel.conf for Debian
and have armel-maverick.conf for maverick etc. Similarly for
crosschroot.conf and so on.
2) Flat URLs
My test case is:
deb http://people.canonical.com/~hrw/ubuntu-maverick-armel-cross-compilers ./
Setting
source=http://people.canonical.com/~hrw/ubuntu-maverick-armel-cross-compilers
suite=./
components=
does not work, because then multistrap tries to qualify the specified
packages using the suite "./" and then fails to find e.g.
g++-4.4-arm-linux-gnueabi/ with a "Release '' not found" whinge.
So I fixed up the code such that you can specify
source=http://people.canonical.com/~hrw/ubuntu-maverick-armel-cross-compilers ./
and multstrap ensures that suite and component are not set (for any source
ending in a path with a trailing slash). The regexp could be made more
paranoid or possibly there is already some perl code for parsing such deb lines?
The problem with this is that nothing now ensures that the specified packages
actually come from the specified repository, even if explicitsuites is set.
The package/suite syntax cannot be used so I think that specifying package
pinning would be the only way to do it. Something to investigate one day, but
it's not a problem currently.
Here is my patch. It could probably be improved, as my perl is weak (maybe suite should be set to something other than "")?
--- multistrap.orig 2010-08-06 02:07:04.000000000 +0100
+++ multistrap 2010-08-19 17:49:33.000000000 +0100
@@ -243,9 +243,15 @@
open (SOURCES, ">>${dir}etc/apt/sources.list.d/multistrap-${aptsrc}.list")
or die _g("Cannot open sources list"). $!;
$mirror = $sources{$aptsrc};
- $suite = $suites{$aptsrc};
- $component = (defined $components{$aptsrc}) ? $components{$aptsrc} : "main";
- if (defined $mirror and defined $suite) {
+ # don't set suite or component if URL is of apt-ftparchive trailing-slash form
+ if (($mirror =~ /.* .*\/$/)) {
+ $suites{$aptsrc}=$suite="";
+ $components{$aptsrc}=$component="";
+ } else {
+ $suite = $suites{$aptsrc};
+ $component = (defined $components{$aptsrc}) ? $components{$aptsrc} : "main";
+ }
+ if (defined $mirror and (defined $suite or ($mirror =~ /.* .*\/$/))) {
print SOURCES "deb $mirror $suite $component\n";
print SOURCES "deb-src $mirror $suite $component\n" if (not defined $omitdebsrc{$aptsrc});
close SOURCES;
@@ -305,10 +316,10 @@
my @long=split (/ /, $packages{$sect});
foreach my $l (@long) {
chomp ($l);
if (defined $explicit_suite) {
# instruct apt to get packages from the specified
# suites (when the package exists in more than one).
- $str .= " $l/$suites{$sect}" if ((defined $l) and ($l !~ /^\s*$/));
+ $str .= " $l/$suites{$sect}" if ((defined $l) and ($l !~ /^\s*$/) and ($suites{$sect} ne ""));
} else {
$str .= " $l" if ((defined $l) and ($l !~ /^\s*$
}
3) Nobbling services
This prevents a basic maverick chroot being multstrapped, as procps
fails when it can't find upstart. In fact it's important to stop apps
starting services, as higher-level services will result in things like
apps in the chroot being bound to socket on the base system.
I fixed this the same way that debootstrap deals with it - by
replacing initctl and start-stop-daemon with little shell script that
say 'in chroot, doing nothing'. This is implmented in a script
chroot.sh which is called using hte setupscript mechanism.
however for cross-chroots we already call setcrossarch.sh.
I'm not sure how the config cascading stuff works with multiple
setupscripts options - do you get the first one, the last one, all of
them? (in what order?). So I decided that it was alwyas correct for
setcrossarch.sh to call chroot.sh which does the initctl nobbling.
(because cross-build chroots are always hosted chroots not real
filesystems.
For real filesystems we need to do something a bit more sophisticated
- temporaily divert the files during the dpkg configure stage, but
then put them back again for when the system is booted on real
hardware. Aparently debootstrap has a little package that it installs
and then removes to accomplish this. Shall I file a bug about this to
keep it in mind?
here is what the new chroot.sh script does:
TARGET=$1
if [ -x "$TARGET/sbin/initctl" ]; then
mv "$TARGET/sbin/start-stop-daemon" "$TARGET/sbin/start-stop-daemon.REAL"
echo \
"#!/bin/sh
echo
echo echo \"Warning: Fake start-stop-daemon called, doing nothing\"" > "$TARGET/sbin/start-stop-daemon"
chmod 755 "$TARGET/sbin/start-stop-daemon"
fi
if [ -x "$TARGET/sbin/initctl" ]; then
mv "$TARGET/sbin/initctl" "$TARGET/sbin/initctl.REAL"
echo \
"#!/bin/sh
echo
echo \"Warning: Fake initctl called, doing nothing\"" > "$TARGET/sbin/initctl"
chmod 755 "$TARGET/sbin/initctl"
fi
There may be more that should be done to safely install stuff in
chroots?
And in fact I wonder if these scripts should go in a scripts dir under
/usr/share/multistrap/ just to stop mixing the scripts and the
configs? maybe it doesn't matter and shorter paths is good?
4) multistrap distinguishes between aptsources being set or just using the host sources files.
if using the host then the list is put in ${dir}etc/apt/sources.list.d/multistrap.sources.list
otherwise it is put in ${dir}etc/apt/sources.list.d/multistrap-${aptsrc}.list
But the apt call always inlcudes -o Dir::Etc::SourceList=${dir}${etcdir}sources.list.d/multistrap.sources.list
even when it doesn't exist. It still finds the .list files we did create in ${dir}${etcdir}sources.list.d/ but the output
prints that apt is using this wrong path, which certainly confused me for a bit.
I patched it a bit to set SourceParts or Sourcelist depending which 'mode' it is in, but setting SourceList just results in a
long list of 'duplicate sources list' complaints, so clearly it is already finding them. I don't quite understand why.
so the patch is now:
-$sourcesname = "sources.list.d/multistrap.sources.list";
-$config_str .= " -o Dir::Etc::SourceList=${dir}${etcdir}$sourcesname";
+if (defined $deflist) {
+ $sourcesname = "sources.list.d/multistrap.sources.list";
+ $config_str .= " -o Dir::Etc::SourceList=${dir}${etcdir}$sourcesname";
+} # else {
+# $sourcesname = "sources.list.d";
+# $config_str .= " -o Dir::Etc::SourceParts=${dir}${etcdir}$sourcesname";
+# }
which at least stops it printing confusing paths.
BTW the apt docs mention Dir::Etc::SourceList but don't mention Dir::Etc::SourceParts at all. I'll put in a doc-patch for that.
5) Installing packages in un-specified stanzas
With this config file:
General]
include=/usr/share/multistrap/crosschroot-maverick.conf
omitrequired=false
configscript=
setupscript=/usr/share/multistrap/setcrossarch.sh
debootstrap=Ubuntu MarcinToolchains
aptsources=Ubuntu Ubuntuports MarcinToolchains
tarballname=pdebuild-cross.tgz
noauth=true
[Toolchains]
packages=g++-4.2-arm-linux-gnueabi linux-libc-dev-armel-cross
reinstall=binutils-multiarch
source=http://www.emdebian.org/debian
keyring=emdebian-archive-keyring
suite=stable
[MarcinToolchains]
packages=g++-4.4-arm-linux-gnueabi linux-libc-dev-armel-cross
reinstall=binutils-multiarch
source=http://people.canonical.com/~hrw/ubuntu-maverick-armel-cross-compilers ./
the initial apt install will try to install both the g++-4.4-arm-linux-gnueabi
we asked for and also the g++-4.2-arm-linux-gnueabi we didn't. Exactly why is
beyond my perl-fu to work out. Something to do with the cascading?
This problem means you can't have one config file listing severl possible toochain sources and then
select one. (well, you can, but you have to explicitly comment-out the unwanted ones)
I'm hoping Codehelp can sort that one out.
If there are no violent objections I'll check this lot in.
Once all this was done pdebuild-cross 'just worked'. I haven't done much testing yet, but it looks promising.
Wookey
--
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/
Reply to: