[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Experiments with Crush - initial figures



On Fri, 2 Apr 2010 21:03:48 +0200
Hector Oron <hector.oron@gmail.com> wrote:

> > 3. Keeping the same source package name also means not having to
> > alter the filenames of original tarballs, making builds easier.
> >
> > 4. svn-buildpackage, svn-inject and svn-update used to try and
> > maintain the packages.
> >
> > So far, half the necessary packages have built and one (busybox) is
> > largely ready.
> 
> I have been also thinking this approach is best with current resources
> we have, but more like we do in toolchain packages which build-depend
> on *-source binary packages containing the source. So the question is
> if we could have a real gain if all Essential Debian packages could
> provide a *-source package (maybe those are unneeded).

The problem with that is Emdebian Crush users will each want different
configurations of things like busybox in particular, and then you've
got all the problems of rebuilding stuff against uclibc.

Some figures (amd64 tests):
partial multistrap config:

packages=busybox-crush debconf-crush cdebconf dpkg-crush gnupg-crush apt
apt source=http://localhost/crush
keyring=emdebian-archive-keyring
suite=sid

$ sudo du -hs /tmp/multistrap/crush
21M	/tmp/multistrap/crush

(Unpacked, not configured - an 8Mb tarball).
Note this is a specially modified multistrap that drops
packages from Grip that are "Priority: required" unless they are also
dependencies of the specified packages. This can mean that you get a
completely empty multistrap if you mess up the config.

$ du -k /tmp/multistrap/crush|sort -n
...
264	/tmp/multistrap/crush/usr/lib/gnupg
276	/tmp/multistrap/crush/usr/lib/cdebconf
352	/tmp/multistrap/crush/usr/lib/apt/methods
356	/tmp/multistrap/crush/usr/lib/apt
708	/tmp/multistrap/crush/sbin
844	/tmp/multistrap/crush/bin
1208	/tmp/multistrap/crush/var/lib/dpkg/info
1300	/tmp/multistrap/crush/var/lib/dpkg
1336	/tmp/multistrap/crush/var/lib
1364	/tmp/multistrap/crush/var
2976	/tmp/multistrap/crush/usr/bin
5028	/tmp/multistrap/crush/lib
6536	/tmp/multistrap/crush/usr/lib/gconv
9692	/tmp/multistrap/crush/usr/lib
12952	/tmp/multistrap/crush/usr
21032	/tmp/multistrap/crush

So, again, something needs to be decided about /usr/lib/gconv/ but
using uclibc would make a huge difference. (There's no kernel in the
above either.)

To avoid apt and gnupg, ensure you don't specify any keyring packages
and use the noauth option to multistrap too.

http://www.emdebian.org/packages/keys.html

As this test was a native multistrap, I can simulate removing apt and
gnupg:

$ sudo chroot crush/ /bin/sh
(note use of /bin/sh as busybox-crush doesn't yet give us /bin/bash)
# dpkg -P gnupg apt
# exit
$ sudo du -hs /tmp/multistrap/crush
18M	/tmp/multistrap/crush

Other problems then come into play. e.g. dbus cannot be added to this
rootfs because there's no adduser package yet.

The -crush packages are now in Emdebian SVN:

http://www.emdebian.org/svn/browser/current/target

http://www.emdebian.org/docs/svn.html

To build from a checkout, one or two hints first:

1. pushd ../tarballs ; apt-get source $package ; popd
2. svn-buildpackage -sa
You need to use -sa to be able to put these packages into any new
repository - except with native packages like debconf and dpkg.
3. Don't try using emdebuild on these yet but you should be able to
pass the -a$arch option to svn-buildpackage (which simply passes it
down to dpkg-buildpackage). I've just build dpkg-crush for armel that
way but svn-bp does get confused about the final .changes file. Don't
forget the -B option too if you've already built one arch, like native.
4. Ignore any lintian problems if you've got emdebian-buildsupport
installed.
5. dpkg-vendor support is pending - the effects of pushing these
packages through emgrip for Crush have been simulated in the figures
above. The packages you build from SVN will contain unwanted content
currently. (e.g. dpkg-crush still
contains /usr/share/perl5/Dpkg/Dpkg.pm)

So, this is about the smallest that standard Crush is going to be able
to get - small reductions of 1 to 2 Mb at most. It needs cdebconf and
dpkg to be able to work. Other gremlins include a need for update-rc.d
and the fact that dpkg-crush doesn't appear to show up in dpkg -l even
when installed.

# dpkg -s libc6|grep Installed-Size
Installed-Size: 9788
# dpkg -s libc-bin|grep Installed-Size
Installed-Size: 1320

# dpkg -s busybox-crush|grep Installed-Size
Installed-Size: 964

BTW, another gotcha:
dpkg: dependency problems prevent removal of libstdc++6:
 lzma depends on libstdc++6 (>= 4.2.1).

dpkg-crush_1.15.5.6em1_amd64.deb
--------------------------------
Pre-Depends: libc6 (>= 2.3.4), busybox-crush | coreutils (>= 5.93-1), lzma

So that's another problem - we can't drop lzma and remain compatible
with Debian packages, therefore we cannot drop libstdc++6.

# dpkg -s libstdc++6|grep Installed-Size
Installed-Size: 1268
# dpkg -s lzma|grep Installed-Size      
Installed-Size: 128

ii  busybox-crush  1:1.15.3-1em1  Tiny utilities for small and embedded system
ii  cdebconf       0.149em1       Debian Configuration Management System (C-im
ii  debconf-crush  1.5.30em1      Debian configuration management system
ii  gcc-4.4-base   4.4.3-5em1     The GNU Compiler Collection (base package) (
ii  libacl1        2.2.49-2em1    Access control list shared library (gripped)
ii  libattr1       1:2.4.44-1em1  Extended attribute shared library (gripped)
ii  libc-bin       2.10.2-6em1    Embedded GNU C Library: Binaries (gripped)
ii  libc6          2.10.2-6em1    Embedded GNU C Library: Shared libraries (gr
ii  libdebian-inst 0.69em1        Library of common debian-installer functions
ii  libgcc1        1:4.4.3-5em1   GCC support library (gripped)
ii  libnewt0.52    0.52.10-8em1   Not Erik's Windowing Toolkit - text mode win
ii  libselinux1    2.0.89-4em1    SELinux runtime shared libraries (gripped)
ii  libslang2      2.2.2-4em1     The S-Lang programming library - runtime ver
ii  libstdc++6     4.4.3-5em1     The GNU Standard C++ Library v3 (gripped)
ii  libtextwrap1   0.1-8em1       text-wrapping library with i18n - runtime (g
ii  lzma           4.43-14em1     Compression method of 7z format in 7-Zip pro

# dpkg -s libacl1|grep Installed-Size
Installed-Size: 56
# dpkg -s libattr1|grep Installed-Size 
Installed-Size: 72
# dpkg -s libdebian-installer4|grep Installed-Size
Installed-Size: 76
# dpkg -s libnewt0.52|grep Installed-Size         
Installed-Size: 128
# dpkg -s libselinux1|grep Installed-Size
Installed-Size: 200
# dpkg -s libslang2|grep Installed-Size  
Installed-Size: 1088
# dpkg -s libtextwrap1|grep Installed-Size
Installed-Size: 32

(cdebconf depends on libslang2)

-- 


Neil Williams
=============
http://www.data-freedom.org/
http://www.linux.codehelp.co.uk/
http://e-mail.is-not-s.ms/

Attachment: pgpugbkDqIoat.pgp
Description: PGP signature


Reply to: