Re: Emdebian
On 2006-11-07 08:49 +0000, Neil Williams wrote:
> On 07/11/06 02:19:07, Wookey wrote:
> >On 2006-11-06 20:52 +0000, Neil Williams wrote:
> >
> > I am not really managing to read all this lot in enough detail to
> >make useful comments. I have rather too much work to do (as well as
> >trying to move house).
> >I hope that's not a problem, but as neither of you have been involved
> >in the years of discussion and thought that has got us this far I
> >worry slightly that you will head of in a 'wrong' direction. Perhpas
> >you will simply be freed of earlier prejudice :-)
>
> This has all been a bit "headlong" since Expo. Too fast is just as bad
> as too slow. Time to see if/how all these ideas actually work and then
> we'll have some test code and test packages.
>
> >> We'll need a customised wrapper around dpkg-buildpackage no matter
> >>what else is done - this can be just one part of that wrapper.
> >>Even if the wrapper only calls emlocale [1] and dumps
> >>./usr/share/doc/* to /dev/null it'll need to handle debian/* data
> >>differently to Debian itself.
> > Wasn't the whole point of Julian's suggestions that once we had
> >constructed a suitable new source package with the emdebian diff,
> >that the tools wouldn't need to change at all? I like that concept.
>
> Let me redo that. When converting a Debian package to an Emdebian
> package - i.e. when creating the emdebian diff.gz - we need to call
> emlocale and dump the doc build code. At each Debian upstream release,
> emlocale would be run again (to catch new translations) and there'd
> need to be a check that no new docs have been added or new doc build
> rules added.
>
> >> Once that is done, 'apt-cross --install gpe-contacts' will be a
> >>usable command. That opens the door to using the toolchain created
> >>(and maintained) via emchain in a chroot
> > Why do you want to install a cross-toolchain into a chroot to build
> >things? I had always envisioned doing it in your normal rootfs
> >(unless building for a different target suite to the host).
>
> That's certainly how dpkg-cross does it.
Oh, you mean the hierarchy it fills under /usr/arm-linux-gnu ?
That's not what I think of as a chroot. It's just a place in the
normal root to put libs, includes etc which represent the target build
environment. Obviously it is correct for a cross-build compiler to
reference those.
> I would like some kind of
> pbuilder functionality if we're altering the Build-Depends in Debian
> packages.
Clearly building with pbuild or sbuild is a good idea for the same
reasons that it is with debian packages. It should be
the way the buildds operate, but I don't want to make it a necessity.
> The other problem is that packages installed by dpkg-cross
> onto the host system would need to be removed at the end of the build
> because they hold back the equivalent host package when doing apt-get
> upgrade.
Hmm - you mean they have en explicit dependency on that 'conventional'
package? I've never noticed dpkg-cross packages getting in my way like
this, so I don;t actually think you are right. I think they only
depend on other -cross packages, and thus are entirely independent. In
which case this isn't an issue.
> The alternative is a hook in cron-apt that does what emchain will do
> for gcc, binutils and libc: removes them if new packages are available
> for the host, installs and configures the host packages, rebuilds the
> cross built packages and installs them on the host.
I think you are getting a bit carried away there.
> >>changes file and the packages themselves. (In most cases, a single
> >>Debian package is likely to result in several Emdebian packages
> >>that - combined - still occupy less space than the one original
> >>Debian package.)
> >> 307250 2006-10-03 18:52 libglib-2.0-0_2.12.3-r0_arm.ipk
> >> 547606 2006-10-02 14:47 libglib2.0-0_2.12.4-1_amd64.deb
> >> 275214 2006-10-02 12:47 libglib2.0-data_2.12.4-1_all.deb
> >>
> >> Even with 76 translations (~6000 bytes each in .ipk form), the
> >>repository is at least 56kb better off and the user some 240kb
> >>better off (without counting libglib-2.0-doc {717892 bytes}). The
> >>.ipk will serve as a useful benchmark of how the .emdeb build
> >>performs - it should be very similar.
>
> >Can you show me that again - how come splitting into loads of locale
> >packages ends up with less space taken up than the original. Do you
> >mean that the extra locale overhead is less than the size of the docs
> >and examples we missed out?
>
> Yes - at the repository the difference can be relatively small. The
> *big* gain comes for the user. By specifying their locale, as normal,
> in the installation, only a few (or just one) translation package needs
> to be installed per binary package. This compares to Debian where if a
> package contains dozens translations, the user has no real option but
> to install all of them.
[there is a package which installs an apt-hook to throw away the ones
you didn't wnat, but this is just an aside]
> Packages like glib have >70 translations, so
> that's an instant saving of >60 files on the target system. Glib2
> translations are quite small, larger applications may have translations
> of 300kb EACH! Imagine installing 60 unwanted 300kb files *for each*
> large application! With just a dozen Debian apps, we could lose 20Mb
> just in unnecessary files in /usr/share/locale/* on the target. My iPAQ
> only has 30Mb all in!
> :-)
>
> A virtual locale package would be needed for each language so that all
> translations for that language can be installed as a set.
>
> The individual locale packages contain only the foo.mo file.
>
> On this Debian system, /usr/share/locale/ takes up 288Mb.
> /usr/share/locale/en_GB/ takes up 4.6Mb, .../de/ and .../fr/ take up
> closer to 9Mb each. That's *all* available translations for a full
> Gnome environment in under 10Mb, not almost 300.
Agreed. The advantages on the target are enormous. I'd like debian to
have something like this too. Maybe we should try and push this idea
upstream...
> The user can still specify more than one locale so that the locale can
> be changed at login time. It's just that *only* the translation files
> for specific locales get installed on the target.
>
> Equally, converting a device installed with en_GB as the only locale
> into a device capable of supporting de or pt_BR just means installing
> that one set of language files (and optionally removing the originals).
No argument with any of this but if you add pt_BR later does that mean
you get pt_BR files for packages installed _from now on_ or does it go
back and get the laguage files for all the packages that are currently
installed too? The second is nicer but I don't think a virtual locale
package gets us that for free - we'd need some scriptage somewhere. I
don't think it's too hard. Something to think about.
> The filespace savings of using emlocale make omitting the copyright
> data look small, principally because no matter how long
> debian/copyright becomes, there is only ever one per source package.
> Translations are nearly always multiple.
>
> >> Emdebian will need a revised Policy to cope with the removal of
> >>files that lintian would consider essential (like debian/copyright)
> >>- even the changelog won't appear in the binaries. These changes
> >>are unavoidable to get Debian down from 3-5Gb to <20Mb. Taken as a
> >>set, the Emdebian Policy (when it's written) will need some form
> >>of emlintian.
> >'Need' is perhaps a bit strong. We can do without it for quite some
> >time, but clearly it would help a great deal.
>
> :-)
>
> >I have wwondered about
> >the copyright file. It is nice and simple to just forget about it (in
> >the binaries), but there are probably licences where we really do
> >need to ship some sort of statement with the binaries. And some sort
> >of metadata on what licences areon the system would be good. Just a
> >one-line thing saying 'GPL2' or whatever would be better than
> >nothing.
>
> If we expand the "common-licences" to include the FDL and one or two
> others, it would cut this down but the (long) lists of copyright
> holders would be needed if we are going to install copyright data onto
> the target.
>
> >But many copyright files are clearly excessive. Something to think
> >about that. Can we easily do something better than simply missing it
> >out? Would missing it out get us into and awkward legal trouble?
>
> No more than missing out the ChangeLog - licences like the GPL require
> that changes are identified.
>
> Time to ask the openembedded people - my iPAQ has licences in
> /usr/share/common-licences/ (Artistic, GPL-2, LGPL-2.1, BSD, LGPL-2)
> but no copyright information under /usr/share/foo* - just pixmaps and
> other arch-independent program files. Most GPE applications don't have
> About.. boxes either.
>
> Each bitbake recipe (their build/packaging instruction set) contains a
> LICENCE field and a MAINTAINER field, but no copyright data that I can
> see.
Yes. This does seem to be standard proceedure. We'll do that for now
but don't tell debian-legal just yet :-) Keeping the copyright info with
the source seems fair enough to me.
Wookey
--
Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK Tel +44 (0) 1223 811679
work: http://www.aleph1.co.uk/ play: http://wookware.org/
Reply to: