[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Emdebian Grip Baked (Lenny)

On Tue, 11 May 2010 09:00:06 +1000
"Brendan Simon (eTRIX)" <brendan.simon@etrix.com.au> wrote:

> On 10/05/10 9:48 PM, Neil Williams wrote:
> >>> List of packages passed to apt:
> >>> dash snmpd dropbear openssl dhttpd lighttpd apache2
> >>>       
> >> I think we also need:
> >>     ifupdown, net-tools, inetd, iproute, iptables
> >>     
> > Adding packages to the current repo is hard. (Harder than actually
> > preparing the initial set.)
> Oh.  That I didn't expect :(
> What is the reason for that ??  So, I guess my presumption that just
> running emgrip on a desired package to produce a baked packages is
> incorrect then ??

Your assumption is correct - IF you know, in advance, that you have all
the dependencies, including dependencies of dependencies.

The reason the current repo works is because apt got all the
dependencies of the packages, not just the packages themselves. 5
selections became 78 binaries.

> Is it something to do with generating the Release of Contents-<arch>.gz
> files ??

No, it's to do with /var/lib/dpkg/status.

The problems are:

1. Re-baking the same package at the same version is fatal because the
timestamps of the modified packages will always vary from one run to
the next and therefore the necessary archive integrity check will always
fail. Adding 5 packages to 78 means processing 98 packages (due to extra
dependencies) and then throwing away the 78. That could be scripted but
it's still a waste of processing and therefore time (once per arch).

2. apt does not understand how to calculate for a foreign architecture
based on available Architecture:all packages (or existing Architecture:
$this packages) *unless* a genuine /var/lib/dpkg/status file can be
created. Multiarch is fixing this but it's not in mainstream apt yet.

3. We have to change the version number because we're modifying the
packages, therefore we also change the dependency calculations to match.

4. apt cannot understand the /var/lib/dpkg/status from a baked machine
(prior to the deletion of the same) because it contains versions and
dependencies that do not exist in the Debian archive. This can be
handled, now that we have Grip, by pointing apt at Grip but that is
possible only because Grip has to cope with this issue internally

5. Solving these problems for grip means keeping a pristine partial
mirror updated at all times and then doing lots of calculations to
compare the two repositories. The process is not bug-free currently.
Running the calculations twice is not an option on the current server.

6. Without a valid and accurate /var/lib/dpkg/status, apt has no chance
of downloading only the new packages and the dependencies of the new

7. Dependency tools (like edos-debcheck) stop calculating when a
dependency is missing - therefore excluding the dependencies of the
missing package which could include more missing packages. The only way
around this is to create a memory map of the dependencies from the
Debian data and then apply that through a new mask which copes with the
version string changes, recursively - i.e. reimplement apt.

Therefore, the only sane way to add specific packages is to be
absolutely certain of all of the extra dependencies - including
dependencies of those dependencies, recursively all the way down to
libc6 - then wget the actual .deb files themselves and process them
through emgrip individually.

In practice, it is easier to dump the existing repository and start
again with the full list and let apt work from an
empty /var/lib/dpkg/status.

One possibility is to see if apt-grip can work with
www.emdebian.org/grip as the "incoming" mirror and
a /var/lib/dpkg/status file from a Baked machine - or, alternatively,
create a chroot (using multistrap) using Grip packages and work from
that etc.etc.

In summary, it is either "non-trivial" or a "Simple Matter Of
Programming" - take your pick. Either way, I don't have time to
continually update Baked *and* clean up Grip to be in a suitable state
for release alongside Squeeze. I choose to prioritise Grip.

> >> Are the rc*.d directories supported with Baked ??
> >>     
> > Not unless one of the packages already selected creates them or your
> > multistrap config scripts create them.
> >   
> OK.  So we may also need some of :
>     sysvinit, sysv-rc, sysvinit-utils, initscripts, insserv

Alternatively, you could simply create whichever files you need
in /etc/. That was the expectation with Baked. /etc/ == config, Baked
expects that you set your own config and cannot update files in /etc/
in normal Debian ways, ergo Baked expects files in /etc/ to be handled
manually. (Most packages don't alter /etc/ at runtime, at least, not
during normal user operations.) Baked could quite reasonably support
mounting /etc/ readonly.

> So, given it is hard to add packages to the baked repo, is it feasible
> to start afresh and add all grip packages ??

*This* is why I didn't particularly like the idea of offering a Baked
repository on www.emdebian.org - there is no sane default.

Everyone will want different packages added, everyone will want
different packages excluded (if you allow sysvinit to be in the
repository, everybody gets it by default - you have to use omitrequired
in multistrap and then work out how to get everything else you'll need).

Yes, it is hard to add packages to the existing repo, it is therefore
recommended that people wanting a different selection use the
principles that created the existing repo to create their own local
repo for just the architecture (and package list) that suits them.
Baked is a DIY distro, after all. Static in more ways than one.

The new page on Baked is a fair description of what I did to create the
current repo:



Neil Williams

Attachment: pgpAdE4Qkf3Xs.pgp
Description: PGP signature

Reply to: