Re: apt on handheld issues
On Sun, Mar 02, 2003 at 02:05:35PM +0100, Klaus Weidner wrote:
> The following "sources.list":
> deb http://ftp.debian.org/debian testing main contrib non-free
> deb http://nonUS.debian.org/debian-non-US testing/non-US main contrib non-free
>
> results in a rather large amount of disk space used by apt:
> 7680 /var/lib/apt/lists
> 6040 /var/cache/apt
> 13724 total
It may be possible to change apt to make the cache optional (since it is
just that), though I imagine it will slow down processing greatly. The
package lists are essential in order to get useful work done, though it
should be possible to compress them. It might also be possible to go the
other way, and keep the cache but remove the lists. However, I do not think
that the cache can be compressed, as it is mmapped.
> This is in addition to the "available" information saved by dpkg:
> 4312 /var/lib/dpkg/available
> 4312 /var/lib/dpkg/available-old
> 8624 total
The available file(s) can be eliminated almost completely without too much
trouble:
rm /var/lib/dpkg/available*; touch /var/lib/dpkg/available
When new packages are installed, entries will be added to the available
file, but this is only a few hundred bytes and can be cleaned out via a hook
somewhere. apt does not use these files, but dselect and tasksel will
break. This is OK with me.
> I've been able to mitigate it somewhat by removing the description
> sections from the Package files, which approximately halves the space
> needed by apt. But it's still a large chunk of storage.
This is one thing that I do not want to do; the descriptions are one of the
most useful pieces of information in the package lists.
> One approach I've been thinking about is writing an alternative "apt-get"
> implementation in Perl that supports the basic operations necessary
> (update, dist-upgrade, install, remove) and works directly with gzipped
> package files, without a separate database. Since my simple
> shell+egrep-based solution in the "chroot-debian" bootstrapper worked
> surprisingly well, I think that would be a reasonable approach, especially
> if you don't try to support complexities such as pinning. But that would
> break any frontends that use apt library functions, since I'd only be
> supporting the command line interface.
This is essentially what the ipkg packaging system does, is it not? And
some handheld distributions are starting to outgrow it. It becomes
difficult to support smooth upgrades and complex packages when you do not
have access to a complete dependency graph. And, as you say, we would lose
the option of adapting an existing apt frontend for our use, rather than
writing a new one from scratch.
> Another issue is that it's impossible to store the apt database on a jffs2
> filesystem, since that filesystem doesn't support the mmap operations
> (shared writable?) apt needs. That's unfortunate since the data would
> really benefit from filesystem-level compression.
This is a serious problem, though. Not only would it be a significant
amount of work to change apt to work without writable mmap, but it would end
up using more memory (which is already short on a handheld).
One option that has been brought up is to move the problematic functionality
to a server side program. A lightweight, dumb client would run on the
handheld, and communicate with a smart client on a remote server.
For example:
Client->Server What packages are available?
Server->Client <handheld-tailored package list>
Client->Server Here is my state (dpkg status)
Client->Server I want to install 'hello'
Server->Client Fetch these URLs:
'http://debian/pool/main/h/hello/hello_2.1.1-1_i386.deb' hello_2.1.1-1_i386.deb 47900 e193bf7e7555a8bb62b28a91b410e543
'http://debian/pool/main/g/glibc/libc6_2.3.1-14_i386.deb' libc6_2.3.1-14_i386.deb 3192764 a555b45a42df272467c3a7afc4536ab9
Server->Client Execute this series of actions:
Inst libc6 [2.3.1-14] (2.3.1-14 Debian:unstable)
Conf libc6 (2.3.1-14 Debian:unstable)
Inst hello (2.1.1-1 Debian:unstable)
Conf hello (2.1.1-1 Debian:unstable)
Then the lightweight client does not need to make difficult decisions about
installation ordering, conflict resolution, etc., and does not need to store
the entire package database locally. It basically only needs to be able to
talk to the APT download methods and be a frontend to dpkg.
This is no small amount of work, but I think that it would suit our needs
nicely, and have broader usefulness as well.
--
- mdz
Reply to: