Re: apt on handheld issues
On Sun, Mar 02, 2003 at 03:15:41PM -0500, Matt Zimmerman wrote:
> It may be possible to change apt to make the cache optional (since it is
> just that), though I imagine it will slow down processing greatly. The
> package lists are essential in order to get useful work done, though it
> should be possible to compress them. It might also be possible to go the
> other way, and keep the cache but remove the lists. However, I do not think
> that the cache can be compressed, as it is mmapped.
I have the impression that the cache is rebuilt very frequently anyway,
so I think it would be tolerable to get rid of it entirely. On flash
devices, writing data is very slow anyway, so it might even be faster.
How complicated would it be to get rid of the cache? I haven't looked at
the apt source code yet.
> The available file(s) can be eliminated almost completely without too much
> trouble:
>
> rm /var/lib/dpkg/available*; touch /var/lib/dpkg/available
>
> When new packages are installed, entries will be added to the available
> file, but this is only a few hundred bytes and can be cleaned out via a hook
> somewhere. apt does not use these files, but dselect and tasksel will
> break. This is OK with me.
Good idea, thanks. Is there any documentation somewhere about where these
files come from and who reads and writes them?
>
> > I've been able to mitigate it somewhat by removing the description
> > sections from the Package files, which approximately halves the space
> > needed by apt. But it's still a large chunk of storage.
>
> This is one thing that I do not want to do; the descriptions are one of the
> most useful pieces of information in the package lists.
I'm currently keeping the one-line description but deleting the rest of
the information at the moment. But if I use your trick of deleting the
available* files, it should be okay to keep them as they won't be stored
three times.
> This is essentially what the ipkg packaging system does, is it not? And
> some handheld distributions are starting to outgrow it. It becomes
> difficult to support smooth upgrades and complex packages when you do not
> have access to a complete dependency graph. And, as you say, we would lose
> the option of adapting an existing apt frontend for our use, rather than
> writing a new one from scratch.
Not really - the ipkg system is a complete reimplementation of the
packaging system that doesn't use dpkg, and the *.ipk files are not
compatible directly with *.deb files.
I was thinking about keeping the dpkg back-end and work with *.deb files
and the official archives, and just use a simpler script to build the
dependency graph and do the upgrades. But I'm not very keen on doing
that, since the complexity to do it *right* is very large.
> This is a serious problem, though. Not only would it be a significant
> amount of work to change apt to work without writable mmap, but it would end
> up using more memory (which is already short on a handheld).
The cache files are currently around 3-5 MB in size, I don't think it
would be a problem to keep that amount of information in memory,
especially because it could be freed before calling dpkg to do the
installation.
> One option that has been brought up is to move the problematic functionality
> to a server side program. A lightweight, dumb client would run on the
> handheld, and communicate with a smart client on a remote server.
[...]
> Then the lightweight client does not need to make difficult decisions about
> installation ordering, conflict resolution, etc., and does not need to store
> the entire package database locally. It basically only needs to be able to
> talk to the APT download methods and be a frontend to dpkg.
That would be an interesting approach for systems that are extremely
short on storage, i.e. if you don't have a SD/MMC card available and need
to fit everything in a 16MB flash ROM. But for what I'm doing, the space
is available to do everything locally, and I'd prefer to keep the
handheld independent of a server if at all possible. Unless there's a
permanent database available on the Internet, you'd otherwise run the
risk of being able to access the Debian package servers but not your
database.
-Klaus
Reply to: