[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Reducing apt's memory footprint (on small boxes)



Hello,

this story isn't new: Older boxes with rather small memory. In my case,
DockStar with 128 Mbyte RAM. They still serve a job as e.g. a router,
but that limitation becomes more and more a problem. The biggest issue,
at least for me, is apt although it's just the bringer of the bad news:
The packages indexes became that big they no longer fit into memory. In
September 2015, there was a suggestion to create subsets of a release,
but I objected it will be more or less impossible to create them without
breaking some builds or installations for unresolved dependencies.

Time has passed, the problem became worse, and my dockstars are still on
stretch for that very problem - after a buster upgrade, they would
often crash during an automated "apt update".

However, I figured the above dependency problem isn't one I really have
to care about. If there is an unresolvable dependency, I still could
switch to the full packages indexes if I really have to. Also, it's only
about installing packages, building happens on machines with sufficient
ressources.

And so I started a small project:

* Take the list of binary packages that are actually installed, less
  than 500 in my case.
* Have copies of `dists/stretch/main/binary-{all,armel}/Packages`.
* Drop all stanzas that are *not* in the above list.
* gzip, xz. Create a Release file and sign it.
* Place the file structure in an appropriate webroot.
* Have a rewrite rule, so /pool/(.*) is redirected to the next Debian
  mirror.

As a result: Execution time of "apt update" dropped from 52 seconds to
some 12, and the memory pressure is gone.

Next steps were to repeat that for buster, do a dist-upgrade, and
observe how the box makes it through the next hours. Looking good so
far, therefore I'll keep maintaining this, at least for myself.

Now, from my experience: Every good idea that I believe to have found
turned out had been prosoped earlier by somebody else and/or was not as
good as it initially seemed. So, where's the actual catch in that setup?
Or is that something that might be of broader interest and possibly even
become part of a Debian release?

Of course there is the problem of defining which packages should be
included in such a "debian-narrowed"¹. I have some ideas about this but
it's a bit too early to share them. Still, even if the list contains
2000 binary packages, it should have everything for the tasks such a box
would do while still being *way* smaller than the ~75k that I see in
stretch currently,

    Christoph

¹ Initially, I was thinking of "narrow-gauge" but dropped that because
  it abbreviates to "ng" ...

Attachment: signature.asc
Description: PGP signature


Reply to: