Bug#1012173: apt: Locking timeout for archives
On Tue, May 31, 2022 at 1:39 PM Julian Andres Klode <jak@debian.org> wrote:
>
> On Tue, May 31, 2022 at 12:58:05PM +0200, Jouke Witteveen wrote:
> > Package: apt
> > Version: 2.0.8
> > Severity: minor
> >
> > Dear Maintainer,
> >
> > In our setup, we share an archives cache directory [Dir::Cache::archives]
> > between multiple (virtual) machines. This turns out to be an effective
> > way to save bandwidth and disk space. However, each machine may lock the
> > archives directory and make apt unavailable for the other machines. We had
> > hoped that setting a lock timeout [DPkg::Lock::Timeout] would allow the
> > other machines to wait on the one holding onto the lock. Unfortunately,
> > that configuration parameter only applies to the 'admin directory'
> > containing the status file. As a result, apt operations fail immediately
> > when another machine holds a lock on the archives directory.
>
> I do not consider sharing archives between machines to be a valid use
> case. It breaks the assumptions of both the apt(8) model of cleaning up
> after yourself as well as the opposite historical model of autoclean
> (which removes packages no longer available from sources.list).
I see how this could be annoying in our use case. We would need to set
a lot of configuration parameters to reliably prevent cleaning the
archives and I agree that this clearly runs against the assumptions of
apt and autoclean. Thank you for these pointers!
> My suggestion for caching is to use a central squid-deb-proxy.
This looks like a reasonable solution. It is slightly more involved
than just mounting the same archives volume in all the containers, but
still reasonably simple. The biggest downside is that packages are
still copied into the local archives on the containers, thus wasting
disk space. The default docker apt configuration does clean the copies
after use, but I feel our initial naive solution of mounting the same
archives directory more cleanly circumvented the whole issue.
>
> > We can think of a few ways to make the experience better.
> > 1. Respect DPkg::Lock::Timeout also for the other apt locks
> > (e.g. in apt-pkg/acquire.cc:pkgAcquire::GetLock).
> > 2. Add a new lock configuration for the cache/archives
> > (e.g. APT::Cache::Lock::Timeout).
> > 3. Implement per-download locking with a timeout as requested in #478961.
> >
> > If either of these approaches is acceptable, we could propose an implementation
> > if desired.
>
>
> There are three locks at play in apt:
>
> 1. dpkg frontend lock - which apt always holds if it might invoke dpkg later on
> 2. dpkg lock - similar, but released to run dpkg
> 3. archives lock - held if we need to download anything
>
> I implemented waiting for the outer most one, as we can somewhat easily
> reason about this.
>
> Some stuff might not lock the dpkg locks and that is problematic. Like
> apt update should not change lists/ under me, and I should not be able
> to acquire the archives/ lock without holding the frontend lock, as
> otherwise I break the other process that just acquired the frontend
> lock and now wants to install stuff.
>
> I believe that all these different locks were essentially a mistake,
> we really should have had just the one frontend lock.
Sounds reasonable. I'd say this seals the fate of #478961.
> The problem is, if the 2 dpkg locks succeed and we then get to the
> archives lock, we can't go back to releasing the dpkg lock and waiting
> for both of them, as we ought to.
Maybe I'm missing something, but as long as the archives lock is
obtained last, wouldn't it always be possible to wait for it? For the
current situation with 3 locks, this still leaves the possibility of
respecting the lock timeout also for the archives lock. If I read the
code correctly, the timeout logic cannot be added to
pkgAcquire::GetLock since it would apply to not only the archives
lock, but I don't think writing an archives-specific retry-wrapper
would be hard. Would you consider such an addition?
Thanks for your fast response!
- Jouke
Reply to: