[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#1004583: overwriting {,src}pkgcache.bin on near-every APT call strains flash



>>>>> On 2022-02-01 12:27:50 +0100, Julian Andres Klode wrote:

 > Control: reopen -1

	I know there’re different approaches to this, but I believe that
	a report shouldn’t be closed so long as it describes a behavior
	that’s a. reproducible and b. known to affect at least some users.

	At the very least, keeping the report open will help prevent
	others from reporting the same issue over and over again.

	For the cases where the developer finds the existing behavior
	infeasible to alter, Debian BTS provides the ‘wontfix’ tag.

 > On Sun, Jan 30, 2022 at 07:10:34PM +0000, Ivan Shmakov wrote:

 >> A proper solution would be to move to some file format that
 >> allows in-place updates, such as Berkeley DB or SQLite.

 > No it would not be.  The best we can do, and maybe should do
 > is move pkgcache.bin to /run.

 > In-place updates means _a lot_ of things can go wrong.  Like you
 > forget to delete packages that are no longer in the repository.

	While I’m by no means familiar with the code base, this problem
	does not sound insurmountable.  Moreover, this is something
	I personally would be interested in working on.

 > There’s also no way to change this in any case, the database
 > format is an integral part of the public API, it would take
 > decade(s) to switch to anything else.

	I wasn’t aware of this (though it does make sense, given the
	availability of alternative interfaces to APT functions.)

	Where this format, and its intended use, are documented?

 > Unfortunately I only thought about moving pkgcache.bin to
 > /run directly after sending the close email, but I think
 > that might be a reasonable compromise to discuss, and if
 > we support $XDG_RUNTIME_DIR/apt/pkgcache.bin as well, then
 > that would even get us caching for non-root users.

	I can’t say I readily see the utility of that, at least by itself.

	Sure, I’ve used apt-get as a non-root user to download lists
	and packages to copy to systems where the network connectivity
	was poor (or non-existent), but I had to point Dir::Cache and
	Dir::State::Lists to locations writable by that same user
	to achieve my goals, not just pkgcache.bin.

 > How to move that file out of /var/cache/apt is going to be
 > a tricky question, as that location is to some extend part
 > of the API as well - people might set Dir::Cache to /tmp/apt
 > and expect pkgcache.bin to end up in /tmp/apt.

	The APT configuration language doesn’t seem to provide
	a straightforward solution for invalidating a default
	for one option when another one is set by the user, indeed.

	Given that the current behavior is, AFAIK, problematic only in
	two cases (a. /var/cache/apt resides on an SD card; and
	b. block-level backups are used for the filesystem containing
	said directory; which is, incidentally, how I initially became
	aware of this behavior), both of which seem relatively rare,
	I think the best approach would be to either let the user
	decide (such as via debconf), or at least inform them of the
	potential problem.

	Furthermore, as installing Debian on an SD card (or similar
	flash-based device not rated for the amount of writing the
	updating of pkgcache.bin might take) doesn’t seem common outside
	of ARM-based SBCs, it may make sense to restrict the fix,
	whichever that might be, to ARM (arm64, armhf, armel)
	architectures.

	More directly, the fix can be applied if it can be detected with
	a degree of reliability at .postinst time that pkgcache.bin
	might end up on an SD card.  (The necessary info is mostly
	there, see lsblk(8) output for instance, though I know of no
	straightforward way to map a file to the stack of devices
	it resides on.  Presumably we can consider placing pkgcache.bin
	under /run if we find something like mmcblk* while traversing
	the stack upwards from the device that holds the filesystem.)

 > With regards to srcpkgcache.bin, which represents the available
 > packages, but no installed packages (pkgcache.bin is built from
 > it by adding the dpkg status file in-place, in-memory, and then
 > writing it back out), I think having that around after a reboot
 > is what we want such that you don’t need to rebuild the entire
 > cache each boot.

	FWIW, I mainly use Debian on Socket AM2-class hardware (which
	is to say, about 12 years old), and IME it takes something like
	20 seconds to rebuild the cache.  As such, I wouldn’t myself be
	much concerned with rebuilding the cache from scratch.

-- 
FSF associate member #7257  http://am-1.org/~ivan/


Reply to: