[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#952910: [live-build] save compressed contents file to disk



Package: live-build
Version: 1:20191221
Owner: jnqnfe@gmail.com
Severity: wishlist

as pointed out in #952907, the archive contents file is not cached in
an efficient way. #952907 concerns itself with fixing the small aspect
of not pointlessly leaving the file hanging around once it has been
finished with since caching is not actually being exploited.

here we concern ourselves with taking things further in another respect
- instead of storing the decompressed file in the cache, let's tweak
things such that it is the compressed file that is saved to disk.

note that as discussed before, the file for the 'main' archive area is
by far the largest, and is over 600MiB currently for amd64 whilst the
compressed file is ~30MiB.

we currently download the compressed file, piping it into gunzip and
saving the decompressed inner file to disk, parse the file, then delete
it (per #952907 the last used hangs around pointlessly which that
proposes fixing). we could instead save the compressed file to disk,
then pipe the decompression of that (it only holds one file) through
awk to get the list.

this would mean much less disk IO. it also means that if/when we do
change things to properly take advantage of caching then we require
much less disk space to do so.

patch to be submitted via salsa soon


Reply to: