[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#901257: Please consider adopting zchunk for efficient updates



On Sun, Jun 10, 2018 at 07:14:55PM +0200, Michael Stapelberg wrote:
> The zchunk format, outlined in the post, is being introduced in Fedora for
> efficient metadata transfer.

As I understand it, zchunk isn't competition for pdiff (per se), but
competition for the compressed files xz/bz2/gz as files aren't only
downloaded in zchunk format but also stored in that format. So what we
(can) do at the moment with downloading the smallest files (like xz) and
store it in a format for fastest access vs. space usage like
uncompressed or lz4 compressed wouldn't be possible anymore.

It therefore also pre-depends on people and programs stopping to access
files in /var/lib/apt/lists directly, which we would like to reach some
point, but aren't there yet.

This format also depends on having random access to remote files aka
HTTP range requests – suprising as it sounds, some users in the wild
interact with servers who do not support this feature & support for
having multiple ranges in the request is another beast to implement in
both server and client. For those users the format would combine all
disadvantages – and these users happen to be the ones where download
metrics matter the most (at least in my experience).


That is entirely ignoring if that has any actual benefits as I haven't
tried it at all, which would probably be the very first step – to
actually verify that there is a point in all this.

Judging just by the text, it seems to combine all the worst aspects it
can in the worst case and seems to have only marginal benefits above
what we have¹ in the best case, but a reality check might provide
another/better view (not volunteering myself).


Best regards

David Kalnischkies

¹ which is different from Fedora both in that they have no pdiff and
that they don't keep indexes in a as central location as we do (from my
limited understanding of never actually having used yum/dnf).

Attachment: signature.asc
Description: PGP signature


Reply to: