Re: New method for Packages/Sources file updates
Andreas Barth <aba@not.so.argh.org> writes:
> * Goswin von Brederlow (brederlo@informatik.uni-tuebingen.de) [041125 15:35]:
>> [...]
>
> Please see the solution in bug 128818. If you think that there are
> issues with it, please say so (and do it soon). Otherwise, I'll probably
> write the code for katie this weekend.
>
>
> Cheers,
> Andi
I'm aware of a patch file method and from a quick scan of the bug it
sounds like 128818 describes that method.
But the patch files are expensive to generate each day and take a lot
of disk space 2.7Gb (you need to preserve one 15MB Packages file per
day [or 2.5MB bz2 and bunzip2 it each day]). That is for the
one-patch-file idea. For the a-patch-file-per-day method you only need
the space for each diff and one diff per day. But the efficiency
degrades because the same packages get patched multiple times.
The apt rsync method (begining of 128818) for packages updates, of
which there are several fully working versions out too, does put a big
strain on mirrors and is realy no suitable.
The date sorted Packages file and the removal file on the other hand
are very easy to generate (possibly by katie itself) and only need a
few extra K for the removals. It is also about twice as efficient
since removals of packages are mostly implicit (and complete removals
only use the package name).
Summary:
My method is more efficient and more disk/cpu friendly alowing for a
longer timespan between updates.
MfG
Goswin
PS: If you use context diffs or rsync a date sorted Packages file
should also be more efficient. It is less likely for old packages to
get updated and bigger blocks would remain identical at the end of the
file.
PPS: If anyone has the diff method at hand I would like to see the
same numbers I provided for it: daily update through october and
update from oct. 1st.
Reply to: