Re: New method for Packages/Sources file updates

[Goswin von Brederlow <brederlo@informatik.uni-tuebingen.de>]

Thiemo Seufer <ica2_ts@csv.ica.uni-stuttgart.de> writes:

> Goswin von Brederlow wrote:
> [snip]
>> >> With cumulative patches you run into the problem that you need a new
>> >> cummulative patch for every day that contains most of what the
>> >> previous one did. That realy quickly becomes a space issue.
>> >
>> > Errm, no, it doesn't need _one_ new cumulative patch. All the
>> > previously made cumulative diffs need to be updated.
>> 
>> I was thinking of a
>> 
>> -1day.diff
>> -2day.diff
>> -3day.diff
>> ...
>> 
>> So every day a new file appears at the end and contains most of what
>> all the others already contain.
>> 
>> Updating those cummulative diffs is also either inefficient (cat the
>> daily diffs together),
>
> That wouldn't be a cumulative diff.
>
>> very hard (figure out how to make a minimal diff
>> from the daylies) or you need every days Packages file (apt-dupdate
>> does that).
>
> It is not "very hard" to re-diff a few files to incorporate the changes
> between old and new Packages file.

Then how do you do it? If you don't store the Packages files then all
you have are the diffs. That means you have a series of "remove x-y"
and "insert 'text' @ x" blocks. I don't call it simple to figure out
all the overlaps and to generate new delete and insert blocks for it.

Is there any program in Debian to do this for ed script style diffs?

>> Having to store and diff every past days Packages file is a huge
>> resource drain and can't be done for more than a couple of days, maybe
>> up to 2 weeks.
>
> You don't need to store it.
>
>> Ask the apt-dupdate author for how long it takes every night and how
>> much disk space it uses.
>
> If that's true, then apt-dupdate is an example how to not do it.

I've yet to see something better to create the diffs.

>> > If we assume to hold 14 update cycles, have a cutoff if the size of
>> > the cumulative diff exceeds the size of the Packages file, and have
>> > linear growth of the diffs, then the additional space used is at most
>> > seven times the size of the Packages file. Normally it will be much
>> > less, because large archives don't thend to change that quickly.
>> 
>> 14 update cycles is a limitation on the process and isn't needed with
>> sorted Packages files.
>
> It is not a hard limit, and to speedup exorbitant numbers of update
> cycles isn't needed except for pathologic cases.
>
>> Also how do you get 'seven times'?
>
> ... linear growth of the diffs ...

Ahh, sorry. Reasonable assumption.

>...
> Thiemo

MfG
        Goswin