Bug#372712: apt: periodically roll up pdiffs

[Matt Taggart <taggart@debian.org>]

I had a similar idea as Andrea Mennucc mentions in #372712 for the problem of 
so many pdiffs. The idea is similar to a scheme you might use for nightly 
incremental backups. You might run a "zero" backup once a month, a "one" 
backup every 15 days, a "two" every 7, a "three" every 3 and a "four" every 
day". For example:

 July 2006           Aug 2006
            0        0 4 4 3 2
4 4 3 4 4 3 2    4 3 4 4 3 4 2
4 4 3 4 4 3 2    3 4 4 1 4 4 2
1 3 4 4 3 4 2    4 4 3 4 4 3 2
3 4 4 3 4 4 2    4 3 4 4 1
4 1


On any given day you'd need at most 5 patches and many days far less than 
that.  The reason for doing this is not just to reduce the number of files, 
but the overall data, as a lot of the data in the diff is redundant. Consider 
the case of a package that is updated every day for a month. Under the current 
scheme a client not updating for that month would need to download the 
differences for that package 30 times right? Under an incremental scheme the 
worst case is 5 diffs for that package. It's an even bigger win for longer 
periods of time, the current scheme will start really falling down once we get 
a few more months of pdiffs.

Thanks,

-- 
Matt Taggart
taggart@debian.org