Re: split descriptions Re: PROPOSAL to sarge+1 - Split main in sub-repositories
Mike Fedyk <mfedyk@matchmail.com> writes:
> Andrea Mennucc wrote:
>
>> hi
>>
>> first of all I redid the computazion using gzip
>>
>> cp /var/lib/apt/lists/*sarge_main_binary-i386_Packages /tmp/main_all
>> cd /tmp
>> egrep '^ |^Description|^Packa' main_all > main_descr
>> egrep -v '^ |^Description' main_all > main_data
>> ls -s main_*
>> gzip -v main_*
>> ls -s main_*
>>
>> result: 3044 main_all.gz 1220 main_data.gz 1672 main_descr.gz
>>
>> as you see we are saving 1824kb here (each time we do not download
>> the descriptions), or 152 kb, (downloading both)
>
> Andrea, have you not heard of the (I forget who said it first)
> proposal to have a daily and weekly updated packages file?
>
> The daily files only contain the packages that have changed the
> previous day up until the point the packages file was generated.
>
> The weekly only contain the packages that have changed the previous
> week up until the point the packages file was generated.
>
> You could have several weekly files, and even a monthly, but that's
> debatable.
>
> I'd say this would save you much more bandwitdh than splitting out the
> descriptoin files, and it doesn't change the format, only includes
> fewer pakcages in the daily and weekly files. Of course splitting the
> descriptoins into seperate files can be combined with this, but the
> need to do so becomes much less pressing.
>
> What do you guys think?
And an apt sources.list like this? [assuming no new apt syntax]
deb http://ftp.debian.org/debian sid main/weekly
deb http://ftp.debian.org/debian sid main/weekly+1
deb http://ftp.debian.org/debian sid main/weekly+2
deb http://ftp.debian.org/debian sid main/weekly+3
deb http://ftp.debian.org/debian sid main/weekly+4
deb http://ftp.debian.org/debian sid main/weekly+5
deb http://ftp.debian.org/debian sid main/weekly+6
The idea of incremental packages files addresses the problem of
downloading the same data over and over partly. You still download the
same data over and over once a week. Adding a monthly file (+4 updates
for each week) would reduce this further.
What it doesn't address is client diskspace, memory usage and cpu
time.
MfG
Goswin
Reply to: