Re: big Packages.gz file
>>>>> " " == Brian May <bam@debian.org> writes:
>>>>> "zhaoway" == zhaoway <zw@debian.org> writes:
zhaoway> This is only a small part of the whole story, IMHO. See
zhaoway> my other email replying you. ;)
>>> Maybe there could be another version of Packages.gz without
>>> the extended descriptions -- I imagine they would take
>>> something like 33% of the Packages file, in line count at
>>> least.
zhaoway> Exactly. DIFF or RSYNC method of APT (as Goswin pointed
zhaoway> out), or just seperate Descriptions out (as I pointed out
zhaoway> and you got it too), nearly 66% of the bits are
zhaoway> saved. But this is only a hack, albeit efficient.
> At the risk of getting flamed, I investigated the possibility
> of writing an apt-get method to support rsync. I would use this
> to access an already existing private mirror, and not the main
> Debian archive. Hence the server load issue is not a
> problem. The only problem I have is downloading several megs of
> index files every time I want to install a new package (often
> under 100kb) from unstable, over a volume charged 28.8 kbps PPP
> link, using apt-get[1].
I tried the same, but I used the copy method as template, which is
rather bad. Should have used http as starting point.
Can you send me your patch please.
> I think (if I understand correctly) that I found three problems
> with the design of apt-get:
> 1. It tries to down-load the compressed Packages file, and has
> no way to override it with the uncompressed file. I filed a bug
> report against apt-get on this, as I believe this will also be
> a problem with protocols like rproxy too.
> 2. apt-get tries to be smart and passes the method a
> destination file name that is only a temporary file, and not
> the final file. Hence, rsync cannot make a comparison between
> local and remote versions of the file.
I wrote to the deity mailinglist concerning those two problems with 2
possible sollution. Till now the only answere I got was "NO we don't
want rsync" after pressing the issue here on debian-devel.
> 3. Instead, rsync creates its own temporary file while
> downloading, so apt-get cannot display the progress of the
> download operation because as far as it is concerned the
> destination file is still empty.
Hmm, isn't there a informational message you can output to hint of the
progress? We would have to patch rsync to generate that style of
progress output or fork and parse the output of rsync and pass on
altered output.
> I think the only way to fix both 2 and 3 is to allow some
> coordination between apt-get and rsync where to put the
> temporary file and where to find the previous version of the
> file.
Doing some more thinking I like the second solution to the problem
more and more:
1. Include a template (some file that apt-get thinks matches best) in
the fetch request. The rsync method can then copy that file to the
destination and rsync on it. This would be the uncompressed Packages
file or a previous deb or the old source.
2. return wheather the file is compressed or not simply by passing
back the destination filename with the appropriate extension (.gz). So
the destination filename is altered to reflect the fileformat.
MfG
Goswin
Reply to: