mirrors.kernel.org (Re: rsync and debian mirror consistency)
H. Peter Anvin (ftpadmin@kernel.org) wrote:
> Justin Pryzby wrote:
>> Are you sure? My understanding is that a "push" mirror setup (can) use
>> the same rsync script (template) as a pull mirror, but the script is
>> (additionally) run via ssh with authorized_keys restrictions (and
>> without specifying specific files to be updated).
>
> Are you implying that I don't know what happens on our own servers?
>
> For a variety of reasons, we don't use the same push mirror system as
> most of the other mirrors.
Hi Everyone
He seems to say that debian.org pushes archive content to the kernel.org
mirrors (rather than running an ssh trigger causing them to immediately
do a "pull"). It seems that consistency between the .deb files and the
Packages.gz files isn't maintained by this special push, as it is with
the anonftpsync template script. Is this correct? Is it possible to
change the procedure to guarantee consistency of the archive at such
mirror sites? I think the only relevant change is to first send /pool/
without --del, and then to send the remainder of the files, with --del.
AFAICT --delay-updates isn't strictly needed, and --delete-after can
probably be just --del.
This seems to be relevant:
http://lists.debian.org/debian-mirrors/2007/11/msg00029.html
Justin
--- Begin Message ---
Hello,
It seems to be the case that the Debian mirror routine doesn't try to
maintain consistency between the Packages.gz files and the available
.debs. That means that "apt-get update && apt-get upgrade" can fail
due to files that haven't been transferred yet.
That was a known problem with the template "anonftpsync" script.
Recent versions first rsync the .deb files from /pool/, then update
the rest (including the Packages.gz files), as well as removal of old,
no longer referenced .debs. AFAICT the --delay-updates option isn't
strictly needed, and --delete-after could probably be just --del.
Would you consider using a 2-pass rsync for mirroring?
Thanks,
Justin
[0] http://www.debian.org/mirror/anonftpsync
--- End Message ---
--- Begin Message ---
Justin Pryzby wrote:
Hello,
It seems to be the case that the Debian mirror routine doesn't try to
maintain consistency between the Packages.gz files and the available
.debs. That means that "apt-get update && apt-get upgrade" can fail
due to files that haven't been transferred yet.
That was a known problem with the template "anonftpsync" script.
Recent versions first rsync the .deb files from /pool/, then update
the rest (including the Packages.gz files), as well as removal of old,
no longer referenced .debs. AFAICT the --delay-updates option isn't
strictly needed, and --delete-after could probably be just --del.
Would you consider using a 2-pass rsync for mirroring?
We get the rsync pushed out to us, so it's out of our hands.
-hpa
--- End Message ---
Reply to: