Re: rsync in apt sources.list?
From: Dan Jacobson <jidanni@jidanni.org>
>It seems the simplest solution is to just use
>http://home.tiscali.cz:8080/~cz210552/aptrsync.html
>But why does he do at the bottom
>
># Get anything we missed due to failed rsync's. tim@fungible.com 24 Mar 2002.
>os.system('apt-get update')
># Used to have a call to apt-cache gencaches here, but I think that's
># redundant with the apt-get update above. tim@fungible.com 24 Mar 2002.
>
>Doing apt-get update just seems to start downloading the Packages.gz
>even though we just rsynced Packages. Is apt supposed to detect
>Packages are rater fresh and not download? It just downloaded over
^^^^^ I can't quite guess your meaning here.
>again for me.
It could easily be a bug. The rsync servers I was hitting randomly
rejected connections, so I didn't reliably get current Packages files
unless I did the "apt-get update". I couldn't easily test that the
script actually improved performance, because I didn't control the
servers I was hitting and I didn't set up my own server to test
against. I didn't carefully monitor what the "apt-get update" was
really doing.
The entire issue is moot, IMO, since the person running the server
said it couldn't support people routinely doing rsync against it
because rsync's compression used too much CPU time. Unless rsync now
supports precomputing the compression, and the servers are configured
to do that, using aptrsync is not being a good citizen.
I use apt-proxy now and am happy. From apt-proxy's manual entry I
suppose it often uses rsync to do its work, but now I have three
machines using my cache, so I figure the decreased load ought to
compensate for the increased CPU from using rsync.
--
Tim Freeman tim@fungible.com
GPG public key fingerprint ECDF 46F8 3B80 BB9E 575D 7180 76DF FE00 34B1 5C78
P. S. Here's my summary of the previous discussion about why aptrsync
was not a good idea.
Date: Mon, 8 Apr 2002 19:27:59 -0700
To: jgg@debian.org
Cc: 128818@bugs.debian.org, hsn@cybermail.net, deity@lists.debian.org
Subject: Re: Bug#128818: apt-rsync is great
From: tim@fungible.com (Tim Freeman)
From: Jason Gunthorpe <jgg@debian.org>
>I think the folks on -devel have gone over it enough,
For the benefit of other readers, the conversation you're talking
about probably starts at:
http://lists.debian.org/deity/1999/deity-199910/msg00002.html
and continues on the rsync list at:
http://lists.samba.org/pipermail/rsync/1999-October/001403.html
>Quite simply, rsync uses tremendous amounts of disk IO, it reads the
>entire file on the server side and does lots of math on it, http on the
>other hand is intrinsicly rate limited by the requester.
I see. rsync has a "batch" mode which might resolve some of the
issues. It's described as "experimental" in the man entry for version
2.5.4-1 of the rsync package, which is the one in testing right now,
so maybe it makes sense not to depend on it yet. The discussion cited
above is about 2.5 years old and doesn't mention batch mode at all,
perhaps because rsync's batch mode didn't exist then.
The as-yet-nonexistent compressor that is rsync-friendly would be
required to make a good solution for the whole problem.
However, I'm satisfied that building a version of apt-rsync that is
friendly to the server is blocked on the development of this other
software, so it's time to set this issue aside.
--
Tim Freeman
tim@fungible.com
Reply to: