[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Rsync on servers



<sigh> - haven't we had exactly this discussion numerous times before? 
Well, here's my opinion (once more...).

Before starting to develop a new distribution scheme for Debian CD
images, I carefully considered whether to use rsync or not, and came
to the conclusion that while rsync would be possible, there are too
many small problems, too much work to implement things, and too little
gain at the end of the day.

Points in favour of rsync:

- With gzip --rsyncable, large amounts of bandwidth could be saved
- Packages files could be updated extremely efficiently
- Flexibility: Can make use of partial/corrupted downloads later on
- Server speed could be increased by reversing the algorithm so that
  the work is shifted to the client
- Server speed could be increased by not compressing data before
  sending it. I suspect that more than 50% of the time of an rsync is
  spent in zlib.
- rsync is just such a nice tool, it deserves being used everywhere!-)

Points against rsync:

- Patents, patents, patents, both on the regular rsync algorithm and
  on the "reversed" variant mentioned above:
<http://164.195.100.11/netacgi/nph-Parser?Sect1=3DPTO1&Sect2=3DHITOFF&d=3D=PALL&p=3D1&u=3D/netahtml/srchnum.htm&r=3D1&f=3DG&l=3D50&s1=3D'6167407'.WKU.=&OS=3DPN/6167407&RS=3DPN/6167407>
<http://www.delphion.com/details?&pn=3DUS05721907__>
  (Dunno if these links work or are relevant, but the patents exist!)
- We'd need to persuade 200 mirror admins to run rsync on their
  servers - this will never happen
- I suspect rsync may have trouble to fully saturate a fast link, as
  there is a memory/speed tradeoff involved (you need memory for a
  per-connection waiting queue). This is an inherent problem of the
  algorithm.
- The authors of rsync have been "maintaining" rather than
  "developing" it for a long time now - you will have to implement any
  improvements yourself.
- rsync is not as efficient as it could be when dealing with huge
  files (this is especially relevant for CD images), or mostly
  unchanged files. Again, the algorithm could be changed to improve
  performance in these cases.
- Problems when using from restricted network environments - the rsync
  port is often blocked by firewalls. [The author of xdelta appears to
  be working on some kind of "rsync over HTTP" - I haven't looked at
  that.]

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer     |  CS student at the Technische  |  GnuPG key:
  | \/¯|  http://atterer.net  |  Universität München, Germany  |  0x888354F7
  ¯ ´` ¯

Attachment: pgpsnWFPKDJs3.pgp
Description: PGP signature


Reply to: