[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Solving the compression dilema when rsync-ing Debian versions

>>>>> " " == Otto Wyss <otto.wyss@bluewin.ch> writes:

    >>> So why not solve the compression problem at the root? Why not
    >>> try to change the compression in a way so it does produce a
    >>> compressed
     > result
    >>> with the same (or similar) difference rate as the source?
    >>  Are you going to hack at *every* different kind of file format
    >> that you might ever want to rsync, to make it rsync friendly?
     > No, I want rsync not even to be mentioned. All I want is
     > something similar to

     >         gzip --compress-like=old-foo foo

AFAIK thats NOT possible with gzip. Same with bzip2.

     > where foo will be compressed as old-foo was or as aquivalent as
     > possible. Gzip does not need to know anything about foo except
     > how it was compressed. The switch "--compress-like" could be
     > added to any compression algorithmus (bzip?) as long as it's
     > easy to retrieve the compression scheme. Besides the following
     > is completly legal but probably not very sensible

     >         gzip --compress-like=foo bar

     > where bar will be compressed as foo even if they might be
     > totally unrelated.

     > Rsync-ing Debian packages will certainly take advantage of this
     > solution but the solution itself is 100% pure compression
     > specific. Anything which needs identical compression could
     > profit from this switch. It's up to profiting application to
     > provide the necessary wrapper around.

    >> gzip --rsyncable, aloready implemented, ask Rusty Russell.

     > The --rsyncable switch might yield the same result (I haven't
     > checked it sofar) but will need some internal knowledge how to
     > determine the old compression.

As far as I understand the patch it forces gzip to compress the binary
into chunks of 8K. So every 8K theres a break where rsync can try to
match blocks. It seems to help somehow, but I think it handles
movement of data in a file badly (like when a line is inserted).

     > As I read my mail again the syntax for "compressing like" could
     > be

     >         gzip --compress=foo bar

     > where bar is compressed as foo was. Foo is of course a
     > compressed file (how else could the compression be retrieved)
     > while bar is not.

I wish it where that simple.


Reply to: