Re: Idea: rsync-based source format
Hi!
On Fri, 2015-08-21 at 16:32:09 +0100, Ian Jackson wrote:
> (I spoke to Guillem about this at Debconf and promised to write it up
> so he could think about it properly at his leisure.)
[ Checked this over DebConf, but then I could not find you on the
venue anymore. :) ]
I did some preliminary quick pondering and got some concerns, and I think
perhaps a workable alternative solution that might cover your needs?
> None of our existing source package formats can reliably
> represent
> - files which exist upstream but are deleted in Debian
> - symlinks and changes to symlinks
> - empty files vs. nonexistent files
> I would like to propose a format which would address these problems.
Actually you should be able to represent at least these with a git format
patch, which are already supported by the latest patch program (its only
current limitation AFAIR is binary file deltas), and which is required by
dpkg-dev to be able to properly handle them at extraction time.
> It would be more like a successor to 1.0 with diff, than 3.0 (quilt)
> is, in that it wouldn't represent a patch stack, merely a tree.
(From the code PoV, and from the properties you describe it would
probably be more a successor of 2.0 than 1.0, but sure.)
> I am tentatively suggesting that this format should be called
> `3.0 (rsync)' but I have no strong opinions about the name.
> Specification:
> It also contains an rsync batchfile P_V-R.rsync.Z.
This is what triggers my concerns. I was not aware of rsync batchfiles!
So I took a quick look at the man page only (I've not dug further),
and I've got the impression this might not be a good format for long
term storage, given that it seems to rely on the rsync protocol itself
(it is already at version 28; does the program remove support for
ancient protocol versions for example)? It would require anything that
wants to extract the source to have an rsync version >=, so even if a
stable dpkg-source supports the format, it might need an rsync version
as recent as the one being used in sid. It also ties the implentation
of the format to the rsync tool, because I assume we'd not want to
reimplement it ourselves(?), and keep in sync with upstream over time.
And as such it would require pulling rsync into the build-essential
set practically forever, because once there are such source packages
around dpkg-source should be able to at least extract them (well it
could get demoted to Recommends in case we switched to something else).
I'd recommend looking into git format patches, which should be a
stable interchange format, are already supported by our dpkg tools
(although by delegating the work to GNU patch), and should be able
to represent the changes you mentioned before. Not sure if they would
take more space, although I'd assume that should not make much of a
difference once compressed with something like xz.
(Something I've had in mind is to make dpkg-source itself understand
git format patches natively so that those can be used even on systems
where the patch program does not handle them, or in case we want to
also support binary deltas.)
In case we'd still wanted for whatever reason to distinguish this new
format from a quilt one, I guess we could always add a new one such as
«3.0 (delta)» or similar.
Or would that not work for you for some reason or I've missed something
very obvious?
Thanks,
Guillem
Reply to: