Bug#1007717: Draft resolution for "Native source package format with non-native version"
On 11/05/22 at 17:29 +0100, Ian Jackson wrote:
> Lucas Nussbaum writes ("Re: Bug#1007717: Draft resolution for "Native source package format with non-native version""):
> > Out of curiosity, if 3.0 (native) supported multiple tarballs, wouldn't
> > it be a good solution?
>
> Oh, I hadn't thought of that.
>
> > The main limitation I see is that it would not allow to represent
> > efficiently small changes to large text files (which a git-based diff
> > would allow).
>
> That may not be important. My feeling is that it wouldn't be.
>
> However, I think there are some other difficulties with this idea.
>
> *Deletion* of files *is* important. Something would have to be done
> to support that. (Tarball repacking is an abominable workflow which
> we should only do when we must.)
>
> It is important that packing and unpacking these things works roughly
> the same way that things work right now for the diffish formats. Ie,
> for a package with two tarballs, the first tarball would have to omit
> the Debian revision from its filename (so that it needn't be
> re-uploaded), and the second tarball would have to *contain* it.
> dpkg-source -b would have to calculate the appropriate second tarball
> as a diff from the first. (GNU tar has an incremental option that
> could perhaps be used.)
>
> I think, having followed this line of throught, the result looks quite
> like a "3.0 (diff)" only the diff is in the form of an incremental
> tarball rather than a textual patch file.
>
> This could definitely work.
FWIW, GNU tar's incremental mode supports file deletions.
Example:
------------------------------>8
# create a first archive
mkdir t
echo foo > t/f1
echo bar > t/f2
echo baz > t/f3
tar --listed-increment=snapshot-file -czvvvf archive.tgz t
# remove a file, create an incremental archive
rm t/f2
tar --listed-increment=snapshot-file -czvvvf archive.1.tgz t
# remove everything
rm -rf snapshot-file t
# extract incremental archives
tar Gvvvxf archive.tgz
tar Gvvvxf archive.1.tgz
# listing of archive.1.tgz
tar Gvvvtf archive.1.tgz
------------------------------>8
This is implemented using the GNU-specific D entry type. According to
https://www.freebsd.org/cgi/man.cgi?query=tar&apropos=0&sektion=5&format=html :
D This indicates a directory entry. Unlike the POSIX-stan-
dard "5" typeflag, the header is followed by data records
listing the names of files in this directory. Each name
is preceded by an ASCII "Y" if the file is stored in this
archive or "N" if the file is not stored in this archive.
Each name is terminated with a null, and an extra null
marks the end of the name list. The purpose of this en-
try is to support incremental backups; a program restor-
ing from such an archive may wish to delete files on disk
that did not exist in the directory when the archive was
made.
Note that the "D" typeflag specifically violates POSIX,
which requires that unrecognized typeflags be restored as
normal files. In this case, restoring the "D" entry as a
file could interfere with subsequent creation of the
like-named directory.
> But I think that might not meet ftpmaster's review needs. AIUI
> ftpmaster like to review the diff qua diff, and a tarball isn't so
> straightforward. I had a similar idea to use an rsync batchfile as
> the delta, which foundered on the same issue.
Note that it's not worse than using native formats, where ftpmasters get
a single tarball anyway...
Lucas
Reply to: