Bug#718329: apt: Please add support for .deb with uncompressed data.tar member
Control: tags -1 patch
Hi!
On Tue, 2013-07-30 at 12:07:00 +0200, Guillem Jover wrote:
> Package: apt
> Version: 0.9.9.4
> Severity: wishlist
> The apt code does not support uncompressed data.tar .deb members, as
> specified by deb(5), which have been supported by since dpkg 1.10.24.
>
> At least the kernel team is currently using «dpkg-deb -Zgzip -z0» on
> linux-source-3.10, which generates an uncompressed data.tar.gz member
> (which TBH in a strict sense it's bogus for dpkg to create), because
> otherwise dak rejects uncompressed .deb data.tar members by way of
> python-apt, and finally libapt. dpkg-deb calls that “compressor” «none»,
> in case you'd like to have some term consistency.
>
> In any case uncompressed data.tar members are useful in general, when
> the contents are all already compressed and there's no gain in
> recompressing the member again. AFAIK debdelta is also using those
> when reconstructing .debs to feed to dpkg, to avoid duped work.
Here's a small patch series implementing this, and also support for
control.tar and control.tar.xz, added in dpkg 1.17.6.
I've tested the code by using «apt-ftparchive contents»,
«apt-ftparchive packages» and apt-extracttemplates, over two packages
compressed with «dpkg-deb --uniform-compression -Znone -b pkg-none/»
and «dpkg-deb --uniform-compression -Zxz -b pkg-xz/». Let me know what
you think.
Thanks,
Guillem
From da50f97e6fcf3842e1220e903a939dca4920411f Mon Sep 17 00:00:00 2001
From: Guillem Jover <guillem@debian.org>
Date: Sun, 16 Feb 2014 23:29:13 +0100
Subject: [PATCH 1/3] ExtractTar: Allow an empty decompressor program
This allows for uncompressed tar files, as the decompressor process will
not get interposed in-between the file descriptors.
---
apt-inst/contrib/extracttar.cc | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/apt-inst/contrib/extracttar.cc b/apt-inst/contrib/extracttar.cc
index fb4db42..45da3b7 100644
--- a/apt-inst/contrib/extracttar.cc
+++ b/apt-inst/contrib/extracttar.cc
@@ -111,6 +111,12 @@ bool ExtractTar::Done(bool Force)
gzip will efficiently ignore the extra bits. */
bool ExtractTar::StartGzip()
{
+ if (DecompressProg.empty())
+ {
+ InFd.OpenDescriptor(File.Fd(), FileFd::ReadOnly, FileFd::None, false);
+ return true;
+ }
+
int Pipes[2];
if (pipe(Pipes) != 0)
return _error->Errno("pipe",_("Failed to create pipes"));
--
1.9.0.rc3.244.g3497008
From 1b9c1af7f0e8721302c885362de7600aaf1e4f73 Mon Sep 17 00:00:00 2001
From: Guillem Jover <guillem@debian.org>
Date: Mon, 17 Feb 2014 22:02:38 +0100
Subject: [PATCH 2/3] DebFile: Refactor ExtractTarMember() out from
ExtractArchive()
Generalize DebFile::ExtractArchive() to take a member base name, so that
we can reuse it for control.tar member extraction too.
---
apt-inst/deb/debfile.cc | 17 ++++++++++++-----
apt-inst/deb/debfile.h | 1 +
2 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/apt-inst/deb/debfile.cc b/apt-inst/deb/debfile.cc
index 79434d8..15db1a7 100644
--- a/apt-inst/deb/debfile.cc
+++ b/apt-inst/deb/debfile.cc
@@ -88,21 +88,20 @@ const ARArchive::Member *debDebFile::GotoMember(const char *Name)
return Member;
}
/*}}}*/
-// DebFile::ExtractArchive - Extract the archive data itself /*{{{*/
+// DebFile::ExtractTarMember - Extract the contents of a tar member /*{{{*/
// ---------------------------------------------------------------------
/* Simple wrapper around tar.. */
-bool debDebFile::ExtractArchive(pkgDirStream &Stream)
+bool debDebFile::ExtractTarMember(pkgDirStream &Stream,const char *Name)
{
// Get the archive member
const ARArchive::Member *Member = NULL;
std::string Compressor;
- std::string const data = "data.tar";
std::vector<APT::Configuration::Compressor> compressor = APT::Configuration::getCompressors();
for (std::vector<APT::Configuration::Compressor>::const_iterator c = compressor.begin();
c != compressor.end(); ++c)
{
- Member = AR.FindMember(std::string(data).append(c->Extension).c_str());
+ Member = AR.FindMember(std::string(Name).append(c->Extension).c_str());
if (Member == NULL)
continue;
Compressor = c->Binary;
@@ -111,7 +110,7 @@ bool debDebFile::ExtractArchive(pkgDirStream &Stream)
if (Member == NULL)
{
- std::string ext = "data.tar.{";
+ std::string ext = std::string(Name) + ".{";
for (std::vector<APT::Configuration::Compressor>::const_iterator c = compressor.begin();
c != compressor.end(); ++c)
ext.append(c->Extension.substr(1));
@@ -129,6 +128,14 @@ bool debDebFile::ExtractArchive(pkgDirStream &Stream)
return Tar.Go(Stream);
}
/*}}}*/
+// DebFile::ExtractArchive - Extract the archive data itself /*{{{*/
+// ---------------------------------------------------------------------
+/* Simple wrapper around DebFile::ExtractTarMember. */
+bool debDebFile::ExtractArchive(pkgDirStream &Stream)
+{
+ return ExtractTarMember(Stream, "data.tar");
+}
+ /*}}}*/
// DebFile::ControlExtract::DoItem - Control Tar Extraction /*{{{*/
// ---------------------------------------------------------------------
diff --git a/apt-inst/deb/debfile.h b/apt-inst/deb/debfile.h
index 38211fb..ecef71d 100644
--- a/apt-inst/deb/debfile.h
+++ b/apt-inst/deb/debfile.h
@@ -48,6 +48,7 @@ class debDebFile
class ControlExtract;
class MemControlExtract;
+ bool ExtractTarMember(pkgDirStream &Stream, const char *Name);
bool ExtractArchive(pkgDirStream &Stream);
const ARArchive::Member *GotoMember(const char *Name);
inline FileFd &GetFile() {return File;};
--
1.9.0.rc3.244.g3497008
From db50a8666253c506f8361c92611ccb0682441aab Mon Sep 17 00:00:00 2001
From: Guillem Jover <guillem@debian.org>
Date: Sun, 16 Feb 2014 23:30:48 +0100
Subject: [PATCH 3/3] Add support for data.tar, control.tar and control.tar.xz
Sync the deb(5) format support with latest dpkg, by allowing
uncompressed tar members and xz compressed control.tar. This
also refactors the control.tar member extraction by using
ExtractTarMember(), which also means future changes only need
to be implemented in a single place.
---
apt-inst/deb/debfile.cc | 21 ++++++++++-----------
cmdline/apt-extracttemplates.cc | 18 ++++--------------
2 files changed, 14 insertions(+), 25 deletions(-)
diff --git a/apt-inst/deb/debfile.cc b/apt-inst/deb/debfile.cc
index 15db1a7..a811bbe 100644
--- a/apt-inst/deb/debfile.cc
+++ b/apt-inst/deb/debfile.cc
@@ -42,12 +42,15 @@ debDebFile::debDebFile(FileFd &File) : File(File), AR(File)
return;
}
- if (!CheckMember("control.tar.gz")) {
- _error->Error(_("This is not a valid DEB archive, missing '%s' member"), "control.tar.gz");
+ if (!CheckMember("control.tar") &&
+ !CheckMember("control.tar.gz") &&
+ !CheckMember("control.tar.xz")) {
+ _error->Error(_("This is not a valid DEB archive, missing '%s' member"), "control.tar");
return;
}
- if (!CheckMember("data.tar.gz") &&
+ if (!CheckMember("data.tar") &&
+ !CheckMember("data.tar.gz") &&
!CheckMember("data.tar.bz2") &&
!CheckMember("data.tar.lzma") &&
!CheckMember("data.tar.xz")) {
@@ -109,6 +112,9 @@ bool debDebFile::ExtractTarMember(pkgDirStream &Stream,const char *Name)
}
if (Member == NULL)
+ Member = AR.FindMember(std::string(Name).c_str());
+
+ if (Member == NULL)
{
std::string ext = std::string(Name) + ".{";
for (std::vector<APT::Configuration::Compressor>::const_iterator c = compressor.begin();
@@ -201,14 +207,7 @@ bool debDebFile::MemControlExtract::Process(Item &Itm,const unsigned char *Data,
it parses it into a tag section parser. */
bool debDebFile::MemControlExtract::Read(debDebFile &Deb)
{
- // Get the archive member and positition the file
- const ARArchive::Member *Member = Deb.GotoMember("control.tar.gz");
- if (Member == 0)
- return false;
-
- // Extract it.
- ExtractTar Tar(Deb.GetFile(),Member->Size,"gzip");
- if (Tar.Go(*this) == false)
+ if (Deb.ExtractTarMember(*this, "control.tar") == false)
return false;
if (Control == 0)
diff --git a/cmdline/apt-extracttemplates.cc b/cmdline/apt-extracttemplates.cc
index 8e19371..2408a7d 100644
--- a/cmdline/apt-extracttemplates.cc
+++ b/cmdline/apt-extracttemplates.cc
@@ -24,8 +24,7 @@
#include <apt-pkg/pkgcachegen.h>
#include <apt-pkg/version.h>
#include <apt-pkg/tagfile.h>
-#include <apt-pkg/extracttar.h>
-#include <apt-pkg/arfile.h>
+#include <apt-pkg/debfile.h>
#include <apt-pkg/deblistparser.h>
#include <apt-pkg/error.h>
#include <apt-pkg/strutl.h>
@@ -91,18 +90,9 @@ string DebFile::GetInstalledVer(const string &package)
/* */
bool DebFile::Go()
{
- ARArchive AR(File);
- if (_error->PendingError() == true)
- return false;
-
- const ARArchive::Member *Member = AR.FindMember("control.tar.gz");
- if (Member == 0)
- return _error->Error(_("%s not a valid DEB package."),File.Name().c_str());
-
- if (File.Seek(Member->Start) == false)
- return false;
- ExtractTar Tar(File, Member->Size,"gzip");
- return Tar.Go(*this);
+ debDebFile Deb(File);
+
+ return Deb.ExtractTarMember(*this, "control.tar");
}
/*}}}*/
// DebFile::DoItem examine element in package and mark /*{{{*/
--
1.9.0.rc3.244.g3497008
Reply to: