stable update of dpkg (>= 1.17.2) and apt (>= 0.9.16.1) for build profiles
Hi,
adding Adam because of request from Helmut (also added).
Quoting Johannes Schauer (2014-04-26 07:15:03)
> We would like to add build profile support [1] to packages in the archive and
> because build profiles add a new syntax to Build-Depends, tools like dpkg and
> apt need to be able to understand it as they would otherwise throw an error.
> Support for build profiles was integrated into a number of tools already and
> most of them already migrated into testing. You can see [1] for an overview.
>
> Because of this syntax change, packages that use this syntax can only be
> uploaded once tools in Debian stable support it because the build
> infrastructure runs Debian stable. One option would be to wait until the jessie
> release but we fear that if we go that route we might discover too late that we
> missed to patch one important tool and in the worst case have to wait for yet
> another release until packages with build profiles can be uploaded.
>
> Therefore we would like to test whether the Debian build infrastructure can
> handle uploads of source packages using build profiles as early as possible. It
> seems that the two options for making this happen is to either backport the
> current versions of dpkg, apt and python-apt or to make a stable update:
since a backport seems to be out of the question, we'd like to propose three
patches for dpkg, apt and python-apt as a stable update. Please find the three
patches attached.
The patches do not implement full profile support but just allow the new
Build-Depends syntax to be parsed without throwing an error. If the new syntax
is encountered, dependencies will be added as if no build profile was active.
The patches include no way to change the activated build profiles to keep them
as minimial as possible.
Do you think the attached patches are minimal/safe enough to include them as a
stable update?
If these patches were added to stable, then https://bugs.debian.org/744246
could be resolved before jessie release. This in turn would mean that we can
test build profiles before the jessie release. And this means that we can fix
not only bootstrapping bugs in packages before the release but also possibly
missing build profile related bugs in dpkg, apt, other build tools or even the
buildd machinery. That way, possible bugs could be weeded out before the jessie
release. This would also benefit our Google Summer of Code student for the
"Bootstrappable Debian" project whose patches could then immediately be applied
by package maintainers instead of his work only bearing fruit after the jessie
release.
Thank you for your consideration.
cheers, josch
diff --git debian/changelog debian/changelog
index 1f4d107..5ea6fd4 100644
--- debian/changelog
+++ debian/changelog
@@ -1,3 +1,15 @@
+dpkg (1.16.12+nmu1) stable; urgency=medium
+
+ * Non-maintainer upload.
+ * Support for restriction list Build-Depends syntax extension [1]
+ while assuming that no build profile was set
+ - enables dpkg to understand the restriction list syntax which is
+ required for source packages with build profile annotations
+ - does not allow to set build profiles (by default no profile is set)
+ [1] https://wiki.debian.org/BuildProfileSpec
+
+ -- Johannes Schauer <j.schauer@email.de> Wed, 30 Apr 2014 09:00:31 +0200
+
dpkg (1.16.12) stable; urgency=low
* Fix value caching in Dpkg::Arch by not shadowing the variables.
diff --git man/deb-src-control.5 man/deb-src-control.5
index 17f85b3..8273d98 100644
--- man/deb-src-control.5
+++ man/deb-src-control.5
@@ -177,8 +177,9 @@ fields is a list of groups of alternative packages. Each group is a list
of packages separated by vertical bar (or "pipe") symbols, "|". The
groups are separated by commas. Commas are to be read as "AND", and pipes
as "OR", with pipes binding more tightly. Each package name is
-optionally followed by a version number specification in parentheses and an
-architecture specification in square brackets.
+optionally followed by a version number specification in parentheses, an
+architecture specification in square brackets, and a profile specification
+in angle brackets.
The syntax of the
.BR Build\-Conflicts ,
@@ -188,7 +189,8 @@ and
fields is a list of comma-separated package names, where the comma is read
as an "AND". Specifying alternative packages using a "pipe" is not supported.
Each package name is optionally followed by a version number specification in
-parentheses and an architecture specification in square brackets.
+parentheses, an architecture specification in square brackets, and a profile
+specification in angle brackets.
A version number may start with a ">>", in which case any later version
will match, and may specify or omit the Debian packaging revision (separated
@@ -200,6 +202,10 @@ A architecture specification consists of one or more architecture names,
separated by whitespace. Exclamation marks may be prepended to each of the
names, meaning "NOT".
+A profile specification consists of one or more profile names, prefixed
+with the "\fBprofile.\fP" namespace, separated by whitespace. Exclamation
+marks may be prepended to each of the names, meaning "NOT".
+
Note that dependencies on packages in the
.B build\-essential
set can be omitted and that declaring build conflicts against them is
diff --git scripts/Dpkg/Deps.pm scripts/Dpkg/Deps.pm
index 21b8e9c..c05db67 100644
--- scripts/Dpkg/Deps.pm
+++ scripts/Dpkg/Deps.pm
@@ -266,6 +266,26 @@ architecture. This implicitely strips off the architecture restriction
list so that the resulting dependencies are directly applicable to the
current architecture.
+=item use_profiles (defaults to 1)
+
+Take into account the profile restriction part of the dependencies. Set
+to 0 to completely ignore that information.
+
+=item reduce_profiles (defaults to 0)
+
+If set to 1, ignore dependencies that do not concern the current build
+profile. This implicitly strips off the profile restriction list so
+that the resulting dependencies are directly applicable to the current
+profiles.
+
+=item reduce_restrictions (defaults to 0)
+
+If set to 1, ignore dependencies that do not concern the current set of
+restrictions. This implicitly strips off any restriction list so that the
+resulting dependencies are directly applicable to the current restriction.
+This currently implies C<reduce_arch> and C<reduce_profiles>, and overrides
+them if set.
+
=item union (defaults to 0)
If set to 1, returns a Dpkg::Deps::Union instead of a Dpkg::Deps::AND. Use
@@ -286,9 +306,17 @@ sub deps_parse {
$options{use_arch} = 1 if not exists $options{use_arch};
$options{reduce_arch} = 0 if not exists $options{reduce_arch};
$options{host_arch} = get_host_arch() if not exists $options{host_arch};
+ $options{use_profiles} = 1 if not exists $options{use_profiles};
+ $options{reduce_profiles} = 0 if not exists $options{reduce_profiles};
+ $options{reduce_restrictions} = 0 if not exists $options{reduce_restrictions};
$options{union} = 0 if not exists $options{union};
$options{build_dep} = 0 if not exists $options{build_dep};
+ if ($options{reduce_restrictions}) {
+ $options{reduce_arch} = 1;
+ $options{reduce_profiles} = 1;
+ }
+
# Strip trailing/leading spaces
$dep_line =~ s/^\s+//;
$dep_line =~ s/\s+$//;
@@ -310,6 +338,11 @@ sub deps_parse {
$dep_simple->reduce_arch($options{host_arch});
next if not $dep_simple->arch_is_concerned($options{host_arch});
}
+ $dep_simple->{restrictions} = undef if not $options{use_profiles};
+ if ($options{reduce_profiles}) {
+ $dep_simple->reduce_profiles($options{build_profiles});
+ next if not $dep_simple->profile_is_concerned($options{build_profiles});
+ }
push @or_list, $dep_simple;
}
next if not @or_list;
@@ -559,6 +592,7 @@ sub reset {
$self->{'version'} = undef;
$self->{'arches'} = undef;
$self->{'archqual'} = undef;
+ $self->{'restrictions'} = undef;
}
sub parse {
@@ -588,6 +622,11 @@ sub parse_string {
\s* (.*?) # don't parse architectures now
\s* \] # closing bracket
)? # end of optional architecture
+ (?: # start of optional restriction
+ \s* < # open bracket for restriction
+ \s* (.*?) # don't parse restrictions now
+ \s* > # closing bracket
+ )? # end of optional restriction
\s*$ # trailing spaces at end
/x;
if (defined($2)) {
@@ -602,6 +641,9 @@ sub parse_string {
if (defined($5)) {
$self->{arches} = [ split(/\s+/, $5) ];
}
+ if (defined($6)) {
+ $self->{restrictions} = [ map { lc } split /\s+/, $6 ];
+ }
}
sub output {
@@ -616,6 +658,9 @@ sub output {
if (defined($self->{'arches'})) {
$res .= " [" . join(" ", @{$self->{arches}}) . "]";
}
+ if (defined($self->{restrictions})) {
+ $res .= ' <' . join(' ', @{$self->{restrictions}}) . '>';
+ }
if (defined($fh)) {
print $fh $res;
}
@@ -745,6 +790,39 @@ sub has_arch_restriction {
}
}
+sub profile_is_concerned {
+ my ($self) = @_;
+
+ return 0 if not defined $self->{package}; # Empty dep
+ return 1 if not defined $self->{restrictions}; # Dep without restrictions
+
+ my $seen_profile = 0;
+ foreach my $restriction (@{$self->{restrictions}}) {
+ # Determine if this restriction is negated, and within the "profile"
+ # namespace, otherwise it does not concern this check.
+ next if $restriction !~ m/^(!)?profile\.(.*)/;
+
+ my $negated = defined $1 && $1 eq '!';
+
+ if ($negated) {
+ # "!profile.this" includes by default all other profiles
+ # unless they also appear in a "!profile.other".
+ $seen_profile = 1;
+ }
+ }
+ return $seen_profile;
+}
+
+sub reduce_profiles {
+ my ($self) = @_;
+
+ if (not $self->profile_is_concerned()) {
+ $self->reset();
+ } else {
+ $self->{restrictions} = undef;
+ }
+}
+
sub get_evaluation {
my ($self, $facts) = @_;
return undef if not defined $self->{package};
diff --git scripts/dpkg-checkbuilddeps.pl scripts/dpkg-checkbuilddeps.pl
index c203710..38c9bdf 100755
--- scripts/dpkg-checkbuilddeps.pl
+++ scripts/dpkg-checkbuilddeps.pl
@@ -113,11 +113,13 @@ my (@unmet, @conflicts);
if ($bd_value) {
push @unmet, build_depends('Build-Depends/Build-Depends-Arch/Build-Depends-Indep',
deps_parse($bd_value, build_dep => 1, host_arch => $host_arch,
+ reduce_profiles => 1,
reduce_arch => 1), $facts);
}
if ($bc_value) {
push @conflicts, build_conflicts('Build-Conflicts/Build-Conflicts-Arch/Build-Conflicts-Indep',
deps_parse($bc_value, build_dep => 1, host_arch => $host_arch,
+ reduce_profiles => 1,
reduce_arch => 1, union => 1), $facts);
}
diff --git scripts/dpkg-gencontrol.pl scripts/dpkg-gencontrol.pl
index 8df486e..b6ba7e2 100755
--- scripts/dpkg-gencontrol.pl
+++ scripts/dpkg-gencontrol.pl
@@ -247,6 +247,7 @@ $facts->add_installed_package($fields->{'Package'}, $fields->{'Version'},
$fields->{'Architecture'}, $fields->{'Multi-Arch'});
if (exists $pkg->{"Provides"}) {
my $provides = deps_parse($substvars->substvars($pkg->{"Provides"}, no_warn => 1),
+ reduce_profiles => 1,
reduce_arch => 1, union => 1);
if (defined $provides) {
foreach my $subdep ($provides->get_deps()) {
@@ -269,6 +270,7 @@ foreach my $field (field_list_pkg_dep()) {
msg_prefix => sprintf(_g("%s field of package %s: "), $field, $pkg->{Package}));
if (field_get_dep_type($field) eq 'normal') {
$dep = deps_parse($field_value, use_arch => 1,
+ reduce_profiles => 1,
reduce_arch => $reduce_arch);
error(_g("error occurred while parsing %s field: %s"), $field,
$field_value) unless defined $dep;
@@ -277,6 +279,7 @@ foreach my $field (field_list_pkg_dep()) {
push @seen_deps, $dep;
} else {
$dep = deps_parse($field_value, use_arch => 1,
+ reduce_profiles => 1,
reduce_arch => $reduce_arch, union => 1);
error(_g("error occurred while parsing %s field: %s"), $field,
$field_value) unless defined $dep;
diff --git scripts/dpkg-shlibdeps.pl scripts/dpkg-shlibdeps.pl
index e257369..1aa99d8 100755
--- scripts/dpkg-shlibdeps.pl
+++ scripts/dpkg-shlibdeps.pl
@@ -141,7 +141,7 @@ my $control = Dpkg::Control::Info->new();
my $fields = $control->get_source();
my $build_depends = defined($fields->{"Build-Depends"}) ?
$fields->{"Build-Depends"} : "";
-my $build_deps = deps_parse($build_depends, build_dep => 1, reduce_arch => 1);
+my $build_deps = deps_parse($build_depends, build_dep => 1, reduce_restrictions => 1);
error(_g("error occurred while parsing %s"), "Build-Depends") unless defined $build_deps;
my %dependencies;
diff --git apt-pkg/deb/deblistparser.cc apt-pkg/deb/deblistparser.cc
index b84bd6f..aa6e656 100644
--- apt-pkg/deb/deblistparser.cc
+++ apt-pkg/deb/deblistparser.cc
@@ -473,6 +473,12 @@ const char *debListParser::ParseDepends(const char *Start,const char *Stop,
string &Package,string &Ver,
unsigned int &Op, bool const &ParseArchFlags,
bool const &StripMultiArch)
+ { return ParseDepends(Start, Stop, Package, Ver, Op, ParseArchFlags, StripMultiArch, false); }
+const char *debListParser::ParseDepends(const char *Start,const char *Stop,
+ string &Package,string &Ver,
+ unsigned int &Op, bool const &ParseArchFlags,
+ bool const &StripMultiArch,
+ bool const &ParseRestrictionsList)
{
// Strip off leading space
for (;Start != Stop && isspace(*Start) != 0; Start++);
@@ -480,7 +486,8 @@ const char *debListParser::ParseDepends(const char *Start,const char *Stop,
// Parse off the package name
const char *I = Start;
for (;I != Stop && isspace(*I) == 0 && *I != '(' && *I != ')' &&
- *I != ',' && *I != '|' && *I != '[' && *I != ']'; I++);
+ *I != ',' && *I != '|' && *I != '[' && *I != ']' &&
+ *I != '<' && *I != '>'; ++I);
// Malformed, no '('
if (I != Stop && *I == ')')
@@ -597,6 +604,53 @@ const char *debListParser::ParseDepends(const char *Start,const char *Stop,
for (;I != Stop && isspace(*I) != 0; I++);
}
+ if (ParseRestrictionsList == true)
+ {
+ // Parse a restrictions list
+ if (I != Stop && *I == '<')
+ {
+ ++I;
+ // malformed
+ if (unlikely(I == Stop))
+ return 0;
+
+ const char *End = I;
+ bool Found = false;
+ bool NegRestriction = false;
+ while (I != Stop)
+ {
+ // look for whitespace or ending '>'
+ for (;End != Stop && !isspace(*End) && *End != '>'; ++End);
+
+ if (unlikely(End == Stop))
+ return 0;
+
+ if (*I == '!')
+ {
+ NegRestriction = true;
+ ++I;
+ }
+
+ if (*End++ == '>') {
+ I = End;
+ break;
+ }
+
+ I = End;
+ for (;I != Stop && isspace(*I) != 0; I++);
+ }
+
+ if (NegRestriction == true)
+ Found = !Found;
+
+ if (Found == false)
+ Package = ""; /* not for this restriction */
+ }
+
+ // Skip whitespace
+ for (;I != Stop && isspace(*I) != 0; I++);
+ }
+
if (I != Stop && *I == '|')
Op |= pkgCache::Dep::Or;
diff --git apt-pkg/deb/deblistparser.h apt-pkg/deb/deblistparser.h
index 386d291..e60a557 100644
--- apt-pkg/deb/deblistparser.h
+++ apt-pkg/deb/deblistparser.h
@@ -72,11 +72,16 @@ class debListParser : public pkgCacheGenerator::ListParser
bool LoadReleaseInfo(pkgCache::PkgFileIterator &FileI,FileFd &File,
std::string section);
-
+
static const char *ParseDepends(const char *Start,const char *Stop,
std::string &Package,std::string &Ver,unsigned int &Op,
bool const &ParseArchFlags = false,
bool const &StripMultiArch = true);
+ static const char *ParseDepends(const char *Start,const char *Stop,
+ std::string &Package,std::string &Ver,unsigned int &Op,
+ bool const &ParseArchFlags, bool const &StripMultiArch,
+ bool const &ParseRestrictionsList);
+
static const char *ConvertRelation(const char *I,unsigned int &Op);
debListParser(FileFd *File, std::string const &Arch = "");
diff --git apt-pkg/deb/debsrcrecords.cc apt-pkg/deb/debsrcrecords.cc
index ce55ccd..c6d39d8 100644
--- apt-pkg/deb/debsrcrecords.cc
+++ apt-pkg/deb/debsrcrecords.cc
@@ -90,7 +90,7 @@ bool debSrcRecordParser::BuildDepends(std::vector<pkgSrcRecords::Parser::BuildDe
while (1)
{
Start = debListParser::ParseDepends(Start, Stop,
- rec.Package,rec.Version,rec.Op,true, StripMultiArch);
+ rec.Package,rec.Version,rec.Op,true, StripMultiArch,true);
if (Start == 0)
return _error->Error("Problem parsing dependency: %s", fields[I]);
diff --git debian/changelog debian/changelog
index 4da43fd..7f35305 100644
--- debian/changelog
+++ debian/changelog
@@ -1,3 +1,16 @@
+apt (0.9.7.9+deb7u2) stable; urgency=medium
+
+ * Non-maintainer upload.
+ * Support for restriction list Build-Depends syntax extension [1]
+ while assuming that no build profile was set
+ - enables apt to understand the restriction list syntax which is
+ required for source packages with build profile annotations to
+ be uploaded because dak uses python-apt [2]
+ [1] https://wiki.debian.org/BuildProfileSpec
+ [2] https://bugs.debian.org/744246
+
+ -- Johannes Schauer <j.schauer@email.de> Tue, 29 Apr 2014 22:41:34 +0200
+
apt (0.9.7.9) stable; urgency=low
[ Ludovico Cavedon ]
diff --git debian/libapt-pkg4.12.symbols debian/libapt-pkg4.12.symbols
index bf42e8b..77e261f 100644
--- debian/libapt-pkg4.12.symbols
+++ debian/libapt-pkg4.12.symbols
@@ -369,6 +369,7 @@ libapt-pkg.so.4.12 libapt-pkg4.12 #MINVER#
(c++)"debListParser::VersionHash()@Base" 0.8.0
(c++)"debListParser::Architecture()@Base" 0.8.0
(c++)"debListParser::ParseDepends(char const*, char const*, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, unsigned int&, bool const&, bool const&)@Base" 0.8.0
+ (c++)"debListParser::ParseDepends(char const*, char const*, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, unsigned int&, bool const&, bool const&, bool const&)@Base" 0.9.7.9+deb7u2
(c++)"debListParser::ParseDepends(pkgCache::VerIterator&, char const*, unsigned int)@Base" 0.8.0
(c++)"debListParser::ParseProvides(pkgCache::VerIterator&)@Base" 0.8.0
(c++)"debListParser::ArchitectureAll()@Base" 0.8.0
diff --git debian/changelog debian/changelog
index 85b7e3b..5ffa35a 100644
--- debian/changelog
+++ debian/changelog
@@ -1,3 +1,14 @@
+python-apt (0.8.8.2+nmu1) stable; urgency=medium
+
+ * Non-maintainer upload.
+ * Use new libapt-pkg functionality to understand the restriction
+ list syntax [1]. This is necessary for dak [2] to understand uploaded
+ packages with build profiles.
+ [1] https://wiki.debian.org/BuildProfileSpec
+ [2] https://bugs.debian.org/744246
+
+ -- Johannes Schauer <j.schauer@email.de> Wed, 30 Apr 2014 09:20:03 +0200
+
python-apt (0.8.8.2) unstable; urgency=low
[ David Prévot ]
diff --git debian/control debian/control
index 1619aac..b5b893c 100644
--- debian/control
+++ debian/control
@@ -10,7 +10,7 @@ Build-Depends: apt (>= 0.9.6),
apt-utils,
debhelper (>= 7.3.5),
fakeroot,
- libapt-pkg-dev (>= 0.8.11),
+ libapt-pkg-dev (>= 0.9.7.9+deb7u2),
python-all-dev (>= 2.6.6-3~),
python-all-dbg,
python3-all-dev (>= 3.1.2-10~),
diff --git python/apt_pkgmodule.cc python/apt_pkgmodule.cc
index 7704aa0..70d2eee 100644
--- python/apt_pkgmodule.cc
+++ python/apt_pkgmodule.cc
@@ -186,7 +186,7 @@ static const char *parse_src_depends_doc =
"only contains those dependencies for the architecture set in the\n"
"configuration variable APT::Architecture";
static PyObject *RealParseDepends(PyObject *Self,PyObject *Args,
- bool ParseArchFlags, std::string name,
+ bool ParseArchFlags, bool ParseRestrictionsList, std::string name,
bool debStyle=false)
{
std::string Package;
@@ -210,7 +210,7 @@ static PyObject *RealParseDepends(PyObject *Self,PyObject *Args,
break;
Start = debListParser::ParseDepends(Start,Stop,Package,Version,Op,
- ParseArchFlags, StripMultiArch);
+ ParseArchFlags, StripMultiArch, ParseRestrictionsList);
if (Start == 0)
{
PyErr_SetString(PyExc_ValueError,"Problem Parsing Dependency");
@@ -243,11 +243,11 @@ static PyObject *RealParseDepends(PyObject *Self,PyObject *Args,
}
static PyObject *ParseDepends(PyObject *Self,PyObject *Args)
{
- return RealParseDepends(Self, Args, false, "parse_depends");
+ return RealParseDepends(Self, Args, false, false, "parse_depends");
}
static PyObject *ParseSrcDepends(PyObject *Self,PyObject *Args)
{
- return RealParseDepends(Self, Args, true, "parse_src_depends");
+ return RealParseDepends(Self, Args, true, true, "parse_src_depends");
}
#ifdef COMPAT_0_7
static PyObject *ParseDepends_old(PyObject *Self,PyObject *Args)
Reply to: