Bug#740607: lintian: Please support build-profiles
Hi,
Quoting Jakub Wilk (2014-03-19 12:01:56)
> >In that case, can you give me an example of a tag that checks
> >debian/control?
>
> binary-control-field-duplicates-source
thanks! That was very helpful :)
please find attached a new patch with one more added tag.
All three students forgot to add the build-profiles field for their patches so
it seems something one easily forgets.
cheers, josch
From 878e6fdeeeeaed3a1f88e58d3c4d619af576b486 Mon Sep 17 00:00:00 2001
From: josch <j.schauer@email.de>
Date: Mon, 3 Mar 2014 15:58:28 +0100
Subject: [PATCH] Build-profiles support
- added 3 new tags to detect errors in restriction list syntax
* invalid-restriction-term-in-source-relation
* invalid-restriction-namespace-in-source-relation
* invalid-restriction-label-in-source-relation
- added 4 new tags to ensure that if restrictions lists are used, a
versioned build dependency on dpkg-dev and (if applicable) debhelper
is added and no conflicts with them exist
* restriction-list-without-versioned-dpkg-dev-dependency
* restriction-list-with-versioned-dpkg-dev-conflict
* restriction-list-with-debhelper-without-debhelper-version
* restriction-list-with-debhelper-with-conflicting-debhelper-version
- added one new tag to detect a missing build-profiles field in binary
stanzas for build profiles stage1 and stage2
* stageX-profile-used-but-no-binary-package-dropped
- added data/fields/dependency-restrictions and a parser to keep record
of valid namespaces and labels for restriction lists
---
checks/control-file.pm | 46 +++++++++++++
checks/fields.desc | 62 +++++++++++++++++
checks/fields.pm | 80 +++++++++++++++++++---
data/fields/dependency-restrictions | 4 ++
lib/Lintian/Relation.pm | 47 +++++++++----
.../debian/debian/control.in | 7 +-
t/tests/fields-build-depends-general/desc | 8 +++
t/tests/fields-build-depends-general/tags | 8 +++
8 files changed, 239 insertions(+), 23 deletions(-)
create mode 100644 data/fields/dependency-restrictions
diff --git a/checks/control-file.pm b/checks/control-file.pm
index 2f16f31..0f9064c 100644
--- a/checks/control-file.pm
+++ b/checks/control-file.pm
@@ -292,6 +292,52 @@ sub run {
}
}
+ # check the Build-Profiles field
+ # this has to checked here because the Build-Profiles field does not appear
+ # in DEBIAN/control and even if it should in the future, some binary
+ # packages might never be built in the first place because of build
+ # profiles
+
+ # check which profile names are supposedly supported according to the build
+ # dependencies
+ my %used_profiles=();
+ for my $field (
+ qw(build-depends build-depends-indep build-conflicts build-conflicts-indep)
+ ) {
+ if (defined $info->source_field($field)) {
+ for my $dep (split /\s*,\s*/, $info->source_field($field)) {
+ for my $alt (split /\s*\|\s*/, $dep) {
+ while ($alt =~ /<([^>]+)>/g) {
+ for my $restr (split /\s+/, $1) {
+ if ($restr =~ m/^!?profile\.(.*)/) {
+ $used_profiles{$1} = 0;
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+
+ # find those packages that do not get built because of a certain build
+ # profile
+ for my $bin (@package_names) {
+ my $raw = $info->binary_field($bin, "build-profiles");
+ next unless $raw;
+ for my $prof (split /\s+/, $raw) {
+ if ($prof =~ s/^!//) {
+ $used_profiles{$prof} = 1;
+ }
+ }
+ }
+
+ # find out if the developer forgot to mark binary packages as not being
+ # built
+ while (my ($k, $v) = each(%used_profiles)) {
+ tag 'stageX-profile-used-but-no-binary-package-dropped'
+ if (($k eq "stage1" || $k eq "stage2") && $v == 0);
+ }
+
return;
}
diff --git a/checks/fields.desc b/checks/fields.desc
index d55c11a..97e17b9 100644
--- a/checks/fields.desc
+++ b/checks/fields.desc
@@ -637,6 +637,68 @@ Info: The architecture string in this source relation has some
negated. This is not permitted by Policy. Either all architectures must
be negated or none of them may be.
+Tag: invalid-restriction-term-in-source-relation
+Severity: important
+Certainty: certain
+Info: The restriction list in the source relation includes a term which
+ does not contain exactly one dot separating restriction namespace and label.
+ A term in a restriction list is of the form "namespace.label" (without the
+ quotes).
+
+Tag: invalid-restriction-namespace-in-source-relation
+Severity: important
+Certainty: possible
+Info: The restriction list in the source relation includes a term with
+ an unknown namespace. The only allowed namespace is "profile" (without the
+ quotes).
+
+Tag: invalid-restriction-label-in-source-relation
+Severity: important
+Certainty: possible
+Info: The restriction list in the source relation includes a term with
+ an unknown label. The only allowed labels are "stage1", "stage2", "notest"
+ and "cross".
+
+Tag: restriction-list-without-versioned-dpkg-dev-dependency
+Severity: normal
+Certainty: certain
+Info: If a restriction list appears in the build dependencies, then the
+ source package has to build depend on dpkg-dev (>= 1.17.2) for minimal
+ restriction list support.
+
+Tag: restriction-list-with-versioned-dpkg-dev-conflict
+Severity: normal
+Certainty: certain
+Info: If a restriction list appears in the build dependencies, then the
+ source package has to build depend on dpkg-dev (>= 1.17.2) for minimal
+ restriction list support. It must not conflict with version 1.17.2.
+
+Tag: restriction-list-with-debhelper-without-debhelper-version
+Severity: normal
+Certainty: certain
+Info: If a restriction list appears in the build dependencies and the
+ package uses debhelper, then the source package has to depend on at least
+ debhelper 9.20140227.
+
+Tag: restriction-list-with-debhelper-with-conflicting-debhelper-version
+Severity: normal
+Certainty: certain
+Info: If a restriction list appears in the build dependencies and the
+ package uses debhelper, then the source package has to depend on at least
+ debhelper 9.20140227. It must not conflict with version 9.20140227.
+
+Tag: stageX-profile-used-but-no-binary-package-dropped
+Severity: normal
+Certainty: certain
+Info: You used a stage1 or stage2 build profile restriction in the build
+ dependencies but you did not mark any binary packages as not being built with
+ the used profile activated. Using a stage1 or stage2 build profile restriction
+ means that you intend to change the build process in a way such that some of
+ the build results will be different or not generated at all. All binary
+ packages which would provide different functionality, would be empty or not be
+ built at all under the stage1 or stage2 profiles must be marked as not being
+ generated with the Build-Profiles field.
+
Tag: depends-on-build-essential-package-without-using-version
Severity: important
Certainty: certain
diff --git a/checks/fields.pm b/checks/fields.pm
index aeee252..12f2dbb 100644
--- a/checks/fields.pm
+++ b/checks/fields.pm
@@ -46,6 +46,9 @@ our $known_build_essential
= Lintian::Data->new('fields/build-essential-packages');
our $KNOWN_BINARY_FIELDS = Lintian::Data->new('fields/binary-fields');
our $KNOWN_UDEB_FIELDS = Lintian::Data->new('fields/udeb-fields');
+our $KNOWN_DEPENDENCY_RESTRICTIONS
+ = Lintian::Data->new('fields/dependency-restrictions',
+ qr/\./, \&_load_dependency_restrictions);
our %KNOWN_ARCHIVE_PARTS = map { $_ => 1 } ('non-free', 'contrib');
@@ -730,7 +733,7 @@ sub run {
&& $known_obsolete_emacs{$alternatives[0]->[0]});
for my $part_d (@alternatives) {
- my ($d_pkg, $d_march, $d_version, $d_arch, $rest,
+ my ($d_pkg, $d_march, $d_version, $d_arch, undef, $rest,
$part_d_orig)
= @$part_d;
@@ -935,6 +938,7 @@ sub run {
any { $_ eq $_[0] } qw(build-depends build-depends-indep);
};
+ my $restrictions_used = 0;
my %depend;
for my $field (
qw(build-depends build-depends-indep build-conflicts build-conflicts-indep)
@@ -956,7 +960,7 @@ sub run {
&& &$is_dep_field($field));
for my $part_d (@alternatives) {
- my ($d_pkg, $d_march, $d_version, $d_arch, $rest,
+ my ($d_pkg, $d_march, $d_version, $d_arch, $d_restr, $rest,
$part_d_orig)
= @$part_d;
@@ -968,6 +972,30 @@ sub run {
}
}
+ if (scalar @{$d_restr} >= 1) {
+ $restrictions_used = 1;
+ }
+
+ for my $restr (@{$d_restr}) {
+ my $dotcount = $restr =~ tr/.//;
+ if ($dotcount != 1) {
+ tag 'invalid-restriction-term-in-source-relation',
+ "$restr [$field: $part_d_orig]";
+ next;
+ }
+ $restr =~ s/^!//;
+ my ($ns, $label) = split(/\./, $restr, 2);
+ if ($KNOWN_DEPENDENCY_RESTRICTIONS->known($ns)) {
+ tag 'invalid-restriction-label-in-source-relation',
+ "$label [$field: $part_d_orig]"
+ unless any { $_ eq $label }
+ @{$KNOWN_DEPENDENCY_RESTRICTIONS->value($ns)};
+ } else {
+ tag 'invalid-restriction-namespace-in-source-relation',
+ "$ns [$field: $part_d_orig]";
+ }
+ }
+
if ( $d_pkg =~ m/^openjdk-\d+-doc$/o
or $d_pkg eq 'classpath-doc'){
tag 'build-depends-on-specific-java-doc-package',
@@ -1077,6 +1105,24 @@ sub run {
}
}
+ # if restrictions are found in the build-depends/conflicts, then
+ # package must build-depend on dpkg (>= 1.17.2)
+ if ($restrictions_used) {
+ my $build_conflicts_all = $info->relation('build-conflicts-all');
+ tag 'restriction-list-without-versioned-dpkg-dev-dependency'
+ unless ($build_all->implies('dpkg-dev (>= 1.17.2)'));
+ tag 'restriction-list-with-versioned-dpkg-dev-conflict'
+ if ($build_conflicts_all->implies_inverse('dpkg-dev (<< 1.17.2)'));
+ # if the package uses debhelper then it must require and not
+ # conflict with version >= 9.20140227
+ if ($build_all->implies('debhelper')) {
+ tag 'restriction-list-with-debhelper-without-debhelper-version'
+ unless ($build_all->implies('debhelper (>= 9.20140227)'));
+ tag 'restriction-list-with-debhelper-with-conflicting-debhelper-version'
+ if ($build_conflicts_all->implies_inverse('debhelper (<< 9.20140227)'));
+ }
+ }
+
my (@arch_dep_pkgs, @dbg_pkgs);
foreach my $gproc ($group->get_binary_processables) {
my $binpkg = $gproc->pkg_name;
@@ -1238,16 +1284,16 @@ sub run {
return;
}
-# splits "foo:bar (>= 1.2.3) [!i386 ia64]" into
-# ( "foo", "bar", [ ">=", "1.2.3" ], [ [ "i386", "ia64" ], 1 ], "" )
-# ^^^ ^^
-# count of negated arches, if ! was given ||
-# rest (should always be "" for valid dependencies)
+# splits "foo:bar (>= 1.2.3) [!i386 ia64] <!profile.stage1 !profile.notest>" into
+# ( "foo", "bar", [ ">=", "1.2.3" ], [ [ "i386", "ia64" ], 1 ], [ "!profile.stage1" "!profile.notest" ], "" )
+# ^^^ ^^
+# count of negated arches, if ! was given ||
+# rest (should always be "" for valid dependencies)
sub _split_dep {
my $dep = shift;
- my ($pkg, $dmarch, $version, $darch) = ('', '', ['',''], [[], 0]);
+ my ($pkg, $dmarch, $version, $darch, $restr) = ('', '', ['',''], [[], 0], []);
- my $pkgname = $1 if $dep =~ s/^\s*([^\s\[\(]+)\s*//;
+ my $pkgname = $1 if $dep =~ s/^\s*([^<\s\[\(]+)\s*//;
if (defined $pkgname) {
($pkg, $dmarch) = split /:/, $pkgname, 2;
$dmarch //= ''; # Ensure it is defined (in case there is no ":")
@@ -1267,9 +1313,23 @@ sub _split_dep {
}
$darch->[1] = $negated;
}
+ if ($dep && $dep =~ s/\s*<([^>]+)>\s*//) {
+ my $t = $1;
+ $restr = [split /\s+/, $t];
+ }
}
- return ($pkg, $dmarch, $version, $darch, $dep);
+ return ($pkg, $dmarch, $version, $darch, $restr, $dep);
+}
+
+sub _load_dependency_restrictions {
+ my ($key, $value, $pval) = @_;
+ my $ret = undef;
+ if (not defined $pval) {
+ $ret = $pval = [];
+ }
+ push @{$pval}, $value;
+ return $ret;
}
sub perl_core_has_version {
diff --git a/data/fields/dependency-restrictions b/data/fields/dependency-restrictions
new file mode 100644
index 0000000..270be3c
--- /dev/null
+++ b/data/fields/dependency-restrictions
@@ -0,0 +1,4 @@
+profile.notest
+profile.stage1
+profile.stage2
+profile.cross
diff --git a/lib/Lintian/Relation.pm b/lib/Lintian/Relation.pm
index 1ba783c..6d72176 100644
--- a/lib/Lintian/Relation.pm
+++ b/lib/Lintian/Relation.pm
@@ -115,21 +115,26 @@ sub parse_element {
\s* (.*?) # architectures (5)
\s* \] # closing bracket
)? # end of optional architecture
+ (?: # start of optional restriction
+ \s* < # open bracket for restriction
+ \s* (.*?) # don't parse restrictions now
+ \s* > # closing bracket
+ )? # end of optional restriction
/x;
- my ($pkgname, $march, $relop, $relver, $bdarch) = ($1, $2, $3, $4, $5);
+ my ($pkgname, $march, $relop, $relver, $bdarch, $restr) = ($1, $2, $3, $4, $5, $6);
my @array;
if (not defined($relop)) {
# If there's no version, we don't need to do any further processing.
# Otherwise, convert the legacy < and > relations to the current ones.
- @array = ('PRED', $pkgname, undef, undef, $bdarch, $march);
+ @array = ('PRED', $pkgname, undef, undef, $bdarch, $march, $restr);
} else {
if ($relop eq '<') {
$relop = '<<';
} elsif ($relop eq '>') {
$relop = '>>';
}
- @array = ('PRED', $pkgname, $relop, $relver, $bdarch, $march);
+ @array = ('PRED', $pkgname, $relop, $relver, $bdarch, $march, $restr);
}
# Optimise the memory usage of the array. Understanding this
@@ -191,26 +196,37 @@ sub new {
return $self;
}
-=item new_noarch(RELATION)
+=item new_norestriction(RELATION)
Creates a new Lintian::Relation object corresponding to the parsed
-relationship RELATION, ignoring architecture restrictions. This should be
-used in cases where we only care if a dependency is present in some cases
-and we don't want to require that the architectures match (such as when
-checking for proper build dependencies, since if there are architecture
-constraints the maintainer is doing something beyond Lintian's ability to
-analyze). RELATION may be C<undef> or the empty string, in which case the
-returned Lintian::Relation object is empty (always satisfied).
+relationship RELATION, ignoring architecture restrictions and restriction
+lists. This should be used in cases where we only care if a dependency is
+present in some cases and we don't want to require that the architectures
+match (such as when checking for proper build dependencies, since if there
+are architecture constraints the maintainer is doing something beyond
+Lintian's ability to analyze) or that the restrictions list match (Lintian
+can't handle dependency implications with build profiles yet). RELATION
+may be C<undef> or the empty string, in which case the returned
+Lintian::Relation object is empty (always satisfied).
=cut
-sub new_noarch {
+sub new_norestriction {
my ($class, $relation) = @_;
$relation = '' unless defined($relation);
$relation =~ s/\[[^\]]*\]//g;
+ $relation =~ s/<[^>]*>//g;
return $class->new($relation);
}
+=item new_noarch(RELATION)
+
+An alias for new_norestriction.
+
+=cut
+
+*new_noarch = \&new_norestriction;
+
=item and(RELATION, ...)
Creates a new Lintian::Relation object produced by AND'ing all the
@@ -351,6 +367,13 @@ sub implies_element {
$$q[1] = '' unless defined $$q[1];
return if $$p[1] ne $$q[1];
+ # Since the restriction list is not a set (as the architecture list) there
+ # is no way to calculate a superset or subset of one another. Furthermore,
+ # the evaluation depends on which build profiles are currently activated.
+ # With n being the number of possible build profiles, 2^n checks would
+ # have to be done. We decide not to do that (yet).
+ return if defined $$p[6] or defined $$q[6];
+
# If the names match, then the only difference is in the architecture or
# version clauses. First, check architecture. The architectures for p
# must be a superset of the architectures for q.
diff --git a/t/tests/fields-build-depends-general/debian/debian/control.in b/t/tests/fields-build-depends-general/debian/debian/control.in
index f6546b9..80222ef 100644
--- a/t/tests/fields-build-depends-general/debian/debian/control.in
+++ b/t/tests/fields-build-depends-general/debian/debian/control.in
@@ -8,8 +8,13 @@ Build-Depends: debhelper (>= 9), bd-conflict, revision-1 (>= 1.0-1),
xorg-dev, java-propose-classpath, python3.2-dev, foo [all],
bar [i386 any], baz [source i3!86], baz [i386 !amd64],
other-pkg [kfreebsd-any], yet-another [any-powerpc],
+ big <profile.stage1>, bpfail1 <foobar>,
+ bpfail2 <foo.bar>, bpfail3 <profile.bar>,
packaging-dev, libdb5.1++-dev, libdb5.1-java-dev
-Build-Conflicts: bd-conflict
+Build-Conflicts:
+ bd-conflict,
+ dpkg-dev (>= 1.17.2),
+ debhelper (>= 9.20140227)
Package: {$source}
Architecture: {$architecture}
diff --git a/t/tests/fields-build-depends-general/desc b/t/tests/fields-build-depends-general/desc
index b8b7a1b..96c2918 100644
--- a/t/tests/fields-build-depends-general/desc
+++ b/t/tests/fields-build-depends-general/desc
@@ -16,5 +16,13 @@ Test-For:
depends-on-build-essential-package-without-using-version
depends-on-packaging-dev
invalid-arch-string-in-source-relation
+ invalid-restriction-label-in-source-relation
+ invalid-restriction-namespace-in-source-relation
+ invalid-restriction-term-in-source-relation
ored-build-depends-on-obsolete-package
+ restriction-list-with-debhelper-with-conflicting-debhelper-version
+ restriction-list-with-debhelper-without-debhelper-version
+ restriction-list-with-versioned-dpkg-dev-conflict
+ restriction-list-without-versioned-dpkg-dev-dependency
+ stageX-profile-used-but-no-binary-package-dropped
References: Debian Bug#540594, Debian Bug#551793
diff --git a/t/tests/fields-build-depends-general/tags b/t/tests/fields-build-depends-general/tags
index 0c0222f..5c70314 100644
--- a/t/tests/fields-build-depends-general/tags
+++ b/t/tests/fields-build-depends-general/tags
@@ -9,9 +9,17 @@ E: fields-build-depends-general source: depends-on-build-essential-package-witho
E: fields-build-depends-general source: invalid-arch-string-in-source-relation all [build-depends: foo [all]]
E: fields-build-depends-general source: invalid-arch-string-in-source-relation i3!86 [build-depends: baz [source i3!86]]
E: fields-build-depends-general source: invalid-arch-string-in-source-relation source [build-depends: baz [source i3!86]]
+E: fields-build-depends-general source: invalid-restriction-label-in-source-relation bar [build-depends: bpfail3 <profile.bar>]
+E: fields-build-depends-general source: invalid-restriction-namespace-in-source-relation foo [build-depends: bpfail2 <foo.bar>]
+E: fields-build-depends-general source: invalid-restriction-term-in-source-relation foobar [build-depends: bpfail1 <foobar>]
I: fields-build-depends-general source: build-depends-on-python-dev-with-no-arch-any
I: fields-build-depends-general source: ored-build-depends-on-obsolete-package build-depends: xlibmesa-gl-dev
W: fields-build-depends-general source: build-depends-on-1-revision build-depends: revision-1 (>= 1.0-1)
W: fields-build-depends-general source: build-depends-on-versioned-berkeley-db build-depends:libdb5.1++-dev
W: fields-build-depends-general source: build-depends-on-versioned-berkeley-db build-depends:libdb5.1-java-dev
W: fields-build-depends-general source: depends-on-packaging-dev build-depends
+W: fields-build-depends-general source: restriction-list-with-debhelper-with-conflicting-debhelper-version
+W: fields-build-depends-general source: restriction-list-with-debhelper-without-debhelper-version
+W: fields-build-depends-general source: restriction-list-with-versioned-dpkg-dev-conflict
+W: fields-build-depends-general source: restriction-list-without-versioned-dpkg-dev-dependency
+W: fields-build-depends-general source: stageX-profile-used-but-no-binary-package-dropped
--
1.8.5.3
Reply to: