Re: Your help with QA is needed
Hi Josip!
You wrote:
> > > Anyway, I'd love to see daily lintian runs on stable again. Joy is
> > > currently trying to get the lintian lab running again. If you want to
> > > help, joy@debian.org
> > What exactly is broken about the lintian runs right now? How can I help
> > getting them working again?
> The biggest issue is that the function that gets the names of the
> source/binary packages does 'find <mirrordir>/dists/foo -type f -name
> *.dsc' to locate those files, which of course fails with the pool. We need
> to parse the Sources file, I think.
I finally have had time to take a look at this. I've hacked lintian to
parse the Packages and Sources files instead of doing a find (diff to
cvs version is attached). I've introduced a new option
"LINTAIN_SECTION", which can be used to tell intian which section of the
archive (e.g. sid/contrib) to check.
I've tested these changes on auric with the following environment
settings:
LINTIAN_LAB=/tmp/lintianlab.test.bas
LINTIAN_DIST=/debian/ftp/
LINTIAN_SECTION=sid/non-free
LINTIAN_ARCH=sparc
(note that this doesn't only test the "sparc" architecture, but rather
all files in the sparc Packages file, so it also checks the "all" arch).
I've tested the creation of the laboratory as well as the checking of
individual packages. I didn't test the "check all packages" feature,
because I didn't want to generate too much load on auric.
There is now only the problem that with this patch, lintian can only set
up a lab for one section of the archive. For example, it's not possible
to set up a lab for the entire sid distibution (including man, contrib
and non-free), but only for individual section. This should be rather
easy to solve (so that you could for example set LINTIAN_SECTION=sid/*).
I'll see if I have time to look at this later this week.
I hope this helps in getting the daily lintian runs back online again
soon.
--
Kind regards,
+---------------------------------------------------------------+
| Bas Zoetekouw | Si l'on sait exactement ce |
|--------------------------------| que l'on va faire, a quoi |
| zoetekw@phys.uu.nl | bon le faire? |
| bas@A-Es2.uu.nl | Pablo Picasso |
+---------------------------------------------------------------+
diff -Naur lintian-cvs20000430/frontend/lintian lintian.new/frontend/lintian
--- lintian-cvs20000430/frontend/lintian Fri Dec 22 15:56:14 2000
+++ lintian.new/frontend/lintian Mon Apr 30 11:57:49 2001
@@ -52,6 +52,7 @@
$OPT_LINTIAN_LAB = ""; #string for the --lab option
$OPT_LINTIAN_DIST = ""; #string for the --dist option
$OPT_LINTIAN_ARCH = ""; #string for the --arch option
+$OPT_LINTIAN_SECTION = ""; #string for the --release option
# These options can also be used via default or environment variables
$LINTIAN_CFG = ""; #config file to use
$LINTIAN_ROOT = ""; #location of the lintian modules
@@ -124,7 +125,8 @@
--cfg CONFIGFILE read CONFIGFILE for configuration
--lab LABDIR use LABDIR as permanent laboratory
--dist DISTDIR location of Debian archive to scan for packages
- --arch ARCH scan only packages with architecture ARCH
+ --section RELEASE scan packages in this section (e.g. sid/main)
+ --arch ARCH scan packages with architecture ARCH
--root ROOTDIR use ROOTDIR instead of /usr/share/lintian
Package selection options:
-a, --all process all packages in distribution
@@ -215,6 +217,7 @@
"cfg=s" => \$LINTIAN_CFG,
"lab=s" => \$OPT_LINTIAN_LAB,
"dist=s" => \$OPT_LINTIAN_DIST,
+ "section=s" => \$OPT_LINTIAN_SECTION,
"arch=s" => \$OPT_LINTIAN_ARCH,
"root=s" => \$LINTIAN_ROOT,
@@ -281,6 +284,7 @@
undef $LINTIAN_LAB;
undef $LINTIAN_DIST;
undef $LINTIAN_UNPACK_LEVEL;
+undef $LINTIAN_SECTION;
undef $LINTIAN_ARCH;
# read configuration file
@@ -303,6 +307,8 @@
$LINTIAN_DIST = $1;
} elsif (/^\s*LINTIAN_UNPACK_LEVEL\s*=\s*(.*\S)\s*$/i) {
$LINTIAN_UNPACK_LEVEL = $1;
+ } elsif (/^\s*LINTIAN_SECTION\s*=\s*(.*\S)\s*$/i) {
+ $LINTIAN_SECTION = $1;
} elsif (/^\s*LINTIAN_ARCH\s*=\s*(.*\S)\s*$/i) {
$LINTIAN_ARCH = $1;
} else {
@@ -316,11 +322,13 @@
$LINTIAN_LAB = $ENV{'LINTIAN_LAB'} if $ENV{'LINTIAN_LAB'};
$LINTIAN_DIST = $ENV{'LINTIAN_DIST'} if $ENV{'LINTIAN_DIST'};
$LINTIAN_UNPACK_LEVEL = $ENV{'LINTIAN_UNPACK_LEVEL'} if $ENV{'LINTIAN_UNPACK_LEVEL'};
+$LINTIAN_SECTION = $ENV{'LINTIAN_SECTION'} if $ENV{'LINTIAN_SECTION'};
$LINTIAN_ARCH = $ENV{'LINTIAN_ARCH'} if $ENV{'LINTIAN_ARCH'};
# command-line options override everything
$LINTIAN_LAB = $OPT_LINTIAN_LAB if $OPT_LINTIAN_LAB;
$LINTIAN_DIST = $OPT_LINTIAN_DIST if $OPT_LINTIAN_DIST;
+$LINTIAN_SECTION = $OPT_LINTIAN_SECTION if $OPT_LINTIAN_SECTION;
$LINTIAN_ARCH = $OPT_LINTIAN_ARCH if $OPT_LINTIAN_ARCH;
# LINTIAN_ARCH must have a value.
@@ -338,6 +346,7 @@
$ENV{'LINTIAN_LAB'} = $LINTIAN_LAB;
$ENV{'LINTIAN_DIST'} = $LINTIAN_DIST;
$ENV{'LINTIAN_UNPACK_LEVEL'} = $LINTIAN_UNPACK_LEVEL;
+$ENV{'LINTIAN_SECTION'} = $LINTIAN_SECTION;
$ENV{'LINTIAN_ARCH'} = $LINTIAN_ARCH;
#----------------------------------------------------------------------------
diff -Naur lintian-cvs20000430/unpack/list-binpkg lintian.new/unpack/list-binpkg
--- lintian-cvs20000430/unpack/list-binpkg Sat Mar 13 06:10:10 1999
+++ lintian.new/unpack/list-binpkg Mon Apr 30 11:57:12 2001
@@ -61,6 +61,7 @@
$LINTIAN_DIST = $ENV{'LINTIAN_DIST'};
$LINTIAN_ARCH = $ENV{'LINTIAN_ARCH'};
$LINTIAN_LAB = $ENV{'LINTIAN_LAB'};
+$LINTIAN_SECTION = $ENV{'LINTIAN_SECTION'};
# read old list file (this command does nothing if the file does not exist)
read_bin_list($output_file,1);
@@ -75,39 +76,22 @@
or fail("cannot open list file $output_file for writing: $!");
print OUT "$BINLIST_FORMAT\n";
-# run find to get list of packages
-print "N: Searching for .deb's in directory $LINTIAN_DIST ...\n" if $verbose;
-open(IN,"cd $LINTIAN_DIST; find . -name \"*.deb\" \\( -type f -o -type l \\) |")
- or fail("cannot open input pipe: $!");
-
-while (<IN>) {
- chop;
-
- my $deb_file = $_;
- my ($arch, $pkgdata);
-
- undef $pkgdata;
-
- # Get architecture. There are three sources for this information:
- if (m,/binary-([^/]+)/,) {
- # It's in the directory name
- $arch = $1;
- } elsif (m,/[^/]+_[^/]+_([^/]+).deb$,) {
- # It's in the filename
- $arch = $1;
- } else {
- # Look in the deb file
- $pkgdata = &safe_get_deb_info($deb_file);
- next if not defined $pkgdata;
- $arch = $pkgdata->{'architecture'};
- }
-
- # correct architecture?
- next unless $arch eq $LINTIAN_ARCH;
+# parse Packages file to get list of packages
+my $packages = "$LINTIAN_DIST/dists/$LINTIAN_SECTION/".
+ "binary-$LINTIAN_ARCH/Packages";
+print "N: Parsing $packages ...\n" if $verbose;
+open(IN,"$packages") or fail("cannot open Packages file: $!");
+
+while (!eof(IN)) {
+ do { $line = <IN> } until ($line =~ m/^Architecture: (.*)$/m);
+ my $arch = $1;
+ do { $line = <IN> } until ($line =~ m/^Filename: (.*)$/m);
+ my $deb_file = $1;
+ do { $line = <IN> } until ($line =~ m/^\s*$/m);
# get timestamp...
unless (@stat = stat "$LINTIAN_DIST/$deb_file") {
- print "E: general: cannot-stat $deb_file\n";
+ print "E: general: cannot-stat $LINTIAN_DIST/$deb_file\n";
next;
}
my $timestamp = $stat[9];
diff -Naur lintian-cvs20000430/unpack/list-srcpkg lintian.new/unpack/list-srcpkg
--- lintian-cvs20000430/unpack/list-srcpkg Sat Mar 13 06:10:10 1999
+++ lintian.new/unpack/list-srcpkg Mon Apr 30 11:31:42 2001
@@ -19,7 +19,6 @@
# Software Foundation, Inc., 59 Temple Place - Suite 330, Boston,
# MA 02111-1307, USA.
-
# turn file buffering off:
$| = 1;
@@ -58,6 +57,7 @@
# get variables out of environment
$LINTIAN_DIST = $ENV{'LINTIAN_DIST'};
$LINTIAN_LAB = $ENV{'LINTIAN_LAB'};
+$LINTIAN_SECTION = $ENV{'LINTIAN_SECTION'};
# read old list file (this command does nothing if the file does not exist)
read_src_list($output_file,1);
@@ -71,15 +71,18 @@
open(OUT,">$output_file") or fail("cannot open list file $output_file for writing: $!");
print OUT "$SRCLIST_FORMAT\n";
-# run find to get list of packages
-print "N: Searching for .dsc's in directory $LINTIAN_DIST ...\n" if $verbose;
-open(IN,"cd $LINTIAN_DIST; find . -name \"*.dsc\" |") or fail("cannot open input pipe: $!");
-
-while (<IN>) {
- chop;
-
- $dsc_file = $_;
-
+# parse Sources.gz to get list of packages
+my $sources = "$LINTIAN_DIST/dists/$LINTIAN_SECTION/source/Sources.gz";
+print "N: Parsing $sources ...\n" if $verbose;
+open(IN,"zcat $sources|") or fail("Cannot open input pipe: $!");
+
+while (!eof(IN)) {
+ do { $line = <IN> } until ($line =~ m/^Directory: (.*)$/m);
+ my $pkg_dir = $1;
+ do { $line = <IN> } until ($line =~ m/^ [0-9a-f]{32} [0-9]+ (.+\.dsc)$/m);
+ my $dsc_file = "$pkg_dir/$1";
+ do { $line = <IN> } until ($line =~ m/^\s*$/m);
+
# get timestamp...
unless (@stat = stat "$LINTIAN_DIST/$dsc_file") {
print STDERR "error: cannot stat file $dsc_file: $!\n";
Reply to: