Stats and history of archive-wide Lintian runs
Hi,
You may still remember that a while ago I suggested visualizing the
evolution of Lintian tags[1], but the initial proposal wasn't entirely
integrated into lintian/reporting.
I'm now attaching a patch to optionally store statistics of archive-wide
Lintian runs as part of html_reports (it has been sitting in my working
copy for months, I almost forgot about it).
I don't know how exactly lintian.debian.org is set up, so I would like
to get some feedback to make sure the patch is good enough before
committing. If this patch is included, the idea is to store stats for a
few weeks and then, if everything works fine, start generating the
graphs.
1. http://lists.debian.org/debian-lint-maint/2009/06/msg00278.html
>From e967e425d879be9ce9fa7d808b11b150d59e4fc2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jord=C3=A0=20Polo?= <jorda@ettin.org>
Date: Tue, 5 Jan 2010 20:22:21 +0100
Subject: [PATCH] Keep a record of the stats of previous executions
This change is mostly intended to graph the evolution of tags on
lintian.debian.org. It is currently disabled by default since it isn't
necessarily useful for everyone, but can be easily enabled using the $HISTORY
configuration variable.
When enabled, the script records the statistics of the current execution.
Global statistics are stored in $HISTORY_DIR/statistics.dat, while
tag-specific statistics are stored in $HISTORY_DIR/tags/$tag.dat.
These files follow the standard Gnuplot format with columns separated by white
spaces, and each row represents a new execution or record. The global
statistics history file includes the following columns: timestamp (1), lintian
version (2), number of maintainers (3), number of source (4), binary (5) and
udeb packages (6), and number of error (7), warning (8), info (9),
experimental (10) and overriden tags (11). The tag-specific history files
include the following columns: timestamp (1), total number of appearances (2),
overrides (3) and number of different packages in which it is present (4).
* reporting/{config,html_reports}:
+ [JP] Make it possible to keep a record of statistics of previous
archive-wide Lintian executions.
---
debian/changelog | 3 +++
reporting/config | 3 +++
reporting/html_reports | 34 +++++++++++++++++++++++++++++++++-
3 files changed, 39 insertions(+), 1 deletions(-)
diff --git a/debian/changelog b/debian/changelog
index 8a08f40..ad293f5 100755
--- a/debian/changelog
+++ b/debian/changelog
@@ -29,6 +29,9 @@ lintian (2.3.2) UNRELEASED; urgency=low
+ [ADB] Correct a reference in the documentation indicating that the
module collects data for source packages.
+ * reporting/{config,html_reports}:
+ + [JP] Make it possible to keep a record of statistics of previous
+ archive-wide Lintian executions.
* reporting/templates/maintainer.tmpl:
+ [RA] Really fix the logic to optionally display the archive area of
additional package entries under the same source package.
diff --git a/reporting/config b/reporting/config
index adf652f..27dfed6 100644
--- a/reporting/config
+++ b/reporting/config
@@ -17,6 +17,9 @@ $LINTIAN_UNPACK_LEVEL = "";
$LINTIAN_BIN_DIR = "$HOME/bin";
$LINTIAN_GPG_CHECK = 0;
+$HISTORY = 0;
+$HISTORY_DIR = "$HOME/history";
+
$log_file = "$LOG_DIR/harness.log";
$changes_file = "$LOG_DIR/setup-lab.log";
$list_file = "$LOG_DIR/changed-packages.list";
diff --git a/reporting/html_reports b/reporting/html_reports
index e99d8b2..5dd3884 100755
--- a/reporting/html_reports
+++ b/reporting/html_reports
@@ -38,7 +38,7 @@ our $MAX_TAGS = 8;
# FIXME: $statistics_file should be in all caps as well.
our ($LINTIAN_ROOT, $LINTIAN_LAB, $LINTIAN_ARCHIVEDIR, $LINTIAN_DIST,
$LINTIAN_SECTION, $LINTIAN_ARCH, $HTML_TMP_DIR, $statistics_file,
- $LINTIAN_AREA);
+ $LINTIAN_AREA, $HISTORY, $HISTORY_DIR);
# Read the configuration.
require './config';
@@ -563,6 +563,38 @@ close STATS or die "cannot write to $statistics_file: $!\n";
area => join(', ', split(/\s*,\s*/, $LINTIAN_AREA)),
);
output_template ('index.html', $templates{index}, \%data);
+
+exit 0 if (not $HISTORY);
+
+# Update history.
+my $unix_time = time();
+mkdir("$HISTORY_DIR")
+ or die "cannot create history directory $HISTORY_DIR: $!\n"
+ if (not -d "$HISTORY_DIR");
+mkdir("$HISTORY_DIR/tags")
+ or die "cannot create tag history directory $HISTORY_DIR/tags: $!\n"
+ if (not -d "$HISTORY_DIR/tags");
+
+my $history_file = "$HISTORY_DIR/statistics.dat";
+my $stats = "";
+for my $attr (@attrs) {
+ $stats .= " $statistics{$attr}";
+}
+open(HIST, '>>', $history_file)
+ or die "cannot open history file: $!\n";
+print HIST "$unix_time $LINTIAN_VERSION$stats\n";
+close HIST or die "cannot write to $history_file: $!\n";
+
+for my $tag (sort keys %tag_statistics) {
+ $history_file = "$HISTORY_DIR/tags/$tag.dat";
+ $stats = $tag_statistics{$tag};
+ open(HIST, '>>', $history_file)
+ or die "cannot open tag history file $history_file: $!\n";
+ print HIST "$unix_time $stats->{'count'} $stats->{'overrides'} " .
+ "$stats->{'packages'}\n";
+ close HIST or die "cannot write to $history_file: $!\n";
+}
+
exit 0;
# ------------------------------
--
1.6.5.7
Reply to: