Bug#805238: debian-timeline: Add script for automatic update of the bug events database
Package: debian-timeline
Version: 37
Severity: wishlist
Tags: patch
You will find attached to this bug report, as a patch for Git, a script
called update-bug-list.py. Launching it from the top level directory
like this:
./update-bug-list.py
will automatically update the database in data/events/bugs, by
querying the bugs.d.o database through the SOAP interface.
You might be interested in integrating this script into the
debian-timeline package.
Best regards,
Rafael Laboissière
-- System Information:
Debian Release: stretch/sid
APT prefers testing
APT policy: (650, 'testing'), (600, 'unstable'), (550, 'stable'), (500, 'experimental')
Architecture: i386 (i686)
Kernel: Linux 3.18.0-trunk-686-pae (SMP w/2 CPU cores)
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages debian-timeline depends on:
ii libjs-simile-timeline 2.3.0+dfsg1-2
debian-timeline recommends no packages.
debian-timeline suggests no packages.
-- no debconf information
>From 20a84014590733ea1faab44899e87522327b0ac4 Mon Sep 17 00:00:00 2001
From: Rafael Laboissiere <rafael@laboissiere.net>
Date: Sun, 15 Nov 2015 22:54:35 +0100
Subject: [PATCH] update-bug-list.py: Add script for automatic update of bug
events list
---
update-bug-list.py | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
create mode 100755 update-bug-list.py
diff --git a/update-bug-list.py b/update-bug-list.py
new file mode 100755
index 0000000..aaa039f
--- /dev/null
+++ b/update-bug-list.py
@@ -0,0 +1,93 @@
+#!/usr/bin/env python
+
+## Copyright (C) 2015 Rafael Laboissiere
+##
+## This program is in the public domain
+
+## Use this script for updating file data/events/bugs with more recent
+## bugs. Launch the script from the top level directory, like this:
+##
+## ./update-bug-list.py
+##
+## The file data/events/bugs will be overwritten.
+
+import re
+import SOAPpy
+from debian import deb822
+from datetime import datetime
+
+### This is the granularity for inclusion of bugs in the events database
+granularity = 10000
+
+### URL of the Debian bugs website
+debbug_url = 'https://bugs.debian.org'
+
+### Initialize the SOAP server for accessing the Debian bugs database
+url = '%s/cgi-bin/soap.cgi' % debbug_url
+namespace = 'Debbugs/SOAP'
+server = SOAPpy.SOAPProxy (url, namespace)
+
+### Get the newest bug number
+newest_bug = server.newest_bugs (1) [0]
+
+### Read the bugs database
+bug_events_file = 'data/events/bugs'
+input = file (bug_events_file).read ().decode ('utf-8').split ('\n')
+current_bugs = []
+bug_list = deb822.Deb822.iter_paragraphs (input, use_apt_pkg = False)
+
+### Initialize the array for storing the information on the bugs
+bug_paras = []
+
+### Regular expression for getting the bug number from the source field
+debbug_url_re = re.compile ('%s/(\d+)' % debbug_url)
+
+### Iterate over the paragraphs, storing the bug numbers and building
+### their string representation
+for para in bug_list:
+ m = debbug_url_re.search (para ['source'])
+ if m:
+ current_bugs.append (int (m.group (1)))
+ bug_paras.append ('''Title: %s
+Date: %s
+Source: %s''' % (para ['title'], para ['date'], para ['source']))
+
+### Get the bug recorded with highest number
+last_recorded_bug = max (current_bugs)
+
+### Regular expression for extracting the full name of the bug origniator
+email_re = re.compile ('([^<]+)')
+
+### Check whether there are new bugs to retrieve
+if newest_bug >= last_recorded_bug + granularity:
+
+ ## Yes : get them
+ for i in range (1, 1 + (newest_bug - last_recorded_bug) / granularity):
+
+ ## Get next bug
+ bug = last_recorded_bug + i * granularity
+ status = server.get_status (bug) [0]
+
+ ## Get the full name of the submitter
+ originator = status.value.originator
+ m = email_re.search (originator)
+ if m:
+ originator = m.group (1).rstrip ()
+
+ ## Get the date of submission
+ date = datetime.fromtimestamp (status.value.date).strftime ("%b %d %Y")
+
+ ## Add paragraph to the database
+ bug_paras.append ('\n'.join (['Title: Debian Bug #%d reported by %s' % (bug, originator),
+ 'Date: %s' % date,
+ 'Source: %s/%d' % (debbug_url, bug)]))
+ print 'New bug found: #%d' % bug
+
+ ## Write the new database
+ output = open (bug_events_file, 'w')
+ output.write ('\n\n'.join (bug_paras).encode ('utf-8') + '\n')
+ output.close ()
+
+else:
+ ## No: nothing to do
+ print "No new bugs."
--
2.6.2
Reply to: