[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#621790: marked as done (qa.debian.org: add pubDate to PTS package news RSS feeds)

Your message dated Mon, 2 May 2011 01:16:06 +0200
with message-id <20110501231606.GA20042@rivendell.home.ouaza.com>
and subject line Re: Bug#621790: qa.debian.org: add pubDate to PTS package news RSS feeds
has caused the Debian Bug report #621790,
regarding qa.debian.org: add pubDate to PTS package news RSS feeds
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
621790: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=621790
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: qa.debian.org
Severity: wishlist
Tags: patch
Usertags: pts

It would be nice to have pubDate elements in the news.rss20.xml of PTS to allow
better sorting of items aggregated from multiple packages' feeds. The attached
patch adds more detailed date information to items in news.xml and transforms
them into pubDate elements in the RSS items.


Regards
Jan

-- System Information:
Debian Release: wheezy/sid
  APT prefers stable
  APT policy: (990, 'stable'), (500, 'unstable'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/8 CPU cores)
Locale: LANG=de_DE.utf8, LC_CTYPE=de_DE.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

-- 
Jan Dittberner - Debian Developer
GPG-key: 4096R/558FB8DD 2009-05-10
         B2FF 1D95 CE8F 7A22 DF4C  F09B A73E 0055 558F B8DD
http://ddportfolio.debian.net/ - http://people.debian.org/~jandd/

Subject: add pubDate to package RSS feeds
Author: Jan Dittberner <jandd@debian.org>
Description: This patch adds a pubDate element to item elements in the RSS
 feeds for package's news and fixes deprecation warnings.
 
 * common.py: use hashlib.md5 instead of the md5 module

 * update_news.py:
   - use subprocess.Popen instead of deprecated os.popen3 for mhonarc
     invocation
   - use the xml.dom.minidom implementation included in Python since 2.0
   - add timestamp_to_rfc822date(timestamp) to convert the timestamp format of
     news filenames to RFC 822 timestamps
   - generate a new attribute rfc822date in news.xml item elements

 * news2rss.xsl: add transformation from rfc822date attribute in news.xml to
   pubDate element in RSS


Index: www/xsl/news2rss.xsl
===================================================================
--- www/xsl/news2rss.xsl	(Revision 2506)
+++ www/xsl/news2rss.xsl	(Arbeitskopie)
@@ -69,6 +69,7 @@
 	<xsl:text>/</xsl:text> <xsl:value-of select="$hash" />
 	<xsl:text>/</xsl:text> <xsl:value-of select="@url" />
       </xsl:variable>
+      <pubDate> <xsl:value-of select="@rfc822date" /> </pubDate>
       <guid> <xsl:value-of select="$id" /> </guid>
       <link> <xsl:value-of select="$id" /> </link>
       <description>
Index: www/bin/common.py
===================================================================
--- www/bin/common.py	(Revision 2506)
+++ www/bin/common.py	(Arbeitskopie)
@@ -7,7 +7,7 @@
 # This file is distributed under the terms of the General Public License
 # version 2 or (at your option) any later version.
 
-import md5, os, os.path, re, rfc822, time, email
+import hashlib, os, os.path, re, rfc822, time, email
 from email import Utils, Header
 
 from config import root
@@ -135,7 +135,7 @@
     f = open(fname,"r")
     if not f: return 0
 
-    hash = md5.new()
+    hash = hashlib.md5()
     for line in f.readlines():
         hash.update(line)
     f.close()
Index: www/bin/update_news.py
===================================================================
--- www/bin/update_news.py	(Revision 2506)
+++ www/bin/update_news.py	(Arbeitskopie)
@@ -6,11 +6,12 @@
 
 # Copyright 2002 Raphaël Hertzog
 # Copyright 2006 Jeroen van Wolffelaar
+# Copyright 2011 Jan Dittberner
 # This file is distributed under the terms of the General Public License
 # version 2 or (at your option) any later version.
 
-import os, rfc822, sys, string, re, email, common
-from xml.dom import implementation, ext
+import os, rfc822, sys, string, re, email, time, common, subprocess
+from xml.dom.minidom import getDOMImplementation
 
 from config import dir, odir, root
 from common import hash_name
@@ -29,10 +30,13 @@
     dir = os.path.dirname(htmlfile)
     if not os.path.exists(dir): os.makedirs(dir)
 
-    (stdin, stdout, stderr) = os.popen3("""cd %s; \
+    mhproc = subprocess.Popen("""cd %s; \
         mhonarc -rcfile /dev/stdin \
         -rcfile %s/etc/mhonarc.rc \
-        -single %s 2> /dev/null""" % (dir, root, source))
+        -single %s 2> /dev/null""" % (dir, root, source), shell=True,
+        stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+        close_fds=True)
+    (stdin, stdout, stderr) = (mhproc.stdin, mhproc.stdout, mhproc.stderr)
     stdin.write("""
      <MSGHEAD>
      <h3><a href="../../%s.html">Back to %s PTS page</a></h3>
@@ -45,6 +49,17 @@
     os.chmod(htmlfile+".new", 0664)
     os.rename(htmlfile+".new", htmlfile)
 
+
+def timestamp_to_rfc822date(timestamp):
+    """Format a timestamp in format YYYYMMDDThhmmssZ to a RFC 822 String.
+    
+    >>> timestamp_to_rfc822date('20090718T163915Z')
+    'Sat, 18 Jul 2009 16:39:15 GMT'
+    """
+    return time.strftime("%a, %d %b %Y %H:%M:%S GMT",
+                         time.strptime(timestamp, '%Y%m%dT%H%M%SZ'))
+
+
 def add_resume_for_msg(pkg, file, elt, doc, kind):
     if not os.path.isfile(file):
         pass
@@ -64,34 +79,37 @@
         sub_elt.setAttribute("url", htmlfile)
     if info.has_key("date"):
         sub_elt.setAttribute("date", info["date"])
+    sub_elt.setAttribute("rfc822date",
+                         timestamp_to_rfc822date(info["timestamp"]))
     if info.has_key("from_name"):
         sub_elt.setAttribute("from", info["from_name"])
     elt.appendChild(sub_elt)
 
-# Create the XML documents
-while 1:
-    line = sys.stdin.readline()
-    if not line: break #eof
-    pkg = line.strip()
 
-    doc = implementation.createDocument(None, "news", None)
-    root_elt = doc.documentElement
-    hash = hash_name(pkg)
+if __name__ == '__main__':
+    # Create the XML documents
+    while 1:
+        line = sys.stdin.readline()
+        if not line: break #eof
+        pkg = line.strip()
 
-    # Get news information : static/news/auto
-    for type, number in [("static", 5), ("news", 30)]:
-        elt = doc.createElement(type)
-        dir = "%s/base/%s/%s/%s" % (root, hash, pkg, type)
-        if not os.path.exists(dir): os.makedirs(dir)
-        entries = os.listdir(dir)
-        entries.sort()
-        entries.reverse()
-        for entry in entries[:number]:
-            add_resume_for_msg(pkg, dir+"/"+entry, elt, doc, type)
-        root_elt.appendChild(elt)
+        doc = getDOMImplementation().createDocument(None, "news", None)
+        root_elt = doc.documentElement
+        hash = hash_name(pkg)
 
-    # Output the data to the XML file
-    f = open("%s/%s/%s/news.xml" % (odir, hash, pkg), "w")
-    ext.PrettyPrint(doc, f, 'utf8')
-    f.close()
+        # Get news information : static/news/auto
+        for type, number in [("static", 5), ("news", 30)]:
+            elt = doc.createElement(type)
+            dir = "%s/base/%s/%s/%s" % (root, hash, pkg, type)
+            if not os.path.exists(dir): os.makedirs(dir)
+            entries = os.listdir(dir)
+            entries.sort()
+            entries.reverse()
+            for entry in entries[:number]:
+                add_resume_for_msg(pkg, dir+"/"+entry, elt, doc, type)
+            root_elt.appendChild(elt)
 
+        # Output the data to the XML file
+        f = open("%s/%s/%s/news.xml" % (odir, hash, pkg), "w")
+        f.write(doc.toprettyxml(encoding='utf8'))
+        f.close()

Attachment: signature.asc
Description: Digital signature


--- End Message ---
--- Begin Message --- 
On Fri, 08 Apr 2011, Jan Dittberner wrote:
> It would be nice to have pubDate elements in the news.rss20.xml of PTS
> to allow better sorting of items aggregated from multiple packages'
> feeds. The attached patch adds more detailed date information to items
> in news.xml and transforms them into pubDate elements in the RSS items.

Thanks for the patch, I applied it.

The feeds will soon be updated, I tested it with
http://packages.qa.debian.org/d/dpkg/news.rss20.xml

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
                      ▶ http://RaphaelHertzog.fr (Français)

--- End Message ---
Reply to: