[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#702085: marked as done (UDD: upload_history importer broken since move to new-master)

Your message dated Wed, 18 Mar 2015 16:07:03 +0100
with message-id <20150318150703.GA16565@xanadu.blop.info>
and subject line Re: Bug#702085: Report on what is changed to solve this bug
has caused the Debian Bug report #702085,
regarding UDD: upload_history importer broken since move to new-master
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
702085: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=702085
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: qa.debian.org
Severity: serious
User: qa.debian.org@packages.debian.org
Usertags: udd

Hi,

The upload history data is generated by parsing the debian-devel-changes
mailing list archives.

The way it used to work is:
- a script on master parses the archives mbox and generates a summarized version
- the summarized version is exported in http://master.debian.org/~lucas/ddc-parser/
- on ullmann, a script downloads those files and import them into the DB

This is broken since the move to the new master, since it doesn't have an httpd
anymore.

A solution that would work for me would be to have a regular (r)sync of
debian-devel-changes archives from master to ullmann.debian.org. That would
also allow to drop the dependency on my user account on master.debian.org.

A service affected by that is the cgi used to generate the "new contributors"
section of the weekly news.

Lucas

--- End Message ---
--- Begin Message --- 
On 05/07/13 at 18:30 -0400, Asheesh Laroia wrote:
> Here is what I did.
> 
> == Configured inbound email ==
> 
> I did what was suggested in the earlier comment to configure inbound emails
> to debian-devel-changes to arrive in
> /srv/udd.debian.org/incoming-mail/changes .
> 
> == Copy archives to ullmann ==
> 
> I copied the archives on master to:
> /srv/udd.debian.org/email-archives/debian-devel-changes/
> 
> I also created a symlink in
> /srv/udd.debian.org/email-archives/debian-devel-changes/ for a virtual
> archive mailbox called "current" that points to
> /srv/udd.debian.org/incoming-mail/changes . This way, no further mailing
> list archive syncing is needed.
> 
> I documented this somewhat in /srv/udd.debian.org/email-archives/ and if
> other email archives are needed, they can go in that directory one day.
> 
> == Reconfigure config-ullmann.yaml accordingly ==
> 
> You all saw this patch, probably.
> 
> == Store the upload-history code in /srv/udd.debian.org/upload-history ==
> 
> I created /srv/udd.debian.org/upload-history which is a copy of
> ~lucas/public_html/upload-history from master.debian.org.
> 
> paulproteus@ullmann:/srv/udd.debian.org/upload-history$ svn info | grep Root
> Repository Root: svn+ssh://lucas@svn.debian.org/svn/collab-qa
> 
> This is that same old code, with a trivial change to 'Makefile' to adjust it
> to find the mbox (or mbox.gz) files in
> /srv/udd.debian.org/email-archives/debian-devel-changes/.
> 
> == Adjust cron accordingly ==
> 
> I added a job to update the *.out files that live in
> /srv/udd.debian.org/upload-history:
> 
> udd@ullmann:~$ crontab -l | grep make
> 49 */6 * * * (( cd /srv/udd.debian.org/upload-history ; make ) ;  $UAR upload-history ubuntu-upload-history) | /srv/udd.debian.org/udd/scripts/filter-output.rb
> 
> This is prefixed to the cron job as uses "udd.py" to import upload-history.
> 
> == Thanks ==
> 
> That's all I have for now. It seems stable and sane. Feedback welcome.

1.5 years later, I can confirm that this still works. Thanks Asheesh!

Closing the bug now.

- Lucas

--- End Message ---
Reply to: