Bug#753076: UDD/ftpnew: Various packages missing

[Andreas Tille <tille@debian.org>]

Hi James,

thanks for your bug report.

On Sat, Jun 28, 2014 at 10:33:48PM -0400, James McCoy wrote:
> Package: qa.debian.org
> Severity: normal
> User: qa.debian.org@packages.debian.org
> Usertags: udd
> 
> Not all of the packages from the NEW queue are showing up in UDD's
> tables.  This affects rmadison/madison.cgi's information with respect to
> the affected packages.
> 
> As a simple example, pyjamas has been in NEW for a few months:
> 
> $ curl https://ftp-master.debian.org/new.822 2>/dev/null | grep-dctrl -FSource pyjamas
> Source: pyjamas
> Binary: pyjamas-pyjs, pyjamas-ui, pyjamas-gmap, pyjamas-canvas, pyjamas-doc, pyjamas, pyjamas-gchart
> Version: 0.8.1
> Architectures: source, all
> Age: 3 months
> Last-Modified: 1395106402
> Queue: new
> Maintainer: Luke Kenneth Casson Leighton <lkcl@lkcl.net>
> Changed-By: lkcl <lkcl@lkcl.net>
> Sponsored-By: philh@debian.org
> Distribution: unstable
> Fingerprint: DFF1415ACE3227FCF20707D6D04BA3A00125D5C0
> Closes: #710033
> Changes-File: pyjamas_0.8.1_amd64.changes
> 
> 
> However it doesn't show up in UDD:
> 
> $ rmadison -u new pyjamas

That's true.  The ftpnew gatherer was written in the idea that only
"really" new packages are injected (not those with changed binaries for
instance).  Now we have a case were the source package name string
'pyjamas' is in UDD but the release of this entry is oldstable.  With
the current philosophy that's not really "new" since the package is
known in oldstable.

So the question is rather to define first, what we want to see in the
new_packages table and than adapt the importer code.  Until today I was
not aware that madison is using new_packages in UDD.  While I think it
is cool that it has gained some wider use we should obviously specify
what should be in the table to make it fit for this purpose since as I
tried to explain above some packages are left out *intentionally* for
the purpose the table was created for (= usage in Blends tasks pages
where this information would only add noise).

I think to make the importer flexible enough it would at least need
another column featuring a flag about the status of the package in new.

Feel free to propose patches to

   ssh://alioth.debian.org/git/collab-qa/udd.git
     udd/ftpnew_gatherer.py

Kind regards and thanks for you bug report

     Andreas.

-- 
http://fam-tille.de