Re: [GSoC] Adding information to UDD and inject the rendering to tasks,py

Hi,

On Fri, Apr 17, 2015 at 6:21 PM, Andreas Tille <andreas@an3as.eu> wrote:

Hi Akshita,

The thing is that the resulting debian.bib and debian.tex should include
only those citations that belong to packages which are just *inside*
Debian.

Does this mean that debian.bib and debian.tex should include citations for packages that are only in the Debian package list ('source' column of 'sources' table). 'bibref' table may have 'source' that is not in 'source' of 'sources' table, since bibref_gatherer.py gets its data from VCS. We do not want the citations of these 'source's to be incuded in debian.bib and debian.tex files. Am I correct ?

That's the reason why it is done inside the bibref_gatherer.py.

I do not clearly understand this statement. Does this mean that all the source and their citations are included from VCS into bibref table ?

So our later corrections will not come into effect. Blends-prospective
will be called later as you can see in scripts/cron_ftpnew_blends.sh
which has the only reason to define a sequence.

I do not understand this very clearly either. Why will update not work?

ufile = upath+'/'+source+'.upstream'

if exists(ufile):

cur.execute("EXECUTE check_reference (%s)", (source,))

if cur.fetchone()[0] == 0:

upstream = upstream_reader(ufile, source, self.log)

if upstream.references:

if not prospective:

# Valid references found in this upstream file and it is valid YAML even if it was not fetched by bibref gatherer

self.log.warning("%s of %s has upstream file but no references in UDD" % (source, sprosp['blend']))

upstream.parse()

for ref in upstream.get_bibrefs():

bibrefs.append(ref)

The code snippet above from blends_prospective_gatherer.py checks if there is a .upstream file for the source and then appends it to bibrefs. The data is from VCS and this is what is inserted in bibref table. So, are you saying that this insert will not matter as bibref_gatherer.py runs before Blends-prospective according to scripts/cron_ftpnew_blends.sh and so it will not be reflected in .tex and .bib files ?

If we mix this with data from prospective packages and create
debian.{bib,tex} afterwards to many references would be included. On
the other hand we could do this if we would add a join to the sources
table and include only those citations that have a matching source
package name.

Does this mean that creating a debian.bib or debian.tex file afterwards, will include references for 'source's that are there in bibref table but not necessarily present in 'source' column of 'sources' table, implying that the package is not *inside* Debian ? This can be prevented if we join on 'source' column of sources and bibref table.

"SELECT distinct s.source from bibref b join sources s on s.source = b.source;"

mentioned in Proof Of Principle (https://wiki.debian.org/UpstreamMetadata).

This would mean:

split the generation of debian.{bib,tex} from bibref_gatherer.py
This could be even a separate script appart from all the udd
configuration stuff. Looks like a good idea anyway perhaps with
an extra parameter if somebody wants to have all the citation.
(=per default use a join to the sources table but enable an option
to provide all)

Does this coarsely mean, moving the class (with changes):

class bibref_gatherer(gatherer):

"""

Bibliographic references from debian/upstream files

"""

from bibref_gatherer.py to a different .py file, and inserting the references based on the 'join' condition mentioned above ? Also, including a parameter, to allow anyone who wants all the citations (irrespective of whether the 'source' is in Debian package list or not) to ignore the 'join' and get all the citations from bibref table for a particular 'source'

Is my description of the problem clear enough?

I think so. I will keep troubling you with more questions though :)