Re: [GSoC] Adding information to UDD and inject the rendering to tasks,py

[Andreas Tille <andreas@an3as.eu>]

Hi Akshita,

On Fri, Apr 17, 2015 at 03:40:54PM +0530, Akshita Jha wrote:
> On Tue, Apr 7, 2015 at 1:46 PM, Andreas Tille <andreas@an3as.eu> wrote:
> 
> >
> >  ./only-run.sh blends-prospective
> > Traceback (most recent call last):
> >   File "/srv/udd.debian.org/udd//udd.py", line 86, in <module>
> >     exec "gatherer.%s()" % command
> >   File "<string>", line 1, in <module>
> >   File "/srv/udd.debian.org/udd/udd/blends_prospective_gatherer.py", line
> > 422, in run
> >     cur.execute(query)
> > psycopg2.ProgrammingError: FEHLER:  Syntaxfehler bei „REPLACE“
> > LINE 1: ...RE bibref_insert (text, text, text, text, int) AS REPLACE IN...
> >                                                              ^
> >
> 
> The REPLACE function works only in MySQL. In postgresql, there is not
> in-built function to update or insert.

This explains why I never have heard from this. :-)

> I am sending a patch which defines
> modifies 'udd/blends_prospective_gatherer.py'. I have written a function
> "Upsert" which returns a string - "UPDATE ..... SET .... WHERE...." or
> "INSERT INTO .... VALUES ...". This string is then appended to a "PREPARE
> ..... AS ...." query, and the final query is executed.
> 
> I am sending a 'rows' parameter to Upsert() which basically is the count of
> rows in bibref table where the primary keys (source, key, package, rank)
> are checked and LIMIT 1. So,
> if rows == 0:
>     "insert"
> else:
>     "update"
> There are no packages which get updated, because there seem to be no
> records with duplicate primary keys. PFA  the output for the above (insert
> / update) code snippet. Can you please review the patch ?

It looks good so far but when I think about it it seems does not really
solve the problem since my advise was not well thought.  But we are on
our way at least. :-)

The thing is that the resulting debian.bib and debian.tex should include
only those citations that belong to packages which are just *inside*
Debian.  That's the reason why it is done inside the bibref_gatherer.py.
So our later corrections will not come into effect.  Blends-prospective
will be called later as you can see in scripts/cron_ftpnew_blends.sh
which has the only reason to define a sequence.

If we mix this with data from prospective packages and create
debian.{bib,tex} afterwards to many references would be included.  On
the other hand we could do this if we would add a join to the sources
table and include only those citations that have a matching source
package name.

This would mean:

   split the generation of debian.{bib,tex} from bibref_gatherer.py
   This could be even a separate script appart from all the udd
   configuration stuff.  Looks like a good idea anyway perhaps with
   an extra parameter if somebody wants to have all the citation.
   (=per default use a join to the sources table but enable an option
   to provide all)

Is my description of the problem clear enough?

Kind regards

        Andreas.

-- 
http://fam-tille.de