Re: [GSoC] Adding information to UDD and inject the rendering to tasks,py
Hi Akshita,
On Mon, Mar 30, 2015 at 12:07:40PM +0530, Akshita Jha wrote:
> As you suggested, I would like to work on the warming up task of importing
> the Registration and Donation fields from debian/upstream/metadata into UDD
> and injecting the rendering into the current tasks.py. It will also help me
> in my actual task.
>
> It would be great if you could help me get started on this.
I'd like to try some hints which I hope to be helpful. I just "hope"
since for somebody who is more familiar with UDD I migth assume things
to be clear which are actually not. So feel free to bother me with
more detailed questions.
If you checkout UDD code from Git[1] you could do
./update-and-run.sh blends-prospective
which updates machine metadata from several Blends teams. (BTW, the
data are gathered by a script inside the website.git of Blends[2] but
currently I see no reason to touch this - just for leting you understand
the connection.) In this process the set of metadata is unpacked on
your machine and you can see by the following command:
grep -e Registration -e Donation /srv/udd.debian.org/mirrors/machine-readable/*/*.upstream
several upstream metadata files do just contain the Registration and
Donation information. The update-and-run.sh script is refering to
the config file config-ullmann.yaml which contains the following
section for blends-prospective:
blends-prospective:
type: blends-prospective
update-command: /srv/udd.debian.org/udd/scripts/fetch_machine-readable.sh
path: /srv/udd.debian.org/mirrors/machine-readable
table: blends_prospectivepackages
While the update-command simply wgets and unpacks the data the actual
code that feeds the database is
udd/blends_metadata_gatherer.py
Similarly you should have a close look at the bibref importer. Hmmm,
now when I'm writing this I remember there is an actual flar in the
bibref importer. Originally the references were imported from the data
gathered by Umegaya[3] but besides the fact that it is currently broken
this only gathers data of *existing* Debian packages. However, the
Blends web sentinel also wants data from packages which are not yet on
the Debian mirror but only in VCS. So the blends_prospective_gatherer
*also* injects references in the bibref table. Currently it does this
only if an entry for the package in question is lacking and does not
override the bibref gatherer. However, since Charles Plessy
confirmed[4] that there is a problem we probably should force the
reference from machine-readable data in any case. Hmmm, I hope, this
was not too confusing. (Otherwise, I will hopefully find some time to
fix this quickly.)
How to proceed with the upstream metadata in your case: Probably you
need to create a new table for the metadata and probably it makes sense
to accept all keys declared at the Wiki page that defines this file[5].
I'm a bit uncertain whether it is a good idea to keep the references in
a separate table. While I tend to say that we leave it as is and keep
only the other fields in a separate table it somehow conflicts with the
logic to have one store for one type of data. On the other hand we
have some applications that rely on the separate bibref table and this
comes quite handy. What do you think about this?
Was the explanation (with detours into potential pitfalls and problems
helpful)?
Kind regards
Andreas.
[1] ssh://git.debian.org/git/collab-qa/udd.git
^^^ this is non-annonymous since I guess you might like to
commit later. I'm not sure whether you have permission
yet. Try s/ssh:/git:/ otherwise.
[2] https://anonscm.debian.org/cgit/blends/website.git/tree/misc/machine_readable
[3] http://upstream-metadata.debian.net
[4] https://lists.debian.org/debian-qa/2015/02/msg00038.html
[5] https://wiki.debian.org/UpstreamMetadata
--
http://fam-tille.de
Reply to: