Re: DDE, Debian Data Export
[debian-custom list - actually targetted at the to be created debian-blends list
in CC]
On Wed, 11 Feb 2009, Enrico Zini wrote:
* Maintainer <-> Source package mapping
* Popcon rankings
* What is in the new queue
* Package screenshots
* Localisation information
* uscan status
These items of the (slightly cropped) list are extremely interesting for
what we need in the Blends task pages.
And I would like to add
* DebTags of a package (ping: I was asking for a Python interface ...)
* DDTP (if this is not yet includet in "Localisation information"
* Debian Pure Blend specific information
A nightmare, uh?
Until today? Not really a nightmare - but partly really hard to obtain
and thus not finished at the extend I would like to see.
* The solution
DDE is a way to make it simple to publish and download data. The aim is
to be able to access all sorts of Debian information without worrying
about data formats, protocols and access control, and to make it easy to
discover what data is available.
Sounds great.
DDE exports data as a big virtual tree. You can pick a node in the tree
by its URL and download all the data that it contains, in a format of
your choice: currently it supports JSON/JSONP, YAML, CSV and Python
pickled objects.
I have to admit after a (quick) view about the URLs you gave I did not
really understand how the data are entering the tree and how I can pull
the information (it's a shame that I missed your talk).
DDE is not a competitor to UDD (http://wiki.debian.org/UDD): UDD is
about creating a central location where all the data can be accessed,
while DDE is about giving people a simple way to access data or subsets
of data.
So is DDE actually using UDD as input?
In a way, DDE and UDD complete each other: the more data enters UDD, the
more data is available for DDE. In turn, DDE gives a simple interface
to the most popular and useful UDD queries.
Sound like a 'yes' to my questio above. I clever UDD interface would
really rock!
* The dream
Here are some hints at what can be done with this:
* Autocompletion in HTML fields
* Export data to feed external sites like debtags.debian.net or
screenshots.debian.net
* Have a way for package managers to easily access all sorts of data
* Have a way to implement fancy tools that can query massive data sets
without needing to download them locally
Sounds really good!
* A call for action
You can add data to the DDE tree by just putting a data file in yaml,
json or pickle format under `~/.dde`: I've written a specific guide[1]
to this on the Debian wiki, see: http://wiki.debian.org/DDE/HomeFiles
This is the part I'm curious about. Please explain in more detail.
I've written some code which is harvesting data for Blends - if I can
provide some input I'd be happy to do so.
For more complicated cases (like accessing a remote database), it is
possible to extend DDE via python plugins[1]. You can get in touch with
me if you need to go that way.
TOUCH. ;-)
Many thanks for your work on this
Andreas.
--
http://fam-tille.de
Reply to: