[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: RFC: Creation of a PostgreSQL database schema for wanna-build data

On Sat, Jan 17, 2009 at 06:04:13PM +0000, Stephen Gran wrote:
> This one time, at band camp, Roger Leigh said:
> > Dear all,
> <snip good news>
> > We would be grateful for any comments you might have, particularly
> > if you have used the existing wanna-build and can see any defects,
> > omissions, or any suggestions for anything we missed or could
> > implement in a more optimal fashion.  If you are a PostgreSQL guru,
> > a review of the PL/pgSQL and PL/Perl functions and triggers used
> > for custom constraint checking would be much appreciated.
> I am not a dak hacker, nor an ftpteam or wanna-build guy.  That being
> said, it looks like a lot of the information you store and the functions
> for comparison look an awful lot like the stuff dak currently has.  Does
> it make sense to consolidate, and expand dak to handle the extra things
> you need it to do?

I think that's a definite possibility to consider.  dak stores (almost?)
all the package information we are going to import anyway, so the
additional tables we use could just reference those instead.  Looking
in more detail at the dak schema, a lot of the tables are almost
identical, just slightly different implementation details and names.

One thing we're lacking there is a debian version type which supports
the same comparisons as dpkg --compare-versions.  In our schema we
are using such a type for indexing, sorting and joining.  It wouldn't
be hard to update projectb to add such facilities, though.

Fundamentally, what we store in the sources and binaries tables is
the entire contents of the Sources and Packages files for each
suite/component (distribution and section in our schema) and
architecture.  dist_(binaries|sources) then tells us which versions
of which package are available in each suite/component.  If this is
what dak stores, it's already doing a large part of what we want.

Possible differences in what dak stores here are:

- information in Packages/Sources not in dak such as Build-Deps
  (we could use these directly in the DB to determine when to
  reschedule a build).  Is there anything from Packages/Sources
  deliberately not stored in dak?
- we keep a history of old packages and sources; does dak keep
  these around once they are no longer referenced by
  (bin|src)_associations?  The package state history references
  these, and would need periodic pruning.
- builders/build_jobs/logs are wanna-build-specific.

> It could presumably go in a seperate schema with
> grants and so on so that permissions can be kept sensible - all of this
> would need to be worked out with the ftpteam people.  It's just that I
> get sad every time I see needless duplication of information in
> databases :)

I haven't personally used schemas and privileges to any great extent.
But if it's technically feasible, and wouldn't cause too much trouble
to the ftpteam to have us using the same database, that would be great.

One consideration I do have is the ease of setting up a working
wanna-build in the absence of a full dak setup.  This is very
useful if you want to do test builds of the whole archive for
example.  Currently it's very easy to set up independent setups for
testing and development work, and if there's continuing demand for
that, I wouldn't want to lose it.

However, I think a good first step would for both dak and wanna-build
to share as much of their schemas as is reasonably possible.  To that
end, I'll update my wanna-build schema to use the same naming as is
used in projectb as far as is practicable, and then we'll see where
we can go from there.  It will at the very least make merging the
databases possible in the future.


  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.

Attachment: signature.asc
Description: Digital signature

Reply to: