Re: RFC: Creation of a PostgreSQL database schema for wanna-build data
>> I am not a dak hacker, nor an ftpteam or wanna-build guy. That being
>> said, it looks like a lot of the information you store and the functions
>> for comparison look an awful lot like the stuff dak currently has. Does
>> it make sense to consolidate, and expand dak to handle the extra things
>> you need it to do?
> I think that's a definite possibility to consider. dak stores (almost?)
> all the package information we are going to import anyway, so the
> additional tables we use could just reference those instead. Looking
> in more detail at the dak schema, a lot of the tables are almost
> identical, just slightly different implementation details and names.
And its easy to store more, if it helps. We do have to look at the
packages anyway, so getting the data into the db is easy.
> One thing we're lacking there is a debian version type which supports
> the same comparisons as dpkg --compare-versions. In our schema we
> are using such a type for indexing, sorting and joining. It wouldn't
> be hard to update projectb to add such facilities, though.
We do have a versioncmp function, cpp, taken from apt source.
Unifying the various copies of that into one, and modifying dak to use
that instead of what we have would probably be good.
> Fundamentally, what we store in the sources and binaries tables is
> the entire contents of the Sources and Packages files for each
> suite/component (distribution and section in our schema) and
> architecture.
We do have work in progress to enhance projectb to have every data that
is needed to write out contents files. When thats done the next step
will be to also write the packages/sources files using projectb as
input, ie. storing all the data thats neccessary in projectb directly.
(Our benefit is getting rid of long apt-ftparchive runs during dinstall,
by doing the read of data during every package processing, then just
writing data out. Other people might use this data for other purposes :) ).
> - information in Packages/Sources not in dak such as Build-Deps
> (we could use these directly in the DB to determine when to
> reschedule a build). Is there anything from Packages/Sources
> deliberately not stored in dak?
Yes, everything thats written by apt-ftparchive currently. As written
above, we plan to retire that tool (for our use)
> - we keep a history of old packages and sources; does dak keep these
> around once they are no longer referenced by (bin|src)_associations?
> The package state history references these, and would need periodic
> pruning. - builders/build_jobs/logs are wanna-build-specific.
dak stops knowing about packages when it no longer needs the knowledge,
ie. when they are no longer relevant for the archive.
>> It could presumably go in a seperate schema with
>> grants and so on so that permissions can be kept sensible - all of this
>> would need to be worked out with the ftpteam people. It's just that I
>> get sad every time I see needless duplication of information in
>> databases :)
> I haven't personally used schemas and privileges to any great extent.
> But if it's technically feasible, and wouldn't cause too much trouble
> to the ftpteam to have us using the same database, that would be great.
Im not sure how exactly this might work out. I have nothing against
storing more data in projectb, if it makes life easier for other people,
especially with data I have to read at some point anyway.
Yet, we are on different hosts (and thats good). So having you writing
into projectb, no matter if in a schema or wherever, doesn't seem to be
a good idea. Unless we would want to rely on having a working connection
from $wb-host to $ftpmaster-host. And that doesn't sound like a pretty
idea.
>From that on I think projectb should be readonly for you, but be a major
source of data.
--
bye, Joerg
[GFDL]
Well, Debian is not for newbies so the documentation problem is not
really huge
Reply to: