Re: Bootstrapping: list of 81 self-cycles in Debian Sid

[Johannes Schauer <j.schauer@email.de>]

Hi,

Quoting Joerg Jaspert (2013-03-06 22:27:00)
> And it sounds like something that could be done using the archives tools /
> integrated into them. If you are interested to integrate it there properly,
> we are in #debian-ftp on irc.debian.org and also debian-dak@lists.debian.org

So, I now joined #debian-ftp and subscribed to debian-dak@l.d.o. What would the
best way be to integrate the information about self cycles?

> > Yes, the input to the code is just a pair of Packages.bz2 and Sources.bz2
> > files.
> 
> And as they are generated completly out of our archives postgres
> database, that one could be used too, probably not hard to change. I
> wonder if one could "offload" a bit of the work to sql too to help.

The code uses the Cudf representation of binary and source packages in the
Packages.bz2 and Sources.bz2. I dont think a database can lead to any speedup
except if it is a Cudf database which caches installation sets, closures etc...

> 500MB isnt really much space. And as they are mostly for the
> Packages/Sources, its much less for the output you generate...  That is,
> ideally this generates just "index" files, which are then consumed by
> something like the PTS.

"index" files? What are they?

The Ocaml code which does all the dependency resolution currently outputs JSON
files which are then turned into html by a python script.

The quick and dirty implementation can be seen here:

https://gitorious.org/debian-bootstrap/bootstrap/trees/master/webselfcycles

> Also, would "incremental" runs work? Say, the database tells you which
> packages changed recently due to uploads. Only recheck the parts affected by
> it.  Yes, requires state storage.

In theory I guess yes, it would be possible to make it work incrementally. But
in practice implementing that would probably be another GSoC as dose3 cant do
anything incremental by now. So I dont think the amount of required work would
justify the result.

cheers, josch