Re: piuparts PTS integration
On 16/03/09 at 23:07 +0100, Holger Levsen wrote:
> Hi,
>
> http://piuparts.debian.org is more or less back, (a bit polishing is still on
> my todo before I'm comfortable announcing it on d-d-a) and I'm wondering how
> to do the PTS integration.
>
> http://wiki.debian.org/piuparts#PiupartsandPTSintegrationsuggestion has a
> quite specific and easy to follow proposal - but misses on fact: piuparts is
> centered around binary packages, while the PTS references source packages.
>
> One file with results for all binary packages would be easy to implement:
>
> /results-per-binary.txt
> %s pass $URL2MATCHINGLOG
> %s fail $URL2MATCHINGLOG
>
> Could the PTS cope with that in a sensible way?
>
> Other ideas?
I am really not sure that putting piuparts results on the PTS is the way
to go right now.
First, real piuparts failures are usually real RC bugs. If they are only
reported to the PTS, they won't be properly tracked by the release team,
for example. Are you planning to actively report bugs on piuparts
failures in addition to reporting them to the PTS? Have you reported
bugs on the piuparts failures you found so far?
Second, since piuparts failures are real bugs, they should be "pushed"
to the maintainer. I don't think that adding them to the PTS web page is
a "pushy-enough" way to provide that information to maintainers.
Instead, the PTS mail interface could be a nice way to inform the
maintainer of failures.
Third, in my experience, piuparts generates a lot of false positives,
that are either:
- minor bugs (like corner-cases related to symlinks)
- bugs in other packages (packages failing because one of their
dependencies is failing)
Before information is provided automatically to maintainers, I would
like to see numbers about percentage of false positives. Is piuparts
really ready for that?
Fourth, piuparts logs are often difficult to understand for maintainers
who have never used piuparts. It would be needed to provide information
about how to reproduce failures manually. Is it planned?
All in all, I'm not against the idea of automatic reporting of piuparts
failures, but we should really make sure that it provides (mostly)
useful information. If we fail, maintainers will classify piuparts
reports as noise, and will just ignore them.
--
| Lucas Nussbaum
| lucas@lucas-nussbaum.net http://www.lucas-nussbaum.net/ |
| jabber: lucas@nussbaum.fr GPG: 1024D/023B3F4F |
Reply to: