Hi, On Dienstag, 17. März 2009, Lucas Nussbaum wrote: > I am really not sure that putting piuparts results on the PTS is the way > to go right now. Right, we are certainly not there yet, but IMO we are getting close. > First, real piuparts failures are usually real RC bugs. If they are only > reported to the PTS, they won't be properly tracked by the release team, > for example. Are you planning to actively report bugs on piuparts > failures in addition to reporting them to the PTS? Yes. I also plan to use tools for that, either based on your code, of if I fail to understand it, write my own. I also plan to link to those bugs in the piuparts reports on http://piuparts.debian.org :-) > Have you reported > bugs on the piuparts failures you found so far? Not yet, been busy improving piuparts :-) (Which sometimes involved a restart of the run, which then usually also sorted out false-positives so I knew my time was better spend not reading too many logs.. :) > Second, since piuparts failures are real bugs, they should be "pushed" > to the maintainer. I don't think that adding them to the PTS web page is > a "pushy-enough" way to provide that information to maintainers. Well, thats true, but it's also and still a bit pushing, even if not enough :-) The problem is, a piuparts run on all packages in a distro now takes 4-6 weeks, and Debian changes a lot, so there are bugs to be filed constantly. And I definitly suspect there will be sometimes weeks without bugs being reported. That's just how life works. I definitly want to form a team to file bugs based on piuparts, but still then, PTS integration will not hurt and will not stop us from filing bugs. But it will help getting them fixed faster. (The setup tests three distros: sid, squeeze and lenny2squeeze. So there are approx 75000 piuparts runs scheduled...) > Instead, the PTS mail interface could be a nice way to inform the > maintainer of failures. I completly agree with what Loïc wrote. > Third, in my experience, piuparts generates a lot of false positives, > that are either: > - minor bugs (like corner-cases related to symlinks) symlinks are not bugs in that setup. > - bugs in other packages (packages failing because one of their > dependencies is failing) http://piuparts.debian.org/sid/state-dependency-failed-testing.html lists those. Packages with no depends other than priority: required are tested first, so this doesnt really happen. 16782 packages in sid are in state "waiting-for-dependency-to-be-tested" currently, being xorg is not installable wit tested depends atm :-) > Before information is provided automatically to maintainers, I would > like to see numbers about percentage of false positives. Is piuparts > really ready for that? Yes, I think its ready for passivly informing the maintainer via the PTS. > Fourth, piuparts logs are often difficult to understand for maintainers > who have never used piuparts. It would be needed to provide information > about how to reproduce failures manually. Is it planned? Yes, though the prominent link on http://piuparts.debian.org pointing to http://wiki.debian.org/piuparts/FAQ points nowhere atm :-) I'll add "better header for piuparts-logs" to my todo now. Done. > All in all, I'm not against the idea of automatic reporting of piuparts > failures, but we should really make sure that it provides (mostly) > useful information. If we fail, maintainers will classify piuparts > reports as noise, and will just ignore them. Yes. regards, Holger
Attachment:
signature.asc
Description: This is a digitally signed message part.