Re: automatic bug filing by test robot
On 28/08/08 at 09:19 +0200, Raphael Hertzog wrote:
> On Wed, 27 Aug 2008, Ian Jackson wrote:
> > I'm hoping to shortly turn on the automatic bug filing mechanism.
> > I'm writing now to give people a chance to object :-).
> [...]
> > very few other packages. Most autopkgtest reports are FTBFS problems.
>
> I object filing FTBFS automatically. In some cases, the FTBFS is a failure
> from one of the build-dependencies and filing bugs automatically would
> lead to lots of useless/duplicate bugs. Thus a I think that a human filter
> here is most welcome.
>
> Lucas is covering this quite well up to now. If we ever get back to the
> situation where nobody is willing to do that work, then we can reconsider
> this. Also, how would you check that a bug has not already been filed
> manually by another user?
What I do with my archive rebuilds is:
- rebuild everything (it's important to process results only when you
have all the results for all packages, as it makes it easier to find
failures caused by a change in another package)
- have a script extract the failures from all logs, so I can easily get
an overview of all the failures, making it easier to find a regression
in some common tool that caused dozens of new failures.
- for each remaining package, use the BTS SOAP interface the fetch all
the bugs, and filter the bugs using a regexp (basically
/(ftbfs|build)/ ), so I avoid most of the potential duplicates.
- before filing the bug (using a script), list the possibly relevant
bugs from the package.
- usertag the bugs, so I can easily keep track of:
+ new failures, by merging the previous list of failures into the new
one
+ packages that don't fail anymore, whose bug is still open
+ bugs that have been closed, but not fixed
Most of the scripts I use for that are in
http://svn.debian.org/wsvn/collab-qa/collab-qa-tools/?rev=0&sc=0 . It's
hackish, but usable.
--
| Lucas Nussbaum
| lucas@lucas-nussbaum.net http://www.lucas-nussbaum.net/ |
| jabber: lucas@nussbaum.fr GPG: 1024D/023B3F4F |
Reply to: