I'm ccing debian-boot in my reply since I see no reason to keep this in
private mail.
Vincent McIntyre wrote:
> After reading the log of last night's IRC, I thought that maybe
> some of the enthusiasm for debian out there could be harnessed to
> help with triaging of bug reports. I have an idea that may help,
> but I want to run it past you first to see if there is some other
> method of doing the same thing.
>
> My idea is to define a role (a bit like that of the kernel janitors)
> where people who want to help can check for unactioned d-i bug reports
> and "tag" them with information that make it easier for d-i developers
> to find bugs for the arches they care about, and to get an idea of the
> overall status of each arch.
I'm unclear whether you're talking about dealing with all d-i bugs (most
of them are tagged d-i), or with installation reports too. Having some
people work on managing our bugs would indeed be quite useful. Some
examples:
- bugs against individual d-i components are not always tagged d-i,
and so they don't show up in queries like "bts show tag:d-i" and are
easy to miss
- bug priorities are often wrong
- bugs are reported against pseudo-packages "install" and
"installation", or even "boot-floppies" and have to be reassigned to
debian-installer where appropriate to be noticed
Plus we have 500 or so uncategorised installation reports (on the
installation-reports pseudo-package). There's a document[1] that
explains how to categorise them, but it takes a lot of effort to learn
enough about d-i to become good at this, and then the task soon becomes
tedious, and few people have managed to do more than a hundred or so
total. So I think a more accessible way to help with a lower barrier to
entry is a find idea.
> The approach is to grab a list of bugs that need some attention and
> send email to the bug with my analysis tags. The taglines I am thinking
> of sending in are along these lines -
>
> triage-arch: architecture, using the standard abbreviations
> triage-inst-medium: cd/floppy/network/serial
> triage-inst-medium-uri: uri of install medium, if supplied
> triage-related-bug: _possible_ related bugs, candidate for merging
> but needs investigation
> triage-not-related-bug: nope, bug in -related tag wasn't actually
> related.
>
> With tags like these in the bugs, people with access to the mbox folders
> of the Bug Tracking System, could easily accumulate stats with perl or
> whatever. It seems to me this is hard to do at the moment because the bug
> reports come in a fairly free format. They need a human to put a bit of
> structure in them.
I feel that some of this is more suited to subject lines. Architecture
and installation media in particular, and I think those two are the most
important. It would be great to be able to pull up all the d-i tagged
bugs, search for an architecture and know it has found all known
problems with that architecture.
The related bug stuff may not be very useful; it's often not helpful to
speculate on the cause of a bug or whether it's the same as some other
bug, this can sometimes make the actual reproduction and analysis of a
bug harder rather than easier. Both the url to the install media, as
well as the date for daily built images are important, and if they're
not in the bug, we generally have to spend time asking back for the
information.
> It's pretty humble work, but something beginners like me could do.
> A bit of programming would speed it up - eg something to create the
> list of bugs-needing-triage. And with a few hundred manual
> classifications in hand it might be possible to write a tool that
> suggests triage- tags, from a textual analysis (a la spambayes,
> but harder because there are many more bins to choose from).
>
>
> What do you think of this? Would it help at all? The point is to
> help developers understand how d-i is going, this would not necessarily
> be useful elsewhere in debian.
>
> With a feedback mechanism (eg webpage with the accumulated stats)
> it might take off. For the triagers, the main thing would be that
> the list of tags is short & simple so that it is fast to do per bug
> and quick to learn. The Bugs/Developer page is thorough, but quite
> daunting.
Let me diverge a bit and go back to the installation reports, which I
have been worrying about again lately. As I said, we have 500, most of
them have at best been skimmed over a few times. Fully processing them
can be difficult, but there are some very simple things that can be of
great help.
One common thing in an installation report is a complaint that the
installer failed to automatically detect some hardware. If the hardware
is PCI and the installation report includes lspci and lspci -n
information (the template asks for it), then the discover1 developers
need to be made aware of it, so they can update their database. If the
report is missing lspci info, the bug submitter needs to be asked to
provide that information, and the bug marked moreinfo.
I suspect that out of our 500 unprocessed installation reports, many
have such information and could be acted on quickly to really make the
installer better, even if we lack the manpower to fully process other
issues in the report. To quote from another private email I received
just today:
| just a quick e-mail to point out some still very much unresolved issues
| AFAIAC: non-detection by D-I of some kernel-supported hardware.
|
| If you look at the following bug reports (submitted with my home
| address, "bruno@pubnix.qc.ca"):
| #243233: installation report (d-i 20040408)
| #245235: installation report (d-i 20040421)
| #249545: installation report (d-i 20040506)
| #250763: installation report (d-i 20040522)
| #254280: installation report (d-i TC-1)
| you will notice that the only bugs that are common to all of them (I
| have mostly concentrated on the h/w auto-configuration) is the
| non-detection by D-I of:
| (1) my very well known ATI TV Wonder VE tv card (pci)
| (2) the built-in SATA controller used on my mobo (Gigabyte
| GA-7VT600-P-L).
| This controller is integrated in the VT8237 southbridge and was
| always left on for testing.
|
| This, er, "unseen" hardware can be made to work by loading and
| configuring all the relevant kernel drivers "by hand". Problem is, D-I
| was supposed to take care of this for me. Well, that's what I thought.
|
| Now, there are other issues too with D-I, but these can be dealt (sp?)
| with in time, and I can fix them myself in the mean-time. On the other
| hand, the hardware-detection routines would have a much greater
| (negative) impact in the short-term, and I don't think that Debian could
| live with that.
(I've taken care of categorising the bugs he refers to.)
So I was thinking that it would be great if in the near future we could
make a pass over _all_ the installation reports, and scan for this kind
of problem, and collect a list of them for the discover maintainers. And
if we're doing that, we could realively easily also retitle each bug
report as we processed it, to make the subjects more useful (and
incidentially, keep track of which we've processed). I'm thinking of
a standard subject format like this:
[arch] [media] [release-or-date] old-subject-or-better-summary
To compare old and new:
#227917: Boot fails on Alpha DS 10 with illegal instruction
#239501: PXE netboot install i386
#258035: Installation Report
#258117: install-report
#258119: Debian Installer Reports
#227917: [alpha] [cdrom] [20040115] Boot fails on Alpha DS 10 with illegal instruction
#239501: [i386] [netboot] [20040321] success
#258035: [i386] [dvd] [unknown] success
#258117: [i386] [cdrom] [beta4] problem at reboot
#258119: [i386] [cdrom] [beta4] success
Vincent, maybe dealing with the installation reports this way would be a
good first task for a d-i bug janators team, and a way to pull such a
team together. There's a BSP scheduled for this weekend, and perhaps
while the BSP participants are working on squashing RC bugs, others who
arn't able to do that and still want to help make the release better in
a very real way could try to work their way through all 500 installation
reports. If non Debian developers are going to do this, it would be best
to have some very clear and simple to follow instructions, and some
developers around to handle edge cases. Would you like to draw up some
plans for this?
--
see shy jo
[1] http://d-i.alioth.debian.org/svn/debian-installer/installer/doc/installation-reports.txt
Attachment:
signature.asc
Description: Digital signature