Hi m2-, On Thu, Feb 01, 2001 at 11:50:35PM +0100, Marcelo E. Magallon wrote: > I was just watching a discussion on #debian-devel and the subject of QA > has come up yet again. Seems like that happens from time to time :) > The questions that pop up are basically the same as always, namely, how > can we have some sort of QA procedure that *actually* helps in speeding > up the release process? The answer is at least partially evident: we > need people actually doing QA. It looks like Raphael, whom, AFAIK, was > playing QA coodinator, got caught up in RL, and other people are in a > similar situation. Can we have an informal heads count here? Not a lot of answers as of yet... Of course I don't know if somebody answered in private mail but on the list only aj and shorty have answered. > Once we have the people, the other part of the answer is not that > clear. IMO, a possible solution would be to have a sort of test suite. > Since the size of Debian together with the size of the QA team makes > the very idea of a generalized test suite laughable at the moment. Stop! We don't currently even need a test suite. There is a lot of known problems in the dist. Just have a look at the BTS. The QA team should fight against the bug count. As others will probably confirm I was very committed to QA work back when I joined Debian. It was fun to go and fix some bugs - download a package, look at the BTS entry for the package, fix some bugs, submit the patch to the BTS. Problem is that it is quite inefficient to do that. Often it will happen that the maintainer does not even apply the patch. In case he is active I made the experience that the maintainer preferred to fix the problem himself so that my work went down the drain. The other side was that I made some patches and nothing happened for months even though I submitted the patch to the BTS. And you can't even do an NMU without getting flamed. That's not generally true of course but I got the impression not uploading the fixed package is better for my reputation in Debian circles. > Phil did develop such a beast, but if you look in /usr/lib/debian-test, > unless you have a particular set of packages, you'll note it's empty. > Woody's release is hopefully too close to attempt getting these scripts > written for a significant part of Debian.[1] Writing tests for significant parts of Debian would be a nice thing but I thing we have to concentrate on other things first. > Last year a group of developers demed the idea of using a task manager > as a good one, in order to help introduce some order into the apparent > chaos that QA is right now. One example of such software is > SourceForce (SF is more than a task manager, but the task manager > contained there in is a good one, IMO). Roland Mas has packaged > SourceForce for Debian, and as far as I understand it's installed on > some machine (SPI's, not Debian's). The idea here would be to create > tasks that can be taken by different people, which would gain us two > things: splitting the work into small realistic chunks and giving us a > scoreboard where we can come and say "yes, this was checked". This can > also serve as a system where people can simply report "I tested this > particular task on such a system". Of course there's at least one > flaw: we need to push other developers arround in order to get the task > list populated. You know there is some rudimentary task manager on qa.debian.org. From my experience I would say that it does not help at all. When I have some time at my hands I would like to say: "Okay, I have 30 spare minutes". In that case I would need to get something what I can do - fast. Even if it is an hour the time to check the BTS, ask on irc, if somebody else is working on it etc. is just too long. With the changes to the BTS it has become even harder to check if somebody is working on a bug since I have to wait minutes for every page of the bts. Looks like only master is generating the BTS pages right now and that machine is quite overloaded if you ask me. It was better when there were static pages of the logs around. I think we need to fix this problem but I have no idea how. A quick look at some of the BTS script shows that there is a lot one can optimize. For example the bugreport.cgi script seems to load the full list of maintainers on each invocation which looks unneeded to me. > For now, it seems the most urgent task in this yet non-existant list is > to test the boot-floppies. For *woody*. No, not debian-installer. > The woody boot-floppies. The people doing QA should forget about > debian-installer at the moment. AFAIK, only Adam di Carlo is really > working on them (I'll be glad if I get corrected). The first thing to > do to get this going is *building* the boot floppies out of CVS. I'm > not sure if that's doable atm. Adam and the b-f team have done a great > job documenting all this stuff, so it's not that hard to get up and > going. Probably the best way to get this started is having the boot > floppies built on a regular basis, with a list of changes wrt to the > last built (trivial with cvs2cl) with, perhaps, an addiotional list of cvs2cl? What is that? > stuff to actually test. If you ask me the most urgent thing for woody would be to replace the boot-floppies. Better release late then release something as antique as our potato installation system. The core has not changed much since hamm if I am not mistaken and there were so many people working on it, I am not sure if anybody does really know what is happening in all the parts of it. > The other question that popped up is "what's holding up packages in > unstable and what can QA do to help that?" > > It's not hard to tell that I've made a lot of this mail on the spot, > mostly triggered by ideas thrown on #d-d. I'm not trying to get things > going on *right now*, but to get a discussion started about how to go > from here. The general consensus is woody won't get out of the door > unless it starts to get (systematically) tested soon. Sorry, but I don't know a better test than real world usage of the distribution. Most developers are running unstable so that problems are often found very early. I installed testing on a few machines in university without any problems. If you ask me our distribution has still the highest quality. Still, I see some problems. For example we should try to find all packages which are unchanged since, say, slink and try to recompile them or at least check if they are still working. Those that do not work anymore should be moved to orphaned. cu Torsten
Attachment:
pgpKPgUsfEY_a.pgp
Description: PGP signature