Dale Scheetz <dwarf@polaris.net> writes: > Aside from the fact that you have ignored the testing team in your > schedule (I don't really have a problem with this if you can put a better > team in its place) It was just a first stab at things. I propose renaming the Testing team to the "Release Team", and they would be responsible for producing the final release and doing the final release testing. They take over 100% control of the distribution after the developer testing phase ends. Of course, they can be active beforehand. We need an organized developer testing phase. A small testing team, like what we used last time, doesn't have any reasonable way of testing 1500+ packages. However, it does make sense to have a small close knit team to do final release installation/upgrade testing. > This schedule has far too many milestones that are far > to close together. Several of your important milestones are less than 5 > days appart. Experience with the testing team suggests that this is enough > time to do one simply round of boot disk testing and little else. That's why I called it "fast track". Maybe we aren't capable of doing that. But if we let the release slide a few more months, Debian is going to suffer from the lack of momentum. We're already open to lots of criticism - just try raising some capital for a Debian-based business, and you'll know what I'm talking about. I think the tight schedule might work if everybody is working at it together in a coordinated manner. For our last release, the left hand didn't know what the right hand was doing. The looser release schedule just meant things were allowed to slide around more. Overall, we did a good job - but we ought to try to do better. > As to the overall schedule, it seems to me that if we can get to code > freeze by the first of the year (you have a similar milestone here) that > we can release by Feb 1. OK - just a bit of clarificition. I proposed a "new packages" code-freeze for Dec. 15, so that organized testing by developers would start from there. Somebody (probably a team) would organize who tests what before that - so that "developer" testing could start immediately. That would continue until Jan. 10th, when the core release team would "take possession". Hopefully, all the major bugs will have been identified, and mostly fixed by then. That is a pretty short time scale for developer testing. Perhaps we should add another week. The dates I picked were pretty arbitrary, and should probably be aligned with the weekends. I'm arguing for a compressed testing period, because it's not good to expect the developers to hold all their systems in a frozen, limbo state for several months while we do unorganized testing. > My feeling about the last release and its vast improvement over 1.2 was > that it was primarily the result of our first attempt at a Managed testing > proceedure. While we have a long way to go with the organization of the > testing team, pulling a global reorganization of the release proceedure at > this point will not be helpful to this effort. I agree with your first point - disagree with the second. I'm going to offer some constructive criticism. Please don't take it the wrong way. Although I thought your testing team did improve the release, I don't think it accomplished the goal of a bug-free release. I think the problem was that the testing team was a small, closed group that accepted responsibility for the entire release. It's just too large of a testing job to not involve everybody. We've got 1500+ packages - if each package is tested 5 times - that's ~8000 tests, or roughly 40 tests each (based on 200 developers). A core group of 10 QA testers would have to do 800 tests each in order to accomplish the same thing. That's impossible for volunteers on a tight release schedule. My impression was the core testing group basically did run-throughs on the basic installation and upgrades. This is very essential and we still need to do that. My concern was that major bugs (such as the xdm bug) slipped through - and afterwards you said you never tested that. Basically, the QA team only tested a subset of the Debian product - which is good, but I think we need to do more if we are going to claim a fully tested product. I'd rename the QA team to the "Release team". They'd do release testing, and decide which packages go into the release, and which stay out. Last time, all the packages went into the release - which shouldn't happen, because many of the packages were noticeably broken. The Release Team's job should be easy, if we have a round of developer testing first, which uses the bug system to identify broken packages. The release team can monitor this, and prioritize what needs to be fixed. There is a lot of work in organizing the developer testing - I'll volunteer for that job, if this plan goes ahead. We should write a "Release/Testing Strategy Manual" to codify everything so there aren't any misunderstandings this time around. > While your schedule would probably work in a tightly controlled effort, I > don't think it will work here. Our releases should be a tightly controlled effort. So, when are we going to release 2.0, if ever? We should announce a date, and stick to it. Frankly, I think we've been in good shape since August to make a release. We still need to be very organized, even if we extend the release date a few more months. I don't really want to keep my system in a code freeze for 2-3 months. If we can't pull off a quick release turnaround - that's basically saying the Debian open development model is irretrievably broken, and we all might as well switch to Red Hat. Cheers, - Jim
Attachment:
pgp46cROVdIOp.pgp
Description: PGP signature