Re: keeping a fixed bug fixed (was Re: proving a bug is gone)
Hi
>>"Raul" == Raul Miller <rdm@test.legislate.com> writes:
Raul> Manoj Srivastava <srivasta@datasync.com> wrote:
>> Writing regression tests for real is a hard, painstaking effort,
>> often requiring intimate knowledge of the code, and often needs to be
>> tied up with the code itself, changing as the source changes.
Raul> I'm not talking about a complete regression test suite here.
Raul> I'm talking about simple test cases. If the code dumps core
Raul> under some condition, reproduce the condition and see if it
Raul> still dumps core. If it's not easy to write a test, put it on
Raul> a checklist and punt the issue for later.
Not all bugs produce a checklistable item.
Raul> Originally, I was thinking that the hard tests would be
Raul> relatively rare, so that when you couldn't write a test you
Raul> should call attention to the case. However a number of people
Raul> have said that that's not the way things are -- so I'm happy
Raul> with putting the case on a manual checklist.
Raul> I don't think that putting the test case on a checklist is such a
Raul> difficult task that the bug should be closed without updating the list.
Raul> The only exception I can see is where the bug can't be reproduced at
Raul> all. And I'm wondering if we ought to have a special way of closing
Raul> those kinds of bug reports [to enable later analysis].
Let us see how relevant this is Here are my list of resolved
bugs. Let us see..
29 bug reports.
2 cases whre reproducers were possible, one whereit is available
6 possible test, with 3 being noted as tricky
23 cases where one could put thinkgs on a checklist, 9 cases where it
just clutters things up. Therefore only 14 out of 29 cases that even
a checklist makes sense.
That is less than 50% of my closed cases warrant even a
checklist item
No, I strongly object to bureaucratic junk required just to
close bugs, given that not all cases could one even create a
checklist item.
manoj
#25609: Gnus: prerm script failure make it impossible to upgrade/pruge
Package: gnus; Severity: important; Reported by: Manoj
Srivastava <srivasta@datasync.com>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: rewrote scripts totally.
Reproducer: no
Test: no
Checklist: What do we checklist? That the old scripts have not come
back?
#23313: psgml (xml-mode) can't parse xml:space, other xml.* tags
Package: psgml; Reported by: Mark Eichin
<eichin@kitten.gen.ma.us>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: New upstream release
Test: Umm, Dunno how to write one
Reproducer: maybe a document on which it fails could be provided. I
don't have one.
Checklist: I guess. Especially if we get a reproducer.
#25585: gnus depends on xemacs20, but xemacs20 doesn't exist
Package: gnus; Reported by: Matthias Klose
<doko@cs.tu-berlin.de>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: Emacs20 now available
Reproducer: no
Test: no
Checklist: no
#26215: Bug? in libcgi-perl
Package: libcgi-perl; Reported by: Manoj Srivastava
<srivasta@datasync.com>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: Rewrote function
Reproducer: yes
Test: yes
Checklist: yes
#26536: gnus: /usr/lib/emacsen-common/packages/remove/gnus
should return 0 if it does nothing
Package: gnus; Reported by: phil@hands.com; Done: Manoj
Srivastava <srivasta@datasync.com>.
Solution: Added a return 0
Reproducer: not applicable
Test: I guess so but is tricky,
Checklist: One could, but it would only clutter up the real tests
#26843: kernel-package: FLAVOUR not checking for UPPER CASE (not allowed in pkg names)
Package: kernel-package; Reported by: <tbittih@pal.xgw.fi>;
Done: Manoj Srivastava <srivasta@datasync.com>.
Solution: converted t lower case
Reproducer: not applicable
Test: Umm, not easy.
Checklist: Yes.
#27186: pkg-order: pkg-deptree can't handle "libstdc++2.8"
Package: pkg-order; Reported by: Richard Braakman
<dark@xs4all.nl>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: pilot error
Reproducer: not applicable
Test: no
Checklist: no
#27311: kernel-package: flavours code broken
Package: kernel-package; Reported by: Jonathan H N Chin
<jhnc@pfaff.newton.cam.ac.uk>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: rewrote code
Reproducer: no
Test: no
Checklist: I guess.
#27450: latex2html: problem with -html_version 3.2,math
Package: latex2html; Reported by: <kevind@rahul.net>; Done:
Manoj Srivastava <srivasta@datasync.com>.
Solution: pilot error
Reproducer: no
Test: no
Checklist: no
#27521: psgml: Should psgml pre-depend on autoconf?
Package: psgml; Reported by: ssm@debian.org; Done: Manoj
Srivastava <srivasta@datasync.com>.
Solution: run autoconf on machine before packaging
Reproducer: no
Test: no
Checklist: I guess?
#27610: latex2html cannot be installed non-interactively
Package: latex2html; Reported by:
Roman.Hodek@informatik.uni-erlangen.de; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: Made install not ask question
Reproducer: no
Test: no
Checklist: One could, but it clutters up the checking process
#27750: sgml-current-element-name should be buffer-local
Package: psgml; Reported by: Kalle Niemitalo
<tosi@ees2.oulu.fi>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: already in new upstream version
Reproducer: no
Test: not easy, but doable, I think
Checklist: yes
#27842: cvs-buildpackage: use of /tmp/*$$* in an insecure fashion
Package: cvs-buildpackage; Reported by: James Troup
<james@nocrew.org>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: rewrote code
Reproducer: no
Test: no
Checklist: yes
#27937: kernel-package: syntax error
Package: kernel-package; Reported by: <cowboy@vnet.ibm.com>;
Done: Manoj Srivastava <srivasta@datasync.com>.
Solution: fixed code
Reproducer: no
Test: yes
Checklist: yes
#28451: cvs-buildpackage: README seems to be wrong
Package: cvs-buildpackage; Reported by: "Marcelo E. Magallon"
<mmagallo@efis.ucr.ac.cr>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: rewrote docs
Reproducer: no
Test: no
Checklist: One could, but it clutters up the checking process
#28452: cvs-buildpackage: spell check
Package: cvs-buildpackage; Reported by: "Marcelo E. Magallon"
<mmagallo@efis.ucr.ac.cr>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: fixed
Reproducer: no
Test: no
Checklist: no
#28459: cvs-inject(1) typo
Package: cvs-buildpackage; Reported by: "Marcelo E. Magallon"
<mmagallo@efis.ucr.ac.cr>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: fixed
Reproducer: no
Test: no
Checklist: One could, but it clutters up the checking process
#28460: conf_prefix not documented in cvsdeb.conf
Package: cvs-buildpackage; Reported by: "Marcelo E. Magallon"
<mmagallo@efis.ucr.ac.cr>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: added docs
Reproducer: no
Test: no
Checklist: One could, but it clutters up the checking process
#28461: cvs-upgrade acts funny without a debian revision
Package: cvs-buildpackage; Reported by: "Marcelo E. Magallon"
<mmagallo@efis.ucr.ac.cr>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: pilot error
Reproducer: no
Test: no
Checklist: no
#28535: kernel-source-2.0.35: suddenly ships with tarball rather than /usr/src/kernel-source-2.0.35/
Package: kernel-package; Reported by: apharris@onShore.com;
Done: Manoj Srivastava <srivasta@datasync.com>.
Solution: not a bug, added docs
Reproducer: no
Test: no
Checklist: no
#28536: typo in cvsdeb.conf(5)
Package: cvs-buildpackage; Reported by: <mmagallo@jacinta>;
Done: Manoj Srivastava <srivasta@datasync.com>.
Solution: fixed
Reproducer: no
Test: no
Checklist: One could, but it clutters up the checking process
#28723: Make and globbing (alpha)
Package: make; Reported by: Jason Gunthorpe
<jgg@gpu.srv.ualberta.ca>; Done: Manoj Srivastava
<srivasta@datasync.com>.
#28724: Make and globbing (alpha)
Package: make; Reported by: Christopher C Chimelis
<chris@classnet.med.miami.edu>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: added patch
Reproducer: yes
Test: yes
Checklist: yes
#28755: obsolete doc ?
Package: cvs-buildpackage; Reported by: Zane Lewkowicz
<zooko@xs4all.nl>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: Not obsolete
Reproducer: no
Test: no
Checklist: no
#28760: cvs-buildpackage vs. liblockfile0
Package: cvs-buildpackage; Reported by: Zane Lewkowicz
<zooko@xs4all.nl>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: not a bug
Reproducer: no
Test: no
Checklist: no
#28992: make-3.77-3: HAVE_GLOB_H is broken on powerpc
Package: make; Reported by: Hartmut Koptein
<koptein@et-inf.fho-emden.de>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: not a bug
Reproducer: no
Test: no
Checklist: no
#27289: kernel-package could use a nicer prompt
Package: kernel-package; Severity: wishlist; Reported by:
joey@infodrom.north.de (Martin Schulze); Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: changed code
Reproducer: no
Test: no
Checklist: yes
#27390: latex2html: pnmtopng does exist now
Package: latex2html; Severity: wishlist; Reported by:
<reiter@forest.bofh.asn.au>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: changed behaviour
Reproducer: no
Test: no
Checklist: yes
#28514: gnus: postinst is way way too verbose
Package: gnus; Severity: wishlist; Reported by: James Troup
<james@nocrew.org>; Done: Manoj Srivastava
<srivasta@datasync.com>.
Solution: changed code
Reproducer: no
Test: no
Checklist: yes
--
Trifles make perfection, and perfection is no trifle. Michelangelo
Manoj Srivastava <srivasta@acm.org> <http://www.datasync.com/%7Esrivasta/>
Key C7261095 fingerprint = CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E
Reply to: