keeping a fixed bug fixed (was Re: proving a bug is gone)
Manoj Srivastava <email@example.com> wrote:
> Writing regression tests for real is a hard, painstaking effort,
> often requiring intimate knowledge of the code, and often needs to be
> tied up with the code itself, changing as the source changes.
I'm not talking about a complete regression test suite here. I'm talking
about simple test cases. If the code dumps core under some condition,
reproduce the condition and see if it still dumps core. If it's not
easy to write a test, put it on a checklist and punt the issue for later.
Originally, I was thinking that the hard tests would be relatively rare,
so that when you couldn't write a test you should call attention to
the case. However a number of people have said that that's not the way
things are -- so I'm happy with putting the case on a manual checklist.
I don't think that putting the test case on a checklist is such a
difficult task that the bug should be closed without updating the list.
The only exception I can see is where the bug can't be reproduced at
all. And I'm wondering if we ought to have a special way of closing
those kinds of bug reports [to enable later analysis].