Re: testing "testing" (was: Implementing "testing")
Tom Musgrove <TomM@pentstar.com> writes:
TM> Would it be possible/feasible to set up a script that would run Lint,
TM> SLINT, etc. (Perhaps a pretty printer as well, that checks for
TM> inconsistent white space (which generally implies the programmer meant
TM> one thing and typed another...) ) and then files related bug
TM> reports?
Oh dear. Automated processes filing bug reports is probably a bad
idea. What if I disagree with lint's assessment of my package's
source? What if I haven't fixed the lint warnings since the last time
your job ran? What if there are different warnings?
A marginally better approach would probably be what wound up happening
with lintian. Note that we do have a package to check for common
errors in packages, but there's no automated process that files bugs
based on its results. Instead, developers are encouraged to run
lintian before uploading their packages; support is in debhelper to do
this, and dh_make causes it to be done by default. Lintian-clean is
good, but not required and not automagically checked for.
Additionally, I wouldn't want to be held responsible because I
maintained a package that had ugly/inconsistent/unsafe code. Is it
the package maintainer's responsibility to, say, rewrite all of GNOME
because its source is ugly? (Well, uncommented, but that's close
enough.)
This isn't to say that I think automated testing is a bad idea.
Pointless testing, like the following:
TM> After a successful compile, it could also run the program and
TM> use a memory shaker, and/or software that gives the program random
TM> input, and again files a bug report for any crashes.
isn't necessarily the best idea. We don't want to set up a situation
where a package passes all of the automated tests, but this just means
it's lint-clean, lintian-clean, and doesn't barf on random input, not
necessarily that it works correctly.
TM> (PS Please CC me in all replies since I am not subscribed to this list)
(If you're going to make a proposal like this, wouldn't you at least
like to hear the discussion on it?)
--
David Maze dmaze@mit.edu http://www.mit.edu/~dmaze/
"Theoretical politics is interesting. Politicking should be illegal."
-- Abra Mitchell
Reply to: