On Tue, Feb 01, 2011 at 06:17:45PM -0500, Yaroslav Halchenko wrote:
> Unfortunately the core aspect of the current autopkgtest - relying on
> tests in source packages -- imho to be not the ideal solution to
> target both sides of the userbase (i.e. maintainers/QA vs mortals).
Thanks for this "related work" research Yaroslav, I found it to be very
helpful for this discussion.
> Ian -- could you please unroll your arguments a bit? I still do not
> see why source packages are beneficial besides build-time testing
> (which we often do already without any additional framework)
Not that you are explicitly saying they are not, but let me stress that
support for automated testing shipped in the source package is
useful. That way maintainers can keep them in sync with the package,
pretty much as software developers keep test suites in sync with the
code base, by committing them along side the code itself (and possibly
even doing feature commits that check in both the code and the
corresponding test at the same time).
What you are requesting, if I got it right, is support for having the
*possibility* of shipping tests elsewhere, possibly even not in a
package available in the archive. That can indeed come handy in various
scenarios, such as a separate test team with their own batteries of
tests (e.g. someone mentioned that other distros have distro-wide
regression test suites; such initiative can benefit from the extra
feature you are proposing). I also observe that the dependency
interface in the current spec is already pretty sane to handle that: if
the tests are shipped as part of the source package they can benefit
from the "@" syntax, otherwise they'll need to be explicit.
All that considered, I'd like to know the rationale of this initial
design choice as well. In particular, it would be nice to know if anyone
see disadvantages in having *also* (rather then "instead") support for
running tests which are not part of a source package.
> So, where do we start/continue sharing the thoughts on a tentative
> DEP? ;)
Let's see first if we have all the arguments on the table already,
thanks to this thread. I'm willing to co-drive a DEP to finalize the
spec, although I definitely need helping hands (hint, hint!).
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Quando anche i santi ti voltano le spalle, | . |. I've fans everywhere
ti resta John Fante -- V. Capossela .......| ..: |.......... -- C. Adams
Attachment:
signature.asc
Description: Digital signature