On Jul 22, 2010, at 10:00 AM, Yaroslav Halchenko wrote: > >On Thu, 22 Jul 2010, Barry Warsaw wrote: >> >But assuming that in longer run we agree on how we invoke >> >unittesting for Python modules we ship[...] > >> I propose this be spelled: 'python setup.py test'. > >does setup.py test just do unittesting? or + doctesting, regressions, >examples, ... ? It depends on the package. IOW, this command should run whatever core set of tests the package defines. >doesn't it re-build module first? It builds it, but shouldn't rebuild it. >do you advocate also 'module.test()' may be for testing already >installed module whenever setup.py is no longer there? That's an interesting question. I'd rather see runpy supported so you can do things like 'python -m foo --test', but I haven't thought about it (or experimented) in detail yet. >Altogether setup.py test sounds like a viable solution for testing >within build tree (thus rebuilding the module/extensions); and I did >see some projects to provide similar targets (some times called >differently, e.g. nosetest, although 'test' sound better ). > >But it doesn't provide means for testing of already built/installed >module. Somewhat native approaches for this: > * having module.test, so we could just > python* -c 'import module; module.test()' See above. > but that implies upstream willing to do so or our duty to hack it up > * using > nosetests module > but that is not guaranteed to work. Although nose's discovery is > nice etc, project which is not 'nose aware' might get all kinds of > side-effects while invoking tests this way. > on the other hand > nosetests --with-doctests ... > is very nice, but once again, some projects might not care about > doctests So, I would personally let the upstream decide, and in fact make it their responsibility to support the interface. As I said, I'd put some good carrots under their noses <wink> so that they'll *want* to support it, and I'd add whatever is necessary to make it easy (i.e. improve doctest discovery). Of course there's nothing preventing DPMT from adding those hooks. I'm really just proposing a common command line interface that Python modules and applications are encouraged to support. >So, altogether, all those approaches require all upstreams to agree >upon the way things get tested... which I doubt would happen. Well, as I said, if we make opinionated decisions, promote best practices, and provide incentives, many -- and I think most -- will. You're never going to get everyone to do anything, but that's okay, they just won't get the big shiny Happy Icon on Cheeseshop, etc. >To account for heterogeneity among the projects why instead we just >not agree upon few debian/rules targets, e.g. > >debian/rules test-available-* > to run unittest battery (possibly with doctests and > whatelse) with module.test() or > nosetests or whatever upstream's native way. I think that's fine too. What I have in mind is really Debian (or actually distro/platform) independent, so for example if snakebite really ever exists, we could run the tests on all available platforms. Then debian/rules would usually just invoke 'python setup.py test' or 'python -m foo --test', but that could be overridden for the outliers. > Rule might need to 'cd debian' or some other directory to assure > that local source tree is not the one getting tested. > > -* is there for a python version to be used ease test-* below > >debian/rules test-available > to run against all supported python versions > >and, dependent on successful build > >debian/rules test-* > which possibly just adjust the PYTHONPATHS and call corresponding > $(MAKE) test-available-$* >debian/rules test > to test them all > >Then for the validation of the packages which could just use >test-available and test targets within extracted source package Sounds good to me! -Barry
Attachment:
signature.asc
Description: PGP signature