W dniu 15.06.2011 02:28, Yaroslav Halchenko pisze:
On Wed, 15 Jun 2011, Zygmunt Krynicki wrote:That's different. IHMO you ask for make dist-check AFAIR (my automake-foo is getting old). Testing installed stuff is often harder and not supported as we don't want to include tests in the package (for build-deps vs install-deps). Source layout is also different from installed layout. Some data files required for testing are also not present in the production environment.valid points BUT so far the test batteries usually included with the python modules I package relied on reasonably small amount of data files, so upstream usually doesn't mind have them installed alongside with the tests installed alongside with the modules themselves. Benefits: any user can run tests later on (usually by "import m1; m1.test()") to verify that everything on his system is still functioning as promised (e.g. there were no upgrades of the 3rd party modules, breaking the functionality of the module in question; or effects of user-specific environment/settings/etc which could not be foreseeing during package building)
I agree in the value but I don't want to assume that it is a default practice or requirement. I package/hack on the webapp (django) side of tests and after being spoiled with the goodness of python-testtools and python-testscenarios I ventured to create equivalents (compatible wrappers really) for django. This means that my packages build depend on 5 additional packages just for testing. In addition to that (due to the way django works) each django application I make (a python package really) has a helper django project (another python package) that allows it to invoke the test. In practice it means that:
1) Running tests is non-trivial. It works when you do it via setup.py test but it was not easy to get to that point. It's not something you can reproduce easily after installation without shipping the setup file somehow and (unfortunately) setup.py often uses find_packages() to discover where the code is. I did not check this but I suspect it might break in subtle ways as it was never intended to be used post-install.
2) Test code itself is large and has significant dependencies. Separating that from production deployments is important. Perhaps some python packages/modules are easy and have little or no extra data files but we cannot assume that's the case for all modules.
I'm not trying to say "it's a bad idea", I want to figure out how to do it properly.
Testing that python setup.py sdist output can still run tests is valuable but falls under release management - not packaging.oops -- I bluntly believed that we are taking care about both aspects in Debian ;-)
Well joke aside you cannot fix the tarball that people release on pypi with half of the test code and data missing just because their MANIFEST.in was flaky.
In that sense we are not doing "release management" as there is no equivalent of dist-check without being able to fix .orig.tar.gz.
If we are talking from a perspective of upstream developers that also maintain their packages then I would *love* to see setup.py sdist-test and would use it each day.
Thanks ZK