autopkgtest'ing against multiple Python versions

[Jakub Wilk <jwilk@debian.org>]

I started adding DEP-8 tests to my packages. It worked reasonably well 
so far (after I fixed #648148 localy...). However, I have doubts about 
which strategy to choose when it comes to testing with multiple Python 
versions. The following schemes come to my mind:

1) Test only with default version:

	Tests: test-default
	Depends: python-pet-module, python

This is very straight-forward to implement. Also, it looks like that 
soon we'll have only one supported Python 2.X version anyway, so not 
testing against non-default versions wouldn't be a big deal.

2) Provide one test that is run against all supported versions 
(pyversions -r):

	Tests: test-all
	Depends: python-pet-module, python-all

This is also very easy to implement. The disadvantage is that in some 
cases a (non-root) user won't be able to run any test even though he has 
some (or maybe even all) Python interpreters installed.

3) Provide one test that is run against all installed versions 
(pyversions -i):

	Tests: test-installed
	Depends: python-pet-module

This is a variant of 2 which is more friendly to users who can't or 
don't want to install extra packages. It's also less useful for users 
who would install test dependencies automatically anyway (since then  
you do want to install python-all). This issue could be fixed if 
autopkgtest supported weak dependencies, e.g.:

	Tests: test-installed
	Depends: python-pet-module
	Recommends: python-all

4) Provide two tests:
- for testing with default version;
- for testing with all versions.

	Tests: test-default
	Depends: python-pet-module, python

	Tests: test-all
	Depends: python-pet-module, python-all

It his case one could always run at least the first test. But on the 
other hand the second one is wasteful...

5) Provide two tests:
- for testing with default version;
- for testing with non-default versions.

	Tests: test-default
	Depends: python-pet-module, python

	Tests: test-nondefault
	Depends: python-pet-module, python-all

It's a less wasteful variant of 4. However, it's also more difficult to 
implement.


What do you think? I'm currently leaning towards 3.

--
Jakub Wilk