[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Python related autopkgtest anti-pattern

On Thursday, March 19, 2020 6:40:00 AM EDT Simon McVittie wrote:
> On Wed, 18 Mar 2020 at 18:32:22 -0400, Scott Kitterman wrote:
> > We want the tests to run against all versions, but the way to do that is
> > to
> > have your test depend on python3-all (to make sure that all supported
> > versions are installed) and then use the -s flag for py3versions vice -i.
> >  So (in one common pattern) this:
> > 
> > for py in $(py3versions -i); do
> > 
> > changes to:
> > 
> > for py in $(py3versions -s); do
> I think part of the problem here is that py3versions -i treats
> python3.*-minimal as being "installed". Could it perhaps not do
> that? It is true that they are installed in the sense that there is a
> /usr/bin/python3.*, but it isn't a complete, functional version of Python
> (most of the standard library is missing), so many tests are going to
> fail in it.

That's been the behavior since I ported pyversions to python3 about a decade 
ago and it is the way pyversions worked long before that.  I agree it's not 
extraordinarily useful for the reasons you point out, but I don't think there 
is a lot of benefit in changing it at this point.  Personally, I'm not 
convinced the minimal binaries have a lot of value.  I think that's a better 
place to look (not that I expect that to change either).

> Normally, python3.*-minimal would be installed if and only if python3.*
> is, but the qemu backend for the autopkgtest framework requires either
> python3-minimal or python-minimal installed in the VM (to run a small
> Python program to transfer data between the container and the host), so
> the autopkgtest setup script installs python3-minimal, and autopkgtest
> containers will typically have python3-minimal installed (even those
> that are nothing to do with the qemu backend). When testing has an older
> default version of python than unstable, for example 3.7 and 3.8 right
> now, that will result in python3.7-minimal remaining installed after
> python3{,-minimal} gets upgraded to 3.8.
> Perhaps autopkgtest should be using python3 instead of python3-minimal,
> although that would make it impossible for an autopkgtest to detect a
> missing dependency on python3.

I think that's solving the wrong problem in any case.

The fact that these tests are failing right now makes them relatively easy to 
find, but even if they weren't failing, I still think testing against installed 
versions is wrong.  Tests should have an intentional scope, not act of 
whatever happens to be present.  For python related packages that means 
testing against all supported interpreter versions.

This is easily and reliably achieved by using py3versions -s with python3-all.

I think this is true even for applications that will only ever actually be 
used with the default python3.  As an example, in xml2rfc I use the pattern 
I'm suggesting here so that when a new python3 version is added as supported, 
it gets tested right away and I know if there's a problem long before it 
becomes the default.

Scott K

Attachment: signature.asc
Description: This is a digitally signed message part.

Reply to: