[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

gotchas when running tests via pybuild?



Hi,

I'm working on python-eventlet, to fix a FTBFS bug and an
incompatibility with python3.9. I got both fixes ready, but I'm not
fighting with the fact that the test suite fails randomly. In my
experiments, the test suite fails ~40% of the time, but *only when run
during the build* (!!).

- I added a testsuite run to autopkgtest, and it consistently passes,
  100% of the time.
- If I run the same command as pybuild runs manually (python3 -m nose)
  on a fully patched tree, it passes 100% of the time.
- Only during the build -- when executed automatically by pybuild -- the
  test fails ~40% of the time.

The failures are the typical failures you get when testing concurrency
code: timeouts, race conditions, etc, and it happens on different tests
every time. What I don't quite understand yet is why this _only_ happens
during the Debian package build.

I already checked that the tests don't use anything else from the source
tree beyond the tests themselves and the python modules. For example the
autopkgtest copies tests/, and only it, to a temp directory and runs the
tests from there; and works every time. So in principle I wouldn't need
to have an explicit testfiles file.

Does anybody have an insight on cases like this? Are there any details
that I'm missing?

If anyone wants to try it, the git repository is up to date.

Attachment: signature.asc
Description: PGP signature


Reply to: