Re: Auto reject if autopkgtest of reverse dependencies fail or cause FTBFS
Barry Warsaw writes ("Re: Auto reject if autopkgtest of reverse dependencies fail or cause FTBFS"):
> There are occasionally good reasons why an upstream's test suite can't be run
> at build-time, and in those few cases I will run them in an autopkgtest. But
> generally, I think the two are there to test different aspects or lifecycles
> of the package.
This depends very much on the nature of the program.
If it's a self-contained library or tool, full of algorithms and with
few entanglements with dependencies, and the upstream test is mostly
algorithmic "right answer" tests, then what you say is true.
But much software that we have is not like that at all. dgit is a
very good example here.
dgit has many dependencies and is easily broken by "unexpected
behaviours" in those dependencies. Most tests uin the test suite
exercise a mixture of code in dgit, and the dependencies, and the
interaction between those.
There isn't really a distinction between things that are useful to
test during development, and things that are useful as autopkgtests.
Even in software that looks very algorithmic it can be useful to use
the upstream test suite as the autopkgtest. I recently packaged Simon
Tatham's exact real calculator, `spigot'.
spigot's build system as supplied by Simon runs a moderately extensive
test suite.
spigot depends on gmp. If there were some bug in certain gmp
functions, that gmp bug might cause a regression in spigot. How would
I detect this ? Well, I can publish the spigot test suite as an
autopkgtest test. I haven't checked, but I imagine that Simon's test
suite tests most of the interesting codepaths in spigot, so it
probably also tests most of the intersting gmp calls that spigot
makes.
My view is that for most packages, if the upstream test suite _can_ be
made to run against the installed version, and isn't annoying in some
way, it should be advertised as an autopkgtest.
One may want to add other autopkgtests. For example, I added another
test for spigot, to check that it is actually using the gmp
integration, rather than its fallback internal bignum library. AFAICT
this is actually a check that could be done at build time, with
spigot's current fallback behaviour, but autopkgtest is a convenient
place to add Debian-specific tests, and it will spot if spigot gets
dynamic fallback.
Ian.
--
Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Reply to: