Auto-reject tags and Test-Against coverage

[Niels Thykier <niels@thykier.net>]

Hi,

I am considering to add a section in t/COVERAGE for auto-reject tags
without a "Test-Against" test.  I strongly suspect that bugs like
#661561 could have been avoided even with a trivial Test-Against test.

My only real concern is "testing applicability".  Some of current
auto-reject tags are rather difficult to "test against" in a
meaningful/good way.
  Consider, no-version-field[1] as an example - the entire test suite
with one (possibly two) exception is a "test against" no-version-field.

Also, the defaults in our test suite (or debhelper) creates packages
that are (effetively) tests againsts tags like bad-package-name and
file-in-etc-not-marked-as-conffile.  In a few cases we can make a
slightly more interesting test than the "default"[2], but I suspect it
will (still?) smell of "no-value busy work".


So, do you think we should start doing "test-against" for auto-rejects?
 If we allow exceptions, under what conditions do we allow them or is it
a case-by-case basis?  It is acceptable to use (e.g.) t/tests/basic/ as
test-against for "trivial" tags (for some definition of "trivial")?  etc.

~Niels

[1] Other examples include no-*-field, dir-or-file-in-*,
package-not-lowercase, package-has-no-description, ...

[2] e.g. spice up the package name/version for "test against"
bad-package-name/bad-version-number