[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#932862: lintian: check for autopkgtests that do cmd --version/--help but don't have Restrictions: superficial



Control: tags -1 - moreinfo

On Wed, 2019-07-24 at 11:25 -0300, Chris Lamb wrote:

> Thanks for this idea and for mentioning the "superficial" Restriction;
> I was not aware of that. I guess my question at this point is how you
> see this interacting, if at all, with the: no-op-testsuite tag?

I would say that superficial tests do provide a small amount of test
coverage (command-line parsing and options processing), while no-op
commands do not test the package in any way. So superficial tests do
provide a small amount of value. I think that this case is distinct
enough from autopkgtests testing /bin/true that it should be a separate
tag, mainly so that the two issues can be of different severity and
have different explanatory text. Personally I'd promote no-op-testsuite
 to error (and autoreject such packages) and use warning for this one.

> They are, of course, reporting on different things but they
> would somewhat overlap in intention and perhaps merging them might be
> worth considering, or perhaps separating them out even further.

I think that what we can surmise about the intent of the people
creating these errors and thus the lintian info text associated with
these tags is going to be different enough to warrant separation. For
example no-op-testsuite is at best naivety and at worst intentionally
taking advantage of the reduced testing migration delay, while the
superficial tests can either be a precursor to deeper testing or just
the best one can do for some packages.

For example a set of packages I maintain for Discord support can only
be tested by humans since testing them would require a machine to pass
reCAPTCHA, register two accounts, send messages between them and then
delete the accounts. Even if the first requirement were feasible,
testing like this is probably constitutes abuse of the Discord service.
So superficial testing is the best that can be automatically done.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Attachment: signature.asc
Description: This is a digitally signed message part


Reply to: