Hi
On 18-06-18 18:27, Ian Jackson wrote:
> Simon McVittie writes ("Bug#901804: autopkgtest: consider using exit status 8 ("no tests found") if every test was ignored"):
>> On the Gitlab MR, Paul Gevers wrote:
>>> And maybe a regression from 0 to 2 (without changes to the autopkgtest
>>> that runs) should also be a regression for britney (if it is not caused
>>> by a flaky test :( ).
>>
>> I'm not so sure about this. If debci was previously giving tests
>> a particular capability bounding set in their LXC containers, and a
>> security fix to lxc removes a capability that turns out to be exploitable,
>> then tests that were exercising that capability will have to be skipped
>> (ideally they'd already be 'skippable', and exit 77 if they don't have
>> it), so they'd appear to "regress". That seems undesirable.
> The *point* of SKIP is that it isn't a regression or a failure. If a
> feature is removed, and its absence causes SKIP, and this ought to be
> treated as a regression, then the feature ought to have its own test
> (perhaps, somewhere entirely else than the package whose test is now
> SKIPped).
I am convinced.
Paul
Attachment:
signature.asc
Description: OpenPGP digital signature