[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: DEP-8 pseudo-restriction "hint-dpkg-testsuite-triggers"

Hi Ian,

On 18-06-18 13:21, Ian Jackson wrote:
> I wrote:
>> 4. Can we have a way to trigger tests from updates of non-direct
>> rdepends ?  At some point in the future maybe we will run tests of
>> whole batches of updates and then have some algorithm to chop out
>> what the failures are caused by, but for now it would be useful to
>> be able to declare a specific indirect dependency for test trigger.
>> Maybe an XS- header field ?
> In dgit, I wanted to do this because ISTM that some of the most likely
> sources of regressions are a handful of indirect dependencies.

Just curious, but if they are so important for your package, how come
they end up being only indirect dependencies?

> Mattia Rizzolo pointed out:
>> On Thu, May 03, 2018 at 10:38:45PM +0200, Paul Gevers wrote:
>>> Just add it as a test dependency in one of your tests?
>> Just to share a bit that doesn't seem to be of public knowledge:
>> .dsc have a Testsuite-Triggers field that is autopoulated from the
>> d/tests/control file (IIRC).  I believe you are looking exactly for
>> this field.
> I wanted to add some things to Testsuite-Triggers, rather than replace
> the field, so I couldn't just supply it in debian/control.  Instead, I
> am adding, in debian/tests/control:
>   Tests: hint-testsuite-triggers
>   Tests-Directory: tests/tests

Tests-Directory is cruft if you don't expect the test to be run anyways.

>   Depends: gnupg, patch, diffutils
>   Restrictions: hint-testsuite-triggers
> This works, in the sense that it adds the information I wanted to the
> Testsuite-Triggers generated by dpkg-source.  And it should cause all
> DEP-8 test runners to skip the test because of the unknown
> restriction, so it won't add any spurious test load.

You are forgetting to mention the rationale for not adding this to any
existing test. I think I know, but let's see if you have the same in mind.

I would prefer it if we could think of a better way. We are currently
discussing¹ if the exit codes of autopkgtest shouldn't be used further,
e.g. by britney, and one case I may want to consider a regression is if
an autopkgtest (without source change) went from PASS (exit code 0) to
PASS with SKIPped tests (exit code 2). If a test has SKIPped tests by
design that would be a shame.

> I chose to invent the restriction name `hint-dpkg-testsuite-triggers'

                                               ^^^^^ dropped below.

> for this purpose.  Should this be added to README.package-tests.rst ?
>   hint-testsuite-triggers
>      This test exists purely as a hint to suggest when rerunning the
>      tests is likely to be useful.  Specifically, it influences the
>      way dpkg-source generates the Testsuite-Triggeers .dsc header

                                                    ^^ typo

>      from test metadata: the Depends for this test are added to
>      Testsuite-Triggers.
>      The packages listed as Depends for this test are usually indirect
>      dependencies, updates to which are considered likely to cause

I don't like the word "likely" here. Have some trust in your peers. At
most this is "elevated risk". Or "have in the past led to regressions"

>      regressions in other tests defined in this package.
>      There is currently no way to specify this hint on a per-test
>      basis; but in any case the debian.org machinery is not able to
>      think about triggering individual tests.

Are you considering the case here where we would trigger one test out of
the autopkgtest for pkg-a and another for pkg-b? Before that is
possible, we would need to change the Testsuite-Triggers mechanism again.

>      The test with the hint-testsuite-triggers restriction should not
>      actually be run.  Future systems which understand per-test update
>      triggering should treat the Depends of the test with this
>      restriction, as triggering packages for all tests in this
>      debian/tests/control.
> I haven't yet pushed the change with this restriction name in it,
> anywhere, so if people think a different name is better I'm open to
> suggestions.

I don't really care what name this carries though if we go for it.


¹ Weird place, but here:

Attachment: signature.asc
Description: OpenPGP digital signature

Reply to: