Bug#851558: autopkgtest: define Restrictions for tests that aren't suitable for gating CI
- To: Martin Pitt <mpitt@debian.org>
- Cc: 851558@bugs.debian.org
- Subject: Bug#851558: autopkgtest: define Restrictions for tests that aren't suitable for gating CI
- From: Simon McVittie <smcv@debian.org>
- Date: Thu, 14 Jun 2018 17:54:41 +0100
- Message-id: <[🔎] 20180614165441.GB29871@espresso.pseudorandom.co.uk>
- Reply-to: Simon McVittie <smcv@debian.org>, 851558@bugs.debian.org
- In-reply-to: <20170122180749.vua6gsuvz4d5j76y@perpetual.pseudorandom.co.uk>
- References: <20170116090619.676ljcrt2opafrmu@perpetual.pseudorandom.co.uk> <20170116113450.GA1595@donald> <20170116152932.wue3ligikht4vb3s@perpetual.pseudorandom.co.uk> <20170122155509.GC1172@donald> <20170122180749.vua6gsuvz4d5j76y@perpetual.pseudorandom.co.uk> <20170116090619.676ljcrt2opafrmu@perpetual.pseudorandom.co.uk>
On Sun, 22 Jan 2017 at 18:07:49 +0000, Simon McVittie wrote:
> Suppose I want to
> test that some tool (ostree or tar or whatever) can back up and restore
> xattrs. I can't test that unless I have some scratch area that lets the
> current user set xattrs.
...
> Or, imagine I want to test a non-http network protocol, perhaps testing
> whether my XMPP client can connect to an XMPP server. As we discussed
> on #851556, we would expect this to work on ci.debian.net, but not on
> Ubuntu's equivalent. I could probe for this with ": | nc xmpp.debian.net 5222"
> or something.
>
> With Automake, I would be able to exit 77 to get the test recorded as
> having been skipped, but with current autopkgtest I would have to exit 0.
>
> This would hopefully affect autopkgtest's exit status the same way as if
> the test had been skipped due to "Restrictions: something-you-dont-support" -
> so if one or more tests exit 77, autopkgtest would exit 2.
I've implemented this at
<https://salsa.debian.org/ci-team/autopkgtest/merge_requests/20>
since an implementation is probably easier to comment on than an idea.
It's really orthogonal to what I first asked for in #851558.
smcv
Reply to: