[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Review of debusine's autopkgtest related API

Hi,

On 19-10-2023 15:34, Raphael Hertzog wrote:

On Sat, 14 Oct 2023, Paul Gevers wrote:

As on of the maintainers of the infrastructure, I have some ideas or
recommendation to add too:

* make sure you can block tests


Do you mean specific tests within a source package or all the tests of a
source package together?
Well, *at least* the latter. If you want to spend time on failures, the former would be what you want. I wouldn't want to dive that deep. 


In general, I do hope debusine to be a bit more "host"-agnostic, I
would like to be able to express scheduling requirements per
package/suite/architecture with things like:
- minimal number of CPU cores (to ensure reasonable build times for large
   packages for example)
- minimal RAM size
- minimal size of available diskspace
As long as the outcome None from that selection is well handled. And my suggestion is mostly for the infra maintainer, not (only) the requestor. Because ideally an autopkgtest that has somewhat elevated requirements *should* be checking *too* and bail out (exit 77 with the skippable restriction) although for unpacking that happens too late of course. 


   o also *after* the job has been scheduled. As an example, recently
     sbuild started to kill our worker service instances while we had
     quite some backlog, which we had to fully clear to resolve the
     issue.


You mean that you would like the allow/deny lists to also apply on tests
that are already scheduled and are in the queue, right?


yes


Wouldn't that need
be solved by having tools to cancel the problematic tests, i.e. instead of
having to flush the whole queue, we could ask "cancel all autopkgtest jobs
that will run sbuild tests" ?


Indeed. debci currently doesn't have that.


   o as the infra maintainer, e.g. you want to allow more time on slow
     archs: armel and riscv64 at this moment


Do you hardcode bigger values or are you using --timeout-factor?
We currently have --timeout-factor=2 for riscv64 and I'm considering it for armel (as tests with lots of floating point operations are extremely slow there). 


* Maybe autopkgtest should grow an restriction to have test only run
   when explicitly declared on the command interface, to enable writing
   tests that don't need to run by default, but are nice to have during
   development.


Shall I file a wishlist bug for this against autopkgtest?


Sure, but don't wait for it to be implemented.

Paul

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature

Reply to: