Hi Drew, On 26-08-2019 05:24, Drew Parsons wrote: > Hi, I'm wondering what the policies or principles are around > "slow-running" debci tests. Currently, we show on that page everything that takes more than one hour. > Currently dolfin, for instance, is catalogued as slow-running at > https://ci.debian.net/status/slow/, with its tests taking 1 hr 16 mins. Exactly. > Is this considered a problem? Or should the full suite of tests just be > allowed to run? It's not a problem that packages are listed on that page, especially when they PASS. It's a way for us to keep an eye on potential problems. E.g. every failure that takes 2 hours and 47 minutes is likely a timeout issue. But we want to spot potential issues before they become a problem, i.e. the autopkgtest timeout. > In the case of dolfin, again for example, tests cover not only unit > tests but also run demo scripts. Most of the test time is in the demos, > which take about 15 min each full set, run 4 times over [C++, > python]×[serial, mpi] making an hour in total demo time. It would be > simple enough to run a subset of demos if we considered it important to > reduce total test time. Until a huge amount of packages start to have very long running tests, this is not a problem. Please be advised that spreading your tests around multiple stanza in d/t/control even circumvents the 2:47h timeout, as that timeout is *per* autopkgtest. Paul
Attachment:
signature.asc
Description: OpenPGP digital signature