Hi Nicholas On 02-04-2022 00:13, Nicholas D Steeves wrote:
On 24-02-2022 06:13, Nicholas D Steeves wrote:Paul and Debian CI team, do you know if a dummy alsa driver (and dummy sequencer driver) can be enabled on DebCI systems?You are in control of your testbed, can't you do this in your test? Or can you only do this in isolation-machine testbeds (not sure how this driver thing works)?I was thinking that this would require a `modprobe snd-dummy`, and I'm assuming loading arbitrary kernel modules is blocked on DebCI systems.
We don't "block" anything. But on ci.d.n things need to work in lxc for now. Again, I'm not into the details, but I think indeed that you need isolation-machine for that.
It's been a few months since I contacted the CI Team about isolation-machines, but they're not yet ready for use either.
That's still correct, and I don't expect that to change in the short term. We're more focused on adding other architectures to enable the Release Team to test all release architectures and maintaining the infrastructure as-is. That's already more work that I'd expected upfront.
My request for snd-dummy on DebCI isn't arm64 specific; I noticed that jackd2 doesn't have autopkgtest enabled, and hypothesised that missing audio hardware was the reason why. Jackd2 maintainers are now in CC because would know better than I :-)
I see a lot of flaky tests with jack mentioned in the error (failure to start or something). Maybe that's remotely related?
To be honest, I don't really like tweaking the hosts, as other autopkgtest instances outside our control would need to learn to do this too.I completely understand, and the only way this request would be justifiable is if it provided a meaningful boost in quality assurance for audio-related packages.
I'm wondering if this warrants a new autopkgtest restrictions (probably not): isolation-machine-or-needs-${group-of-defined-kernel-modules}. I have the feeling that could become a big list for all kind of use cases.
When I run autopkgtests locally they now error due to missing audio hardware where previously they passed, and this wasn't the case when the package was uploaded.The test on ppc64el now passed after several failure due to segmentation faults. Not sure if this is now a highly flaky test, or just that the issue has been fixed in the mean time on ppc64elWhen I contacted upstream, I learned that the cause may be that the CLI utilities are poorly maintained. Is this sufficient justification for disabling the autopkgtests ie: that it's due diligence and not laziness?
Well, if a test is truly flaky for whatever reason, ci.d.n and the Release Team agree that the test should either not be run or marked as flaky. The latter only has value if some human still regularly checks. The same goes for failure on a particular architecture. If you believe the failure is due to CI and not because the package itself is wrong, then just don't test there (Architecture field in d/t/control). If the package is broken and can't reasonably be fixed, have it removed from that architecture (in unstable; ftp.debian.org pseudo package) and don't build on that architecture anymore. In that case CI found out that the package is broken and that's a good thing.
Thank you again for having this conversation. I want to support our CI team and technical excellence rather than just "unblock migration to testing", but will of course defer to your recommendations.
Sometimes you need to (for the time being) take a practical stance. I'm not saying yet that we'll not enable the snd-dummy module (I hope for follow-up), but in the present case, if the test can't reliably run on the present infrastructure, and nobody is promising a solution on the short term, you should not make your own live too hard and you should not run the tests for now. Or prepare for the isolation-machine situation (I think Ubuntu has that on most architectures) and hope it will arrive someday in Debian too.
Paul
Attachment:
OpenPGP_signature
Description: OpenPGP digital signature