On Sat, Aug 21, 2021 at 11:36:49PM +0100, Samuel Henrique wrote: > Hello team, > > I'd like to expose an issue I have with shellcheck, see if there's any > alternatives and/or hopefully motivate someone to fix it eventually. > > As you can see right now on shellcheck's migration excuses [0]: > autopkgtest for initramfs-tools/0.140: amd64: Regression... > autopkgtest for kdump-tools/1:1.6.8.4: amd64: Regression... > > If you look at the logs, you will notice that these tests are failing > because shellcheck is identifying new issues it wasn't before, this is > not the first time this happens. > > So the issue is that other tools are using shellcheck as part of their > integration testing [1], and every new release of shellcheck gets > blocked when migrating to testing due to other packages relying on it. > > Right now the only options seem to be: > A) Wait for the packages to be fixed > B) Contact the release team and ask them to manually unblock it > C) Mark those tests as flaky > > I would like it if autopkgtest[2] could provide a way of saying a > certain dependency can **never** be the cause of a test failure, and I > assume all static analysis tools would want to have that for any > package relying on it for tests. > If the maintainter of such static analysis tool wants to have those > checks as tests, they can add it directly to the package. > > What are your thoughts on it? static analysis tools are intended to run on in source code, while autopkgtest is intended to run against installed packages, where source code is in principle not relevant; we want to know whether the binary packages, as provided in the Debian archive, work correctly. IMO running this type of tools as integration tests in autopkgtest, or even as build-time tests is Wrong™, and should not be done. So I would add another option: D) report bugs requesting that those packages stop running shellcheck as autopkgtest. For example, debci itself used to run shellcheck as part of it's regular test suite, and this caused problems every time there was a new shellcheck version. I them moved that to a separate/custom gitlab ci job. So new shellcheck issues will fail my CI on salsa, but will not cause build failures, nor autopkgtest failures.
Attachment:
signature.asc
Description: PGP signature