Bug#930487: lintian: speed up test suite CI
Hi Dmitry,
On Sat, Jul 6, 2019 at 11:27 PM Dmitry Bogatov <KAction@debian.org> wrote:
>
> > I think the most common use-case is that I make some change — possibly
> > extremely minor — in, say, checks/foo.pm and I want to re-run the
> > testsuite to check that I've fixed whatever false-positive or edge-case
> > in a tag that I am in the process of implementing. We should optimise
> > for kind of pattern. What do you think?
In that scenario, I usually restrict the run with
"--onlyrun=check:manpages" (after editing checks/manpages.pm). You can
also use the selectors 'test:' or 'tag:'.
> Ideally, I want test suite automatically detect:
>
> * test directory edited, test.deb needs to be rebuilt
> + I do not want re-run perl-critic and perltidy checks aganist files,
> that were not changed
I have not found a way to implement that. An incremental builder I
wrote based on file modification times (in
lib/Test/StagedFileProducer.pm) cannot tell the difference between two
files that were built within one second of another. That is the
granularity of Linux filesystems. The templating mechanism relies on
many intermediate build products. There should be a way to rebuild a
test case if any of the inputs have changed, but I haven't found it.
Patches are welcome.
> In short, I want test suite to be dependable build system from source
> files and test files into tarball of test logs. (~950 files or so)
I think it's more or less dependable. Our issue is speed.
You can find logs for each test case in ./debian/test-out/.../log.
> Right now it feels like build system written in Make -- undercontrained
> in one cases (you can easily get stalled results), overconstrained in
> another cases (you build more then necessary -- perlcritic tests, for
> example).
I think the test suite builds exactly what is necessary. It just takes
a long time. :)
Perlcritic (or more often perltidy) is something we impose on
ourselves. There are maddening moments, particularly when the software
corrects itself, but on balance I am in favor.
> To get feeling what is perfect (in sense of dependencies correctness)
> build system, take a look at `tup'[1]. Read the tutorial, it is worthy
> reading.
How does 'tup' deal with the timestamp granularity? Is that solved
with a 'directed acyclic graph (DAG)'?
> That is why I talk about Shake -- it is more complicated, compared to
> tup, but it supports dynamic (monadic) rules, so previous example with
> `foo.tar' is basic example of Shake.
Perl is a generalized programming language (and Debian runs on it).
Everyone can contribute. I think the best path forward is to try
caching, as Chris suggested. If that does not work, we should consider
building the test packages separately.
Thank you for your interest in the Lintian test suite. The long build
times bother many people. I would very much like to reduce them.
Reply to: