Use files created during CI to seed trust and attract new users?
Hello,
When CI fails, I get an email and can chase things up with a look at the
output files. This is something I like a lot. As a user, though,
especially for scientific packages I would want to see the files that
have been created. This way I could confirm that, e.g., the generated
PDFs have usable fonts and if a test follows an established workflow for
a well-known dataset, I can also compare the results generated during
testing with the ones expected. "Expected" could be the files a Mac user
generated when installing the package locally, a less technical user may
know a tutorial (a nice one that I want to address is on
https://docs.qiime2.org/2021.4/tutorials/moving-pictures/) and would
then compare the CI-generated image with the one presented online by
upstream.
There are many problems with this suggestion, to mind come
* amount of data to be handled as input
* amount of data to be stored
* unclear acceptance by users
* too much extra compute
* ... please add what comes to your mind ...
My sketch of an idea was that we could possibly have in analogy to the
debian/install files an extension to the test instructions that define a
set of files, preferably together with a description, that may be
worthwhile to be inspected by end users as a proof that the package
works. A test report would then be generated with links to these files.
Is this anything we would want to have? Do we have this already and I
just failed to find this?
Many thanks!
Steffen
Reply to: