Re: R package CI test failures
Hi Gordon,
thanks for your investigation.
On Tue, May 03, 2016 at 04:05:22PM +0200, Gordon Ball wrote:
> Having had a better look at some of the less obvious failings:
>
> r-bioc-biocparallel
> -------------------
>
> Non-deterministic? Seems to fail about 80% of the time in
> test_bpaggregate.R when calling
>
> bpworkers(param) <- 2
>
> This is the only call to bpworkers() in the test suite, but I can't
> obviously see why it fails. I'm guessing it is some sort of
> parallel-related race condition.
>
> Unsure how to solve this except removing the affected test.
If we do not receive any hint from upstream removing this test seems to
be the only solution I can imagine (with my admittedly poor background
knowledge in this field).
> r-bioc-biomart
> --------------
>
> The test suite never succeeds, but fails at apparently random points
> with malformed XML errors. Since it relies on internet access, I'm
> guessing the connections don't work reliably from the autopkgtest
> testbed. It consistently works locally, so it isn't the remote resource
> failing.
>
> The tests may need removing (or at least, installing only for local use)
> since they aren't very useful as CI.
A suggested patch (commited to SVN + team upload) would be great.
> r-bioc-cummerbund
> -----------------
>
> Both vignettes are patched (suppress_test_writing_to_usr.patch) removing
> attempts to write to a readonly dir, but leaving subsequent code which
> consequently fails (data hasn't been loaded).
>
> Possibly changing the patch to copy extdata from the installed package
> and replace the `system.file` call with a local dir would let these run.
A suggested patch (commited to SVN + team upload) would be great.
> r-bioc-genefilter
> -----------------
>
> Two vignettes (independent_filtering{,_plots}.Rnw) require knitr
> (unpackaged), not Stangle for compilation.
knitr is ITPed (#808155) but needs some dependencies itself.
> The test script can probably be changed to ignore them.
I'm temped to leave this open to come back to the needed package
r-cran-knitr (which also rings a bell for other packages somehow). To
support this I committed to the packaging of r-cran-highr (#808808) as
well as r-cran-markdown[1]. I pinged Joost van Baal-Ili to get this
finished.
> r-bioc-genomicalignments
> ------------------------
>
> I think this is a bug with the test from upstream.
> test_readGAlignments.R :: test_read_GAlignments_BamViews creates a
> temporary file, doesn't hold onto the file handle and then tries to read
> it. Hence a race condition depending on whether R has yet
> garbage-collected the reference and the file has consequently been
> deleted first.
Hoping for upstream to respond to my first mail. I might ping at
end of the week with these more detailed information.
> r-bioc-limma
> ------------
>
> Compared stdout of the test script to a saved copy. It's probably
> sufficient to just run the script and test the exit code, since this
> seems to fail on trivial differences.
I might have a look. Needs to be updated anyway.
> r-bioc-rsamtools
> ----------------
>
> test_utilities.R::test_catch_samtools expects both a warning and an
> error, but only gets an error (and hence fails). Maybe a behaviour
> change in r base?
I have no idea. :-(
> r-bioc-shortread
> ----------------
>
> A bug somehow related to the package's class system, but I haven't
> managed to identify a cause.
I might ping upstream.
> r-bioc-summarizedexperiment
> ---------------------------
>
> Missing dependency on R database package hgu95av2.db (from
> test_makeSummarizedExperimentFromExpressionSet.R:176 - can probably just
> remove that subtest)
Fixed in recent upload.
> r-cran-afex
> -----------
>
> In run-unit-test, should run `gunzip -r *`, not `*.gz` (all the gz files
> are in a subdir). This hides a couple of errors resulting from trying to
> load the dataset mlmRev::Contraception (mlmRev isn't packaged). The
> individual tests are small enough they can probably be patched out for now.
Fixed in team upload.
> r-cran-contfrac
> ---------------
>
> Should be `gunzip -r *` like r-cran-afex
Fixed in team upload.
> r-cran-doparallel
> -----------------
>
> Uses R CMD BATCH instead of R --no-save < filename to run tests, so all
> the output is hidden in the CI output. Doesn't copy the test files to a
> temporary dir so when they try and create output it fails due to read-only.
(Hopefully) fixed in upload of new upstream version.
> r-cran-dplyr
> ------------
>
> Missing dependency r-cran-rsqlite. Hopefully the last one.
Hopefully. Seems I really need to setup debci here ...
> r-cran-r6
> ---------
>
> Fixed a couple of extra errors - gzipped tests not uncompressed, added a
> patch to remove a one-line use of pryr (unpackaged) in the testsuite.
> Should be ready to upload 2.1.2-3 from git.
Uploaded after bumping Standards-Version.
> r-cran-rocr
> -----------
>
> Cleanup should be rm -rf, not just rm -f at the end of run-unit-test
Fixed in recent upload.
> Errors I couldn't identify
> --------------------------
>
> Some of these have appeared recently and might be related to R 3.3.0
>
> r-cran-bbmisc
> r-cran-futile.logger
> r-cran-msm
Thanks a lot for the cleanup. I hope I was able to polish the tests
even further. I guess after another round of checks we might be able to
contact according upstreams.
Kind regards
Andreas.
[1] https://anonscm.debian.org/git/debian-science/packages/r-cran-markdown.git
--
http://fam-tille.de
Reply to: