Re: R package CI test failures

[Gordon Ball <gordon@chronitis.net>]

Having had a better look at some of the less obvious failings:

r-bioc-biocparallel
-------------------

Non-deterministic? Seems to fail about 80% of the time in
test_bpaggregate.R when calling

    bpworkers(param) <- 2

This is the only call to bpworkers() in the test suite, but I can't
obviously see why it fails. I'm guessing it is some sort of
parallel-related race condition.

Unsure how to solve this except removing the affected test.


r-bioc-biomart
--------------

The test suite never succeeds, but fails at apparently random points
with malformed XML errors. Since it relies on internet access, I'm
guessing the connections don't work reliably from the autopkgtest
testbed. It consistently works locally, so it isn't the remote resource
failing.

The tests may need removing (or at least, installing only for local use)
since they aren't very useful as CI.


r-bioc-cummerbund
-----------------

Both vignettes are patched (suppress_test_writing_to_usr.patch) removing
attempts to write to a readonly dir, but leaving subsequent code which
consequently fails (data hasn't been loaded).

Possibly changing the patch to copy extdata from the installed package
and replace the `system.file` call with a local dir would let these run.


r-bioc-genefilter
-----------------

Two vignettes (independent_filtering{,_plots}.Rnw) require knitr
(unpackaged), not Stangle for compilation.

The test script can probably be changed to ignore them.


r-bioc-genomicalignments
------------------------

I think this is a bug with the test from upstream.
test_readGAlignments.R :: test_read_GAlignments_BamViews creates a
temporary file, doesn't hold onto the file handle and then tries to read
it. Hence a race condition depending on whether R has yet
garbage-collected the reference and the file has consequently been
deleted first.


r-bioc-limma
------------

Compared stdout of the test script to a saved copy. It's probably
sufficient to just run the script and test the exit code, since this
seems to fail on trivial differences.


r-bioc-rsamtools
----------------

test_utilities.R::test_catch_samtools expects both a warning and an
error, but only gets an error (and hence fails). Maybe a behaviour
change in r base?


r-bioc-shortread
----------------

A bug somehow related to the package's class system, but I haven't
managed to identify a cause.


r-bioc-summarizedexperiment
---------------------------

Missing dependency on R database package hgu95av2.db (from
test_makeSummarizedExperimentFromExpressionSet.R:176 - can probably just
remove that subtest)


r-cran-afex
-----------

In run-unit-test, should run `gunzip -r *`, not `*.gz` (all the gz files
are in a subdir). This hides a couple of errors resulting from trying to
load the dataset mlmRev::Contraception (mlmRev isn't packaged). The
individual tests are small enough they can probably be patched out for now.


r-cran-contfrac
---------------

Should be `gunzip -r *` like r-cran-afex


r-cran-doparallel
-----------------

Uses R CMD BATCH instead of R --no-save < filename to run tests, so all
the output is hidden in the CI output. Doesn't copy the test files to a
temporary dir so when they try and create output it fails due to read-only.

r-cran-dplyr
------------

Missing dependency r-cran-rsqlite. Hopefully the last one.


r-cran-r6
---------

Fixed a couple of extra errors - gzipped tests not uncompressed, added a
patch to remove a one-line use of pryr (unpackaged) in the testsuite.
Should be ready to upload 2.1.2-3 from git.

r-cran-rocr
-----------

Cleanup should be rm -rf, not just rm -f at the end of run-unit-test


Errors I couldn't identify
--------------------------

Some of these have appeared recently and might be related to R 3.3.0

r-cran-bbmisc
r-cran-futile.logger
r-cran-msm



Gordon

On 29/04/16 14:42, Andreas Tille wrote:
> Hi Gordon,
> 
> On Fri, Apr 29, 2016 at 01:29:50PM +0200, Gordon Ball wrote:
>>> I confirm that if you use the script for CI *only* this would work
>>> nicely.  However, I consider it a good idea to provide the user with
>>> some test case / example use and thus I always put the script into
>>> /usr/share/doc/<pkgname> where grep-dctrl does not work.  On the other
>>> hand I could check the directory name if all other means to calculate
>>> the package name might fail.
>>
>> I hadn't considered other uses. However, there is maybe the possibility
>> of instead including a generic template and inserting the correct values
>> (determined as above) at build time?
> 
> In principle yes.
>  
>> However, since this would require modifying all the d/rules files, it is
>> maybe too much engineering to squash a few copy-paste bugs which are
>> mostly now fixed - to be added to the wishlist for debhelper-R?
> 
> Could be nice feature but I'd regard this low priority as you said since
> most cut-n-pastos are eliminated.
> 
>>> Yes.
>>
>> Of the original list, I cound 24 (8 bioconductor and 16 cran) packages
>> now passing (on amd64 - most are still marked failed on arm64 but I
>> think that is just that the tests haven't re-run yet). Thanks to those
>> that committed fixes and uploaded.
> 
> I'm observing Debian Med team QA page[1] regarding these issues daily
> since the tests seem to run daily and I squashed some more this morning
> which either were not fully fixed or for whatever reason they were
> missing on the list or I was not reading the list thoroughly.  For the
> BioConductor problems I'd regard real problems I've contacted upstream
> (and I think I CCed you).
>  
>>>> I'll have a better look at the "?" ones.
>>>
>>
>> Etiquette question - for packages in debian-science git requiring
>> trivial changes (eg, a missing package in debian/tests/control), is the
>> preferred course to just go ahead and commit the fix?
> 
> As far as I understand commiting the fix to VCS is really welcome - even
> doing a team upload is fine.  For those packages I'm Uploader I'd be
> really happy about any upload that fixes a problem. :-)
> 
> Kind regards
> 
>       Andreas.
> 
> [1] https://qa.debian.org/developer.php?email=debian-med-packaging%40lists.alioth.debian.org 
>