Re: Debhelper for R packages
(chopping down a little to shorten)
On 9 September 2016 at 10:50, Gordon Ball wrote:
| On 08/09/16 23:25, Dirk Eddelbuettel wrote:
| > Our Depends also need to cover R's "Imports:" which we will not see via ldd. I
| > presume you have that covered, I just thought I'd mention it.
|
| Yes, R:Depends includes both Imports and Depends.
|
| Package resolution is just by looking if `r-{cran,bioc}-lc(packagename)`
| exists, and it doesn't generate an error if dependencies cannot be
| found, just prints a warning in the build log.
|
| (It also will complain it can't find packages for internal namespaces,
| eg `stats`, but that should be fairly easy to add a check for).
Perfect. That's all that the various cran2deb implementations did. [ BTW if
you care I actually have an interface from R to our APT backend in package
RcppAPT on GitHub/CRAN. Doesn't help with Perl, of course :) ]
| The plan was to avoid seeing eg, lintian hardening-no-bindnow warnings
| on packages with compiled extensions. I tried injecting dpkg-buildflags
| LDFLAGS output into the MAKEFLAGS environment for R CMD INSTALL but it
| doesn't appear to work. It is possible this is also not particularly
| important (and/or might interfere with R's own build config) - I don't
| really see security as a major R issue.
It's an R issue. R "remembers" CC, CFLAGS, CXX, CXXFLAGS, etc pp from its
configure run and reuses them. When R runs 'R CMD INSTALL' you can add to
this on a per-package basis via the PKG_* variants from inside the package.
But we can't override from the outside. Minor annoyance (ie would be nice to
strip '-g' sometimes, toggle -O2 for -O3, ...). Needs an upstream change.
| > I had once tried to bring some perspective from the R ecosystem to that, but
| > as I recall I pretty much failed to get my point across. R already does /a
| > lot/ here via R CMD check, and it may be worthwhile to milk that angle rather
| > than to shoehorn R (which builds packages somewhat idiosyncratically) into
| > Debian's framework. Anyway, minor 'day 2' detail.
|
| I suspect our testing is most interesting where non-R dependencies are
| concerned, particularly where we remove bundled copies and force the use
| of system ones.
|
| As with vignette building below, it requires the set of build-deps to be
| much bigger (and particularly for bioconductor, tends to require big
| dataset packages available).
CRAN and BioConductor don't seem to know/care about chroot and minimal
environments. They tend to build on "fat" systems with everything installed.
But they also do it across three or more OSs which we don't so we can take
advantage of Linux tricks where they can't.
Some of the test philosophy is the same and hence redundant. R cares a lot
about build "stability" and "possibility" (ie no API changes), we care a lot
about changes in the toolchain (and so do they). Eg Brian Ripley in Oxford
tends to also check again as-of-yet unreleased (!!) gcc versions to catch
things early (but those checks are extra -- at the core where tests are
automated they are more sensible).
| My workflow is to run autopkgtest (in an schroot) as a postbuild step,
| which tests the built deb (so no workaround for testing a not-yet
| installed package is required, and ensuring the binary dependencies are
I fully agree that chroot tests are good, but our build does thosse ...
| correct), and autodep8 ensures that there is at least a `library(RLIB)`
| test defined for every package.
... because as I tried to explain last time, _each_ R CMD INSTALL involved
in _each_ of our builds includes a load of the just built library. From what
I understood then and understand now you are duplicating that. But I may
miss sufficient coffee... Anyway
| (stdout from R CMD INSTALL includes "checking if the package can be
| loaded" or something like that, but I'm not sure what that is actually
| testing in practice)
The very 'library(foo)' for a the package r-cran-foo you are building.
Which checks dynamic linker as well as R aspects (as the R code needs to
syntactially parseable etc pp). It is a good test. I just fell over it
yesterday on one or two small packages I am building right now.
| > | > It might also be interesting to explore:
| > | >
| > | > * handling binary dependencies between R compiled extensions
| > |
| > | What exactly do you mean here? I admit I'd welcome if this dh module
| > | would be available in sid soon to enable switching packages to it to get
| > | it fully tested soon. This kind of features might wait for later
| > | versions.
|
| See, eg, r-cran-rprotobuf (LinkingTo: Rcpp) or r-bioc-biostrings
| (LinkingTo: IRanges).
Being the author of Rcpp which is used by 750+ CRAN packages via LinkingTo
(whereas we recommended Depends in the day), I am aware of its limitation. It
is (pretty much) R's version of Build-Depends for the C++ case of header-only
libraries.
| My understanding is that this means that these packages build a binary
| extension using header files from another R package, and may link to the
| so file from the other package at runtime.
Not necessarily a so file.
It is for C++ code where you can get by with headers only. Most prominent
example is the BH package I maintain -- it has 120mb of Boost but __without
linking__ as that makes deployment so much easier across OS.
BH, RcppArmadillo, RcppEigen, ... all get used by _their_ client packages
just via headers. What they _would_ need in standalone use (ie if your built
a main()) they get from R: LAPACK, BLAS, ...
| I am wondering if this should be interpreted as a tight dependency on
| the LinkingTo: package, eg
|
| Built-Using: r-cran-rcpp (=version) or
| Depends: r-cran-rcpp (=version)
No.
| or if we need to trigger a transition when a package which can be the
| subject of LinkingTo: changes.
No.
| I'm speculating a bit here - I don't know how often this is a
| problem/how stable in practice package ABIs are. This might just be
| adding more work for a rare problem.
Yes.
Dirk
--
http://dirk.eddelbuettel.com | @eddelbuettel | edd@debian.org
Reply to: