Re: slib and Schemes
Rob Browning <rlb@defaultvalue.org> writes:
> Nick Gasson <nick@nickg.me.uk> writes:
>
>> I verified the latest release with scm, mit-scheme, and guile. Guile and
>> MIT need to have slib loaded once as root in order to generate the
>> catalogue file (there's a bug for this already). As I mentioned before I
>> think the solution is to generate the catalogue for each Scheme
>> implementation using a trigger in the slib package, but I haven't
>> started working on this yet. Do you think that's a good idea?
Nick: If slib is ready for Guile 3.0 in your opinion then please file a
wishlist bug on guile-3.0-libs asking for it to be enabled in the
postinst.
> Offhand, I'd imagine either a trigger or some other maintainer-script
> based approach to keep the file up to date would be appropriate.
>
> I don't know enough about the requirements to have any solid idea
> personally, but I'd guess triggers would be preferable if they'll suit,
> i.e. do they run often/soon enough with respect to either slib and/or
> the various scheme implementation upgrades.
Agreed. I think the way that it's done in guile-libs.postinst should
work.
> At least for guile, unless slib is also going to ship the compiled ".go"
> files, we might have some additional questions (that apply to all
> "add-on" packages).
That is of course a possibility. The slib source package could
build-depend on guile 3.0 and build a guile-slib package that ships the
compiled files. But there are too many Scheme implementations in the
world for that to be practical. Packages like slib would have to know
about all of them, or we'd need some serious infrastructure that
automatically maintains such packages. I don't know of any precedent in
Debian.
> Using guile as an example, ideally, we probably don't want to
> rely on guile's auto-compilation for our packaged .scm files since that
> will end up compiling each .scm file whenever it's first loaded (or the
> timestamp changes), per-user, to XDG_CACHE_HOME, i.e. ~/.cache/...
Guile's usual auto-compilation would give a bad user experience. SLIB is
an unusual fellow in the Guile ecosystem. From what I've seen, Guile
packages normally use autotools and guile.m4 to detect guile, get
themselves compiled and finally installed in the correct location. SLIB
has been around since Guile was forked from SCM, so I guess it has just
been grandfathered in, so to speak.
However, SLIB is not alone. Guile also supports R6RS and R7RS. There are
quite a few portable libraries available in those languages that don't
use autotools for installation (and certainly shouldn't have to).
> Macros raise another concern since guile has no automatic dependency
> tracking for them right now. Conceivably whenever a guile add-on
> package X changes (say slib), every other add-on package that depends on
> it may need to be completely recompiled, immediately, and in a way that
> avoids any auto-compilation.
This is indeed important. The library (x) can export a macro used in the
library (y), and this creates a compile-time dependency, meaning (y)
needs to be recompiled when (x) is upgraded. The same happens when a
file is included using any of the various include forms that are around.
> While I'd much prefer to avoid the complexity, those two considerations
> might mean that what guile really needs is something similar to the
> debian emacs infrastructure, which automatically byte-compiles add-ons
> at install time whenever emacs is updated, *and* whenever a dependency
> changes. No idea if other schemes have similar considerations.
Chez Scheme could also benefit from something like this. Its ABI is
AFAICT automatically generated and changes between every version, so
shipping pre-compiled libraries doesn't seem practical.
Any solution like this would have to be careful to take the target
machine into consideration. It will generally not be possible to use the
compiled code on another machine type, so the compiled files need to be
installed in a path that contains a target triplet or something similar,
plus a Scheme implementation name and version number. Chez has just
recently become open, so nothing like this has been developed for it in
the open. Guile has a head start.
> We might also be able to do something simpler if more expensive (maybe
> a lot more), than emacsen-common. For example, we might be able to just
> disable byte-compilation during package installs/upgrades, delete *all*
> the compiled files from any package's preinst, and rebuild the world in
> a final trigger.
> We could also just rely on auto-compilation, and accept the wasted
> space, longer initial startup-times (per user after an upgrade), and
> potential risks related to changes in macros, but that's perhaps not
> ideal.
For Chez Scheme it would be better to do nothing at all, its compiler is
very fast. Guile's compiler can take some time, so I guess it would be a
large benefit to at least do something. But this also speaks against
this strategy, I've seen *very* slow compilation times with Guile, on
some libraries. It could quickly make apt upgrade very painful. Perhaps
in this case it should be rebuilt in the background if that's feasible;
the user should not wait for it to be completed.
I'd like to see a way forward where we can package R6RS and R7RS
libraries, plus things like SLIB, and they "just work" in all supported
Schemes. My package manager Akku <https://akkuscm.org> can do automatic
analysis of imports and includes, which might help to create some sort
of dh-scheme helper. The data extracted by Akku could be used to
generate a list of dependencies during package builds. Another thing
that Akku helps with is to build all the symlinks required for mapping
library names to filenames. (Each implementation has its own mapping,
they should ideally all be shipped).
A hypothetical scheme-common package could also use Akku. To find the
libraries that need to be recompiled, I suspect this might work: analyze
each installed library, build a graph of its dependencies (i.e. imports
and includes), do a topological sort, and recompile every library where
it or one of its dependencies has changed. A scheme-common package could
do this analysis and then call the compiler for each installed Scheme.
It might also be good if each Scheme provides a file for scheme-common
that tells it how to invoke the compiler. The analysis in Akku works
well for R6RS and R7RS libraries, less well for implementation-specific
modules, and not at all for module-less code. It's possible that we
could do the dependency analysis during package builds and ship the
results in the package. If Akku's analysis doesn't work for a particular
case then it would be possible to create the dependency information
manually in that package.
It's also possible that I'm overthinking everything and we can just use
the Depends: field and then dig out the dependencies with dpkg. It would
be more coarse, and would recompile more libraries, but it would have
fewer moving parts.
--
Göran Weinholt | https://weinholt.se/
Debian Developer | 73 de SA6CJK
Reply to: