Re: Team maintain Guile/Guix-related packages under debian-scheme?
Simon Josefsson <simon@josefsson.org> writes:
> Vagrant Cascadian <vagrant@debian.org> writes:
>> I have also had the impression that basically any time any dependency of
>> a guile package changes, it should probably be binNMUed to regenerate
>> the .go files ... especially when it there is an involved C binding
>> ... which I have sometimes done with sourceful uploads, but definitely
>> could not always keep up with.
>
> I don't fully understand the requirements here, but I also think the
> current situation seems problematic. Expressing the dependency using
> Static-Built-Using would be one step, but I'm not sure if guile itself
> should be in there? One comparison is with binaries built from C, they
> don't have 'Static-Built-Using: gcc' in them. Although I think in
> theory they should -- if gcc generates insecure code, you would want to
> rebuild the binaries built using that compiler. Maybe
> 'Static-Built-Using' is a Debian hack to re-invent the Guix
> bootstrappability graph, but in an incomplete way.
>
> So until we have developed some understanding that we solve any real
> problem, maybe we should hold off adding Static-Built-Using to guile
> packages.
Static-built-using is new to me, so I can't comment on that yet (though
I have guesses). Where is it documented?
But I can comment on guile itself a bit. Format-wise, guile's .go files
are expected to be "backward compatible" across guile X.Y releases,
e.g. if only the Z changes, then they should work with a newer guile.
The .go files do not contain any architecture specific machine code, and
there are only 4 "flavors" covering 32/64-bit and big/little-endian.
(For the guile package itself, I have vaguely wondered about the
possiblity of compiling them once on a fast architecture for the other
architectures; amd64 takes less than 30m where m68k takes 2+ days.)
So just considering that, you don't have to rebuild .go files for every
upgrade, but you might want to anyway if, say, the newer guile compiler
notably improved its optimizations.
Guile itself also intends to automatically make sure you're always
running the latest version of the code, and so if it finds a .scm that's
newer (stat mtime-wise) than a corresponding .go, it will recompile the
.scm and store the resulting .go *per-user* in ~/.guile/cache/... It
will notice this "skew" and recompile during the loading process,
i.e. when the dependency is reached via (use-modules (some dep) ...)
either directly or indirectly via some module-resolution dependency
chain.
But, as discussed here, and in another recent thread, there is *no*
dependency tracking of macros, and so as with many, I think, other lisps
(including elisp I believe), or any language with similar facilities[1]
and no explicit tracking, you just have to be careful with your macros
with respect to dependent, compiled code, i.e. depending on how/where
you use them, they can become part of your API/ABI with respect to
versioning --- even indirectly if you're using a macro from somewhere
else that affects your API (say one of the returned structures) and then
that macro changes in a way that changes the structure incompatibly.
So you can easily alter macros in a way that completely breaks things
(directly or indirectly), and you'll never know it until the relevant
clients are recompiled.
Given that, unless you're *sure* every upstream understands this and is
going to be sufficiently careful, the safest thing to do is to just
re-build all the potentially affected packages' .go files at package
install/upgrade time in "package dependency order". And that means
rebuilding *all* of them in dependency order whenever guile itself
changes.
This is in fact, exactly what the debian-emacs-policy intends for
emacsen, and is notably difficult, given the semantics actually allowed
by debian's maintscripts[2]. The current emacsen-common infrastructure
for this is actually not quite right, and I'm hoping to get back to
overhauling it soonish.
[1] e.g. "public" C preprocessor macros, C++ templates, etc. Though I
haven't followed C++ carefully in a good while --- maybe it tracks
them now, somehow; otherwise you just have to consider changes
carefully with respect to your versioning (SONAME).
[2] For example, the fact that the postinst is the *only* place you can
have high confidence that any package dependencies are actually in a
usable state (and iirc perhaps only for postinst configure and
triggered). It's complicated, and I had no idea how complicated
when we originally set up the current infrastructure.
For reference, policy describes the (lack of) guarantees in the
maintscripts, and this shows the full range of potential "cases" to
consider: https://people.debian.org/~srivasta/MaintainerScripts
As one more concrete case, imagine you're in the guile-3.0 "postinst
configure" and want to rebuild all the guile-related packages in
dependency order. Let's assume you have some way to generate the
right subset of packages; then how do you produce the correct
ordering? (For a long time, emacsen-common has generated it
hackishly via dpkg listing, tsort, etc., but now I think I *might*
have a way to let dpkg handle it via a (possibly too clever) way of
using triggers.) This alo relies on a debian-emacs-polcy
requirement that emacs "add-on" packages expose any relevant
dependencies in their dpkg dependencies, i.e. elpa-magit depends
(and must depend) on elpa-transient so that we can extract that
requirement via dpkg from the relevant postinst invocations (say
for emacs itself, during an emacs upgrade).
Hope this helps
--
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4
Reply to: