packaging GCC plugins using gengtype (e.g. MELT)?
Hello All,
I am not a Debian Developper, and I am not very familiar with the
internals of dpkg, or with GCC packages inside Debian. However, I am a
GCC contributor [maintainer in GCC speak], and the proud author of MELT:
http://gcc.gnu.org/wiki/MiddleEndLispTranslator
Sylvestre Ledru nicely proposed me to try to package MELT for Debian. I
want to help him as much as I can (but I really want MELT to be packaged
when gcc-4.5 is available in Debian, avoiding waiting for gcc-4.6!).
See also the http://gcc.gnu.org/ml/gcc/2010-03/msg00120.html message.
In a few words, MELT (a GPLv3+ licensed, FSF copyrighted software) is an
"infrastructure" to code GCC extensions (like plugins) in a higher
level, domain specific, lisp-like language (called also MELT) which
provide useful features (functional programming, OO programming,
pattern-matching, ...) when coding GCC extensions (i.e. specific passes
in "plugins"). MELT generates C code (that is, it translates the MELT
lispy dialect into C code suitable for GCC, and #include-ing most of GCC
headers).
MELT should be compilable either as a self-contained GCC branch, or as a
plugin to an *unmodified* GCC 4.5 installation. These are two separate
ways of building MELT (either as an entire GCC branch -which is the way
I daily build MELT- or as a melt.so plugin for gcc 4.5; the melt.so
plugin is compiled from gcc/melt-runtime.c of the MELT branch. The
building procedure are somehow documented on the GCC wiki, notably
http://gcc.gnu.org/wiki/MELT%20tutorial and other pages). I suppose that
MELT users would prefer a melt.so module to an unmodified gcc 4.5
MELT has several peculiarities which makes it perhaps difficult to
package (but hopefully useful):
1. MELT dynamically generates C code, which is compiled into a shared
object (that I call a MELT module, not a plugin, because the convention
& API involved are not those of GCC plugins). which is itself dlopen-ed
(and then running). So an execution of a MELT extended GCC compilation
gcc -fplugin=melt.so -fplugin-arg-melt-init=runfile \
-fplugin-arg-melt-arg=user-pass.melt \
-O2 -c user-code-to-be-compiled.c
proceeds as follow:
a. GCC is started with the melt.so plugin and appropriate flags. the
gcc driver forks cc1, which dlopen-s melt.so; then several MELT modules
are dlopen-ed, notably warmelt*.so [which are the MELT translator] by
code inside melt.so. Notice that MELT does use many GCC plugin hooks,
http://gcc.gnu.org/onlinedocs/gccint/Plugins.html#Plugins and notably
PLUGIN_GGC_MARKING & PLUGIN_REGISTER_GGC_ROOTS & many others. So
building melt.so needs to run gengtype. Notice also that the GCC plugin
documentation strongly warns that "plugins needing to use gengtype
require a GCC build directory for the same version of GCC that they will
be linked against".
b. the melt.so plugin & the MELT modules warmelt*.so read some
[user-provided] user-pass.melt source file (in the MELT lispy dialect).
The cc1 + melt.so + warmelt*.so process generates several C files
[usually in some temporary directory], like user-pass.c & user-pass+01.c
c. The same cc1 + melt.so + warmelt*.so process forks a make -f
melt-module.mk command to build the user-pass.so MELT module from
user-pass.c & user-pass+01.c; this make command obviously requires
gcc-plugins-dev package & needs the
d. that dynamically generated user-pass.so MELT module is itself
dlopen-ed by the same cc1 + melt.so + warmelt*.so process, the
start_module_melt routine is dlsym-ed by melt.so inside user-pass.so, it
indirectly calls code to install a new GCC pass coded in MELT inside
user-pass.so
e. The cc1 compilation proceeds as usual, but the newly installed
GCC pass (coded in MELT in file user-pass.melt) do something useful when
compiling the user-code-to-be-compiled.c
Of course, the five steps above are quite an extreme case of MELT usage.
The user has the ability (thru -fplugin-arg-melt-init=translatefile)
to generate his user-pass.c & user-pass+01.c files from user-pass.melt
separately, and then build his user-pass.so MELT module, and have it
loaded in subsequent GCC invocations, without requiring
cc1+melt.so+warmelt*.so+user-pass.so to fork a make -f melt-module.mk
process.
Notice also that MELT is bootstrapped: the MELT translator is coded in
MELT, and the subversion repository contains also old
gcc/melt/generated/warmelt*.c files generated from
gcc/melt/warmelt*.melt. A MELT installation procedure usually build two
stages of the MELT translator (independently of GCC stages).
Now, one of the issues about MELT & Debian packaging is the fact that
melt-runtime.c (the source of melt.so plugin) uses GTY
http://gcc.gnu.org/onlinedocs/gccint/Type-Information.html#Type-Information
& register GGC roots thru PLUGIN_REGISTER_GGC_ROOTS ... Hence, it needs
gengtype (from GCC 4.5 build tree) to generate gt-melt-runtime.h
[#include-ed from melt-runtime.c] so the entire GCC 4.5 source & build
trees are needed to build melt.so (or any other gengtyp-ing GCC plugin).
I am not familiar enough with dpkg & debian packaging, but I believe
this hurts quite a lot and could be difficult to implement. Improving
gengtype to avoid that could be difficult, and definitely out of
question for the gcc-4.5 time frame (and probably painful for gcc-4.6).
An ugly solution might be to include that generated gt-melt-runtime.h
file inside the source package of the MELT module, but then any big
manual edits in melt-runtime.[ch] won't be propagated into
gt-melt-runtime.h. However, I could easily svn add the generated
gt-melt-runtime.h into GCC MELT svn tree (perhaps under contrib/) if
that helps Debian to package it.
Comments are welcome (but beware that I Basile am not very familiar with
Debian packaging, and even that I am still scared by the complexity of
GCC building procedures, notably Makefile.in & configure.ac files).
Thanks for reading.
Regards.
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***
Reply to: