[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

packaging GCC plugins using gengtype (e.g. MELT)?

Hello All,

I am not a Debian Developper, and I am not very familiar with the internals of dpkg, or with GCC packages inside Debian. However, I am a GCC contributor [maintainer in GCC speak], and the proud author of MELT:

Sylvestre Ledru nicely proposed me to try to package MELT for Debian. I want to help him as much as I can (but I really want MELT to be packaged when gcc-4.5 is available in Debian, avoiding waiting for gcc-4.6!).

See also the http://gcc.gnu.org/ml/gcc/2010-03/msg00120.html message.

In a few words, MELT (a GPLv3+ licensed, FSF copyrighted software) is an "infrastructure" to code GCC extensions (like plugins) in a higher level, domain specific, lisp-like language (called also MELT) which provide useful features (functional programming, OO programming, pattern-matching, ...) when coding GCC extensions (i.e. specific passes in "plugins"). MELT generates C code (that is, it translates the MELT lispy dialect into C code suitable for GCC, and #include-ing most of GCC headers).

MELT should be compilable either as a self-contained GCC branch, or as a plugin to an *unmodified* GCC 4.5 installation. These are two separate ways of building MELT (either as an entire GCC branch -which is the way I daily build MELT- or as a melt.so plugin for gcc 4.5; the melt.so plugin is compiled from gcc/melt-runtime.c of the MELT branch. The building procedure are somehow documented on the GCC wiki, notably http://gcc.gnu.org/wiki/MELT%20tutorial and other pages). I suppose that MELT users would prefer a melt.so module to an unmodified gcc 4.5

MELT has several peculiarities which makes it perhaps difficult to package (but hopefully useful):

1. MELT dynamically generates C code, which is compiled into a shared object (that I call a MELT module, not a plugin, because the convention & API involved are not those of GCC plugins). which is itself dlopen-ed (and then running). So an execution of a MELT extended GCC compilation

  gcc -fplugin=melt.so -fplugin-arg-melt-init=runfile \
      -fplugin-arg-melt-arg=user-pass.melt \
      -O2 -c user-code-to-be-compiled.c

proceeds as follow:

a. GCC is started with the melt.so plugin and appropriate flags. the gcc driver forks cc1, which dlopen-s melt.so; then several MELT modules are dlopen-ed, notably warmelt*.so [which are the MELT translator] by code inside melt.so. Notice that MELT does use many GCC plugin hooks, http://gcc.gnu.org/onlinedocs/gccint/Plugins.html#Plugins and notably PLUGIN_GGC_MARKING & PLUGIN_REGISTER_GGC_ROOTS & many others. So building melt.so needs to run gengtype. Notice also that the GCC plugin documentation strongly warns that "plugins needing to use gengtype require a GCC build directory for the same version of GCC that they will be linked against".

b. the melt.so plugin & the MELT modules warmelt*.so read some [user-provided] user-pass.melt source file (in the MELT lispy dialect). The cc1 + melt.so + warmelt*.so process generates several C files [usually in some temporary directory], like user-pass.c & user-pass+01.c

c. The same cc1 + melt.so + warmelt*.so process forks a make -f melt-module.mk command to build the user-pass.so MELT module from user-pass.c & user-pass+01.c; this make command obviously requires gcc-plugins-dev package & needs the

d. that dynamically generated user-pass.so MELT module is itself dlopen-ed by the same cc1 + melt.so + warmelt*.so process, the start_module_melt routine is dlsym-ed by melt.so inside user-pass.so, it indirectly calls code to install a new GCC pass coded in MELT inside user-pass.so

e. The cc1 compilation proceeds as usual, but the newly installed GCC pass (coded in MELT in file user-pass.melt) do something useful when compiling the user-code-to-be-compiled.c

Of course, the five steps above are quite an extreme case of MELT usage. The user has the ability (thru -fplugin-arg-melt-init=translatefile) to generate his user-pass.c & user-pass+01.c files from user-pass.melt separately, and then build his user-pass.so MELT module, and have it loaded in subsequent GCC invocations, without requiring cc1+melt.so+warmelt*.so+user-pass.so to fork a make -f melt-module.mk process.

Notice also that MELT is bootstrapped: the MELT translator is coded in MELT, and the subversion repository contains also old gcc/melt/generated/warmelt*.c files generated from gcc/melt/warmelt*.melt. A MELT installation procedure usually build two stages of the MELT translator (independently of GCC stages).

Now, one of the issues about MELT & Debian packaging is the fact that melt-runtime.c (the source of melt.so plugin) uses GTY http://gcc.gnu.org/onlinedocs/gccint/Type-Information.html#Type-Information & register GGC roots thru PLUGIN_REGISTER_GGC_ROOTS ... Hence, it needs gengtype (from GCC 4.5 build tree) to generate gt-melt-runtime.h [#include-ed from melt-runtime.c] so the entire GCC 4.5 source & build trees are needed to build melt.so (or any other gengtyp-ing GCC plugin). I am not familiar enough with dpkg & debian packaging, but I believe this hurts quite a lot and could be difficult to implement. Improving gengtype to avoid that could be difficult, and definitely out of question for the gcc-4.5 time frame (and probably painful for gcc-4.6). An ugly solution might be to include that generated gt-melt-runtime.h file inside the source package of the MELT module, but then any big manual edits in melt-runtime.[ch] won't be propagated into gt-melt-runtime.h. However, I could easily svn add the generated gt-melt-runtime.h into GCC MELT svn tree (perhaps under contrib/) if that helps Debian to package it.

Comments are welcome (but beware that I Basile am not very familiar with Debian packaging, and even that I am still scared by the complexity of GCC building procedures, notably Makefile.in & configure.ac files).

Thanks for reading.


Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***

Reply to: