Re: debian packaging of findlib
On Sun, Feb 18, 2001 at 01:27:34PM +0100, Gerd Stolpmann wrote:
> On Sun, 18 Feb 2001, Sven LUTHER wrote:
> >Hello, ...
> >
> >First let me present myself, ...
> >
> >I am the debian maintainer of the ocaml package, and some others, and as such,
> >i have to look into stuff like that, ...
> >
> >Also i am the debian sponsor to become maintainer of stefano, and as such i
> >did suggest the curse to take with the findlib package to be debian compliant,
> >as well as it fitting my own vision of the way ocaml should be packaged in
> >debian.
>
> I take this ironical. Of course, there are several ways to package ocaml,
> basing on different conventions, requirements and taste. For myself, I have
> given up to package ocaml binaries, because it's too much work for the two
> or three systems (with different operating systems) I have to maintain. So my
> view is often a "source code view": How do I make it simple to install a
> package from source, or to upgrade an existing package?
Well, yes, but you are not using a linux distribution, with packages that have
to conform to the same installation guidelines.
Then, maybe you use some distrib (maybe rpm based) that don't give you the
possibility to easily do your own packages that are integrated with the main
distribution.
That said, i understand you, but we are in different world, you want to use
ocaml programs in any environment you use, while what we are trying to achive,
ios that the user, who don't want to worry about things like you, can use the
system, and have the guarantee from the packager that everything is in its
right place, and will cooperate and speak with each other ocmponent without
problem. Fot that we have to conform to some standards.
> >Ok, that said, i apologize if maybe you would find that things like what will
> >be listed below may seem a bit exigent to you, but i want you to know that it
> >is what i feel is the best way of handling this on the ocaml side, other
> >people may feel a bit different about this, and you may have your own
> >opinions, so i await feedback on this issue, but anyway, i think maybe some
> >part of this is already possible or easily doable, and some other part of this
> >will be benefitable to you even outside of the debian cadre (as well, debain
> >is following mostly the FS conventions standard on linux system these days).
>
> Looks like a self-made curse... But I must admit that I like it if I find a
> well-packaged development environment on a system.
Well, it is the price to pay for having things well integrated, especially in
a distributed devloped system like debian.
>
> >Ok, now on with the proposals, ...
> >
> >The problem with ocaml packagging and findlib is as follows :
> >
> > * 1st, for installing, it would not do to use findlib, since the packages are
> > built on another system than the one they are used on, and are not
> > installed as root, and in another directory than the one they would finally
> > be in.
>
> Most packages are relocatable (no paths compiled into the archives/executables).
> The notable exception is findlib itself.
Err, .
the probmlem is that ome packages do things like that :
install:
$(OCAMLFIND) install xxx yyy zzz
or something such.
which cannot be done when findlib is not installed ...
> > * Once the package (precompiled) is finnaly installed, the package are just
> > copied to the final location by the package installer, and it is possible
> > to launch a post-install/pre-remove script to do stuff.
> > * well not everyone will have the findlib package installed, so we need
> > somethign that will use findlib if it is installed, and work also if it is
> > not installed. (in particular, i think much of the packages you did uses
> > ocamlfind exclusively for installation, maybe making ocamlfind command line
> > compatible to install would be nice, or if not providing another install
> > target that uses install, in the former case, you could set the
> > $(OCAMLFIND) to "", and then just install ... get executed)
>
> First some general notes about findlib:
>
> Some important packages use findlib to resolve dependencies. For example:
>
> - PXP needs netstring and optionally wlexing
> - netstring needs str
>
> So when I compile a program using PXP, I need not know the direct or indirect
> dependencies. Imagine what happens if I change netstring someday such
> that it bases on pcre and not str for the regular expressions (or that I add
> other prerequisites): Without findlib you have to change the Makefile of the
> program applying PXP, with findlib everything works out of the box. The task of
> findlib is information hiding on package level.
>
Yes, i understand that, from your perspective, but from the dbeian
perspective, this is handled by the packages system, which provide build
dependencies and normal dependencies, so we really don't need findlib in the
nomral usage.
The only real usage is for people doing developpment in ocaml, using not
packaged packages, or own-developped packages, and want to use the facilities
of findlib.
So you see this is not really our preocupation.
> Another example is my RPC package which needs equeue. The current development
> version of netclient needs equeue, too. Of course, the smaller packages do not
> have prerequisites.
Sure, we have all this in the packaging system, even apt-get source -b will
downlaod the needed version of packages that are needed to build the one you
are building. Since this is done on a distribution level, it will cover more
things than what you cover with findlib.
> In the recent packages, I added a "conventional-install" goal to the Makefiles
> which simply installs into an arbitrary directory. But for the bigger packages,
Yes, that is what is needed for debian, we need a override variable (called
DESTDIR) to install in the non-final place.
> you still need ocamlfind for compilation (unless you modify the Makefile;
> perhaps I should include some comments how to do that).
Yes, that is ok, you will then use findlib for the packages you write that use
it for compilation, but the idea is to support findlib for every ocaml debian
package.
> Simply using packages for own projects is simple even without findlib; but you
> have to manually set the directories for ocamlc, and have to manually resolve
> dependencies.
huh ???
> Back to your question. For most packages you do not need a postinstall script
> because they are simply relocatable: move the directory to the location you
> want. The user of the package is free to manually set directories etc for his
> own project he wants to compile, or to call on findlib's help.
err, the post install script will register the ocaml modules and libraries
with the findlib meta package, even without findlib being installed, so that
when later you install findlib to use it for your own developpment, all the
debian modules will be available for it, not only those witten by you and the
few other that support findlib. (in particular, there is no findlib support in
the inria developped libraries, isn't it ?)
> Packages which are not relocatable are problematic. From my packages, only
> findlib itself is not relocatable. I think this problem becomes smaller if the
> compiled findlib only contains the absolute location of a configuration file
> containing the rest. Such a configuration file can be easily updated from a
> postinstall script, if that becomes necessary.
YEs, that is what would be really usefull, and this should be configureable
easily in the Makefile.
> > * what else ? well i guess this is all of it.
> >
> >SO, without having looked much at findlib, i can say that what would be nice
> >would be :
> >
> > - a shell script that does the meta-package information updating when adding
> > or removing a package. This will be launched in the install script of
> > compliant packages, and later on, if you install findlib, it will just
> > work. This meta package could be put in /var/lib/ocaml or even in /etc,
> > following what debian policy has to say about it. Stefano, could you look
> > into it ?
>
> It's even simpler. If the package contains a configuration file called META it
> is already findlib-compliant. For example, here the META file of equeue:
>
> version = "1.1.4"
> description = "Queues, especially for events on file descriptors"
> requires = "unix"
> requires(mt) = "unix,threads"
> archive(byte) = "equeue.cma"
> archive(byte,debug) = "equeue_g.cma"
> archive(byte,mt) = "equeue_mt.cma"
> archive(native) = "equeue.cmxa"
> archive(native,mt) = "equeue_mt.cmxa"
>
> # TCL extension:
> archive(byte,use_tcl_event_loop) = "equeue_tcl.cma"
> archive(native,use_tcl_event_loop) = "equeue_tcl.cmxa"
> linkopts(byte,use_tcl_event_loop) = "-cclib -ltclqueue"
> linkopts(native,use_tcl_event_loop) = "-cclib -ltclqueue"
>
> It contains the compiler options, the requirements, and some information for
> humans. If I invoke the compiler with
>
> ocamlfind ocamlc -package equeue ...
>
> findlib looks into the configured site-lib directory, and checks if there is a
> subdirectory equeue containing the META file. If so, findlib accepts that
> package and loads the data from META file.
But we cannot ask every pacvkage maintainer to provide this file, they don't
know about it, it is heavy syntax and error prone (well when you add typos in
it).
>
> So it is not necessary to update a meta-information db. Just add a META file
> for every package, and it will be found by findlib. Without findlib, the extra
> file can be ignored.
Ok, but is it not possible to generate this meta information directly with a
simple script being invoked at install time ...
> > - a central config file, where you could list one or more places where to
> > search for libraries, and maybe other behavior options ?
>
> I think this is the way to go. At the moment, two options make sense:
>
> - The search path for libraries
> - The directory into which ocamlfind installs by default
>
> >Also from the packages i know (mostly 'official' ones like camltk, labltk, and
> >lablgtk) They install directly into $(CAMLLIB)/package-name. Is this a lot
> >incompatible with your tool ? Maybe not, because if you register all needed
> >stuff into the above mentioned config/database, in /var/lib/ocaml or something
> >such, you don't need a specific directory, like ocaml-site, to put things in.
>
> A specific directory is not necessary. If you add the CAMLLLIB directory to
> findlib's search path, and put a reasonable META file into
> $(CAMLLIB)/package-name, the requirements of findlib are already fulfilled.
>
> However, I do not like this. CAMLLIB is already a quite large directory, and
> this makes it difficult to survey. A separate directory containing all add-on
> packages is simpler to handle, and easier to understand. I can do a
> ls /directory and it lists all installed packages.
what about a finlibtool -list or something such ?
> Unfortunately, the caml team has decided to continue the $(CAMLLIB)/package-name
> pattern. In the upcoming 3.01 release the compiler understands
> -I +package-name and internally translates this option to
> -I (CAMLLIB)/package-name. What do you think about this?
It is really a big plus in developpment when you don't do advanced stuff like
you do.
i would have advocated for a direct search path from $(CMALLIB) without the
need for the +package-name stuff.
That said, mlgtk for example installed in $(CAMLLIB) directly.
> (3.01 is not yet out. There is still the chance that they add a search path for
> packages.)
a search patch, something more than the -I stuff ...
> Findlib has a redirection feature which is currently used for the official
> packages, and for the core packages (from findlib's point of view, the
> libraries str, unix etc. are also packages needing meta-information). For
> example, for camltk I have in site-lib/camltk a META file (and nothing else)
> which redirects the location of the package:
why not put all meta files into /var/lib/ocaml/xxx.meta or somethign such.
> directory = "^camltk41"
> archive(byte) = "tk41.cma"
> ...
>
> Here, ^camltk41 means the same as +camltk41 for the compiler, so tk41.cma is
> looked up in $(CAMLLIB)/camltk41, the official location.
>
> For the future, I expect that some of the features of findlib will also be
> implemented by the compiler. Xavier Leroy already took over the idea of storing
> the linker options together with the installation, and in 3.01 we will see some
> notion of packages inside the compiler. However, the plans of the caml team are
> unclear to me; for example what they think in general about package management.
> In the past they followed a "keep it simple" approach (with success), but the
> caml users become more and more ambitious over time. Perhaps we'll get
> dependencies for archives, and I can drop some of the features of findlib then.
>
> >The /usr/local/lib/ocaml or something such place is only for installing
> >ocaml libs that are not yet packaged by debian, and as thus can be done as you
> >have been doing all the time. Maybe they would take priority over installed
> >packages, don't know, this would be nice to have in the config file.
>
> This is simple, and if I have time, I'll implement a conflict report, too.
Sorry,, i may have been a bit unclear here, and i have to go now, we need to
talk more of it later.
Friendl,y
Sven Lutherr
Reply to: