Re: debian packaging of findlib
On Sun, Feb 18, 2001 at 06:58:52PM +0100, Gerd Stolpmann wrote:
> On Sun, 18 Feb 2001, Sven LUTHER wrote:
> >Well, yes, but you are not using a linux distribution, with packages that have
> >to conform to the same installation guidelines.
> >
> >Then, maybe you use some distrib (maybe rpm based) that don't give you the
> >possibility to easily do your own packages that are integrated with the main
> >distribution.
> >
> >That said, i understand you, but we are in different world, you want to use
> >ocaml programs in any environment you use, while what we are trying to achive,
> >ios that the user, who don't want to worry about things like you, can use the
> >system, and have the guarantee from the packager that everything is in its
> >right place, and will cooperate and speak with each other ocmponent without
> >problem. Fot that we have to conform to some standards.
>
> Yes, you are right, these are not my standards. I have a developer view, and
> one of the goals of findlib is to minimize the time I spend with writing
> configure scripts or intelligent Makefiles or detailed build documentation. I
> want to make it better than in the C world, and so I have written a generic
> tool that simplifies these tasks.
>
> But also plain users benefit from findlib. Imagine somebody downloads a program
> from somewhere, finds that it is written in ocaml, and installs the debian ocaml
> packages to be able to compile it. Our downloader is not a developer, and ocaml
Err, no it don't work like that, he just see's : how there is already a debian
package from this great new stuff, let's install it, he types apt-get install
package-name, and automatically the pre-built package is donwloaded from one
of the officials debian mirrors, where only dbeian developper have right to
upload packages, together with all packages that are needed for it, and it get
installed. No need of findlib or any other such stuff. He don't even need to
know it is an ocaml package.
In this world view, findlib will be usefull only for ocaml developper, the
rest of the world wouldn't care less about it.
> is a new word for him. If the program has a findlib-complying Makefile, he
> needs only call "make" and the program compiles. No configure scripts with
> dozens of parameters, no variables to set, no environment to understand.
See above, we are speaking binary distribution here, not source, altough you
can ever get the source, most people don't care though.
> Standards are a great thing, but they must match what users want. I fully agree
> with you that a package system should guarantee the integration of the
> components.
Anyway, we are debian maintainers, and must comply with the debian standard in
order to have the packages (pre-built) accepted into debian. The dependency
handling stuff is enough to handle any existing dependency, even for the
auto-builder which build the m68k, ppc, sparc or other such packages.
> >> >Ok, now on with the proposals, ...
> >> >
> >> >The problem with ocaml packagging and findlib is as follows :
> >> >
> >> > * 1st, for installing, it would not do to use findlib, since the packages are
> >> > built on another system than the one they are used on, and are not
> >> > installed as root, and in another directory than the one they would finally
> >> > be in.
> >>
> >> Most packages are relocatable (no paths compiled into the archives/executables).
> >> The notable exception is findlib itself.
> >
> >Err, .
> >
> >the probmlem is that ome packages do things like that :
> >
> >install:
> > $(OCAMLFIND) install xxx yyy zzz
> >
> >or something such.
> >
> >which cannot be done when findlib is not installed ...
>
> Maybe there is a misunderstanding; I don't know much about debian packages. Do
> they install themselves from source? In this case this installation method
no, they are binary packages, and as thus, the build process is done by the
maintainer, and the auto-builder for the other supported architecture.
> requires that findlib is already installed. For a purely binary package it
> is not required.
Well, it would be nice if that would not be a requirement. I guess it is
possible (as you said, we only need to provide the meta package, or a quick
way to generate one).
> >> First some general notes about findlib:
> >>
> >> Some important packages use findlib to resolve dependencies. For example:
> >>
> >> - PXP needs netstring and optionally wlexing
> >> - netstring needs str
> >>
> >> So when I compile a program using PXP, I need not know the direct or indirect
> >> dependencies. Imagine what happens if I change netstring someday such
> >> that it bases on pcre and not str for the regular expressions (or that I add
> >> other prerequisites): Without findlib you have to change the Makefile of the
> >> program applying PXP, with findlib everything works out of the box. The task of
> >> findlib is information hiding on package level.
> >>
> >
> >Yes, i understand that, from your perspective, but from the dbeian
> >perspective, this is handled by the packages system, which provide build
> >dependencies and normal dependencies, so we really don't need findlib in the
> >nomral usage.
> >
> >The only real usage is for people doing developpment in ocaml, using not
> >packaged packages, or own-developped packages, and want to use the facilities
> >of findlib.
>
> I don't believe that the debian package system is smart enough to handle _this
> type_ of dependencies. A dependency is not only a precondition that can be
> checked whether it is true or not. For example, if I compile a program using PXP
> with
>
> ocamlfind ocamlc -o sample -package pxp -linkpkg sample.ml
>
> the dependency analysis yields that a number of libraries must be linked into
> the executable, resulting in an effective command such as
>
> ocamlc -o sample -I /xxx/pxp -I /xxx/netstring -I /xxx/str
> str.cma netstring.cma pxp.cma sample.ml
>
> which is extracted from the meta-information data included in the packages.
No, each package maintainer knows which package are needed to build the
package, and lists them (with version requirement, virtual packages and other
such) in the build dependency field of the control file of the package. Later
the auto-builder only have to look at that, and download theneeded packages
before building. If some dependency is missing, the maintainer get's a bug
report about it.
What we want do to find findlib, is that, for packages where only ocamlfind is
available for install, they will build-depend on it, but on other packages,
well upto now, it is not possible to use findlib with them. So it would be
nice to have a small script that will generate the META information at package
install time, so that a user relying on findlib for developpment can make good
use of them. We don't want ot force every ocaml developper to use findlib.
> This is not only useful for developers! Also casual users who only want to
> compile programs by third parties benefit from this analysis because they do
> not need to configure anything, at least regarding the development environment.
Like said, the debian maintainer build programs, the rest of the users only
install binaries, and have no need of findlib.
> >Sure, we have all this in the packaging system, even apt-get source -b will
> >downlaod the needed version of packages that are needed to build the one you
> >are building. Since this is done on a distribution level, it will cover more
> >things than what you cover with findlib.
>
> This is a very debian-centric view. Maybe I do not understand your point: Is
> the solely task of a debian package to build another debian package? I only
> need the compiler and the libraries to _build_ (at some point) applications. And
> it is very likely that the app is not (yet) debianized.
No, it is to provide binary packages to the user, so they can only install
what they need, without ever having to worry about things they don't want to
worry about.
> >> Back to your question. For most packages you do not need a postinstall script
> >> because they are simply relocatable: move the directory to the location you
> >> want. The user of the package is free to manually set directories etc for his
> >> own project he wants to compile, or to call on findlib's help.
> >
> >err, the post install script will register the ocaml modules and libraries
> >with the findlib meta package, even without findlib being installed, so that
> >when later you install findlib to use it for your own developpment, all the
> >debian modules will be available for it, not only those witten by you and the
> >few other that support findlib. (in particular, there is no findlib support in
> >the inria developped libraries, isn't it ?)
>
> Well, you can implement such a registry by having a special place for all the
> META files, and if findlib is being installed, these files are moved to their
> final place. You can do it this way, but it sounds like an overkill to me.
Why not use the special place directly ?
> INRIA packages do not have findlib support as far as I know. Perhaps they have
> more time than I to handle installations in a conventional manner.
But then, i am notsure it is really needed, but then i may be spoiled by the
debian ease of use, ...
> >>[...about the logic of META files...]
> >
> >But we cannot ask every pacvkage maintainer to provide this file, they don't
> >know about it, it is heavy syntax and error prone (well when you add typos in
> >it).
>
> This is of course a problem. I still hope that other people adopt my system.
But you may not count on it, and hoping that other people adopt your system so
to make life easy on you is not a good solution.
What i propose is that it will make it easy for debian users to use findlib,
even for packages where upstream don't propose findlib support.
> For the most important INRIA packages, findlib's configure script adds the
> missing META file. Maybe you need a similar post-install script that adds
> missing META files in the same way. (I have good experiences with generating
> META by m4.)
Yes, that would be the ideal way, i don't know exactly what is needed, but i
guess you can answer us on this.
> >>
> >> So it is not necessary to update a meta-information db. Just add a META file
> >> for every package, and it will be found by findlib. Without findlib, the extra
> >> file can be ignored.
> >
> >Ok, but is it not possible to generate this meta information directly with a
> >simple script being invoked at install time ...
>
> Simple answer: No, it is not possible. META contains the information
> you typically find in _usage_ instructions, such as
> "Programs that use the XXX library must be linked as follows: XXX". It's an
> encoding of these instructions that are normally only available in English.
Err, how do you generate it then ?
> >>[...about $(CAMLLIB)/package-name...]
> >>
> >> However, I do not like this. CAMLLIB is already a quite large directory, and
> >> this makes it difficult to survey. A separate directory containing all add-on
> >> packages is simpler to handle, and easier to understand. I can do a
> >> ls /directory and it lists all installed packages.
> >
> >what about a finlibtool -list or something such ?
>
> This would be an alternative.
>
> >> Unfortunately, the caml team has decided to continue the $(CAMLLIB)/package-name
> >> pattern. In the upcoming 3.01 release the compiler understands
> >> -I +package-name and internally translates this option to
> >> -I (CAMLLIB)/package-name. What do you think about this?
> >
> >It is really a big plus in developpment when you don't do advanced stuff like
> >you do.
>
> To my mind, this sounds rather academic. The ideas in findlib result from
> project experience; maybe it is a bit more work to install the packages but it
> is a lot easier to use them. (Especially to tell other people how to use them.)
>
> >i would have advocated for a direct search path from $(CMALLIB) without the
> >need for the +package-name stuff.
> >
> >That said, mlgtk for example installed in $(CAMLLIB) directly.
> >
> >> (3.01 is not yet out. There is still the chance that they add a search path for
> >> packages.)
> >
> >a search patch, something more than the -I stuff ...
> >
> >> Findlib has a redirection feature which is currently used for the official
> >> packages, and for the core packages (from findlib's point of view, the
> >> libraries str, unix etc. are also packages needing meta-information). For
> >> example, for camltk I have in site-lib/camltk a META file (and nothing else)
> >> which redirects the location of the package:
> >
> >why not put all meta files into /var/lib/ocaml/xxx.meta or somethign such.
>
> You like centralized databases, don't you.
Well, everything belongs to it's right place. that is the advantage we have in
a integrated system, that you can do things properly, without needing to make
workaround because you don't control the layout of your whole system.
that said, i was speaking about a directory which will contain all meta files,
not a single meta file with all the data. A bit like the manpages are handled.
> Currently, I draw the following conclusions from our discussion:
> - findlib needs a configuration file specifying the search path and the default
> installation directory
Yes, this is somethign we agree on, i think it would have been done ever if we
never had this conversation.
> - the installation method needs a way to specify an alternate target directory
Yes, and ideally, a conventional install target, is it possible to
'pre-process' the ocamlfind stuff, in a way similar to how the
autmoake/autoconf stuff work ?
You use it for out_of_cvs versions, but not in released packages.
This is something every debian maintainer of one of your packages could do on
it's own if you don't support this.
> Furthermore, there is a problem with packages that do not support findlib in
> their source distribution, e.g. the INRIA packages. My recommendation is to
> add a post-install script that extracts the necessary data from somewhere and
> generates META.
Yes, this would be nice, but i think it would be best if you did this, as you
have the best knowledge of how things work in findlib, and may even change
things in future releases.
> I don't see the problem resulting from a deferred installation of findlib.
> Every package that wants to be findlib-compliant can install a META file for
> itself, and if findlib is installed later it will find all that already
> existing METAs. There is no way to generate META by a generic method for
> debian packages that do not provide it.
Ok, so each package that want to support findlib has to provide the correct
META file. mmm, ...
Do you have already a public file descirbing the META syntax or somethign
such, and explaining how to generate it ?
I guess that if the whole debina-ocaml stuff supported findlib directly, maybe
it would gain widder acceptance, which would be good ofr you.
Friendly,
Sven Luther
Reply to: