Classpath and versioned libraries for Debian-Java
Hi,
let me introduce myself before I go into detail - I am Max Kellermann, I live in Germany near Cologne and I'm 22 years old, I work as software developer (mostly Perl, and some Java). I have learned my first programming language at the age of 9 (Basic, Assembler, Pascal, C++, C, Java, Perl in that order, plus some more minor ones). I work with Linux since 1996 (well, at the beginning with SuSE), and I have learned Java in 1997, and I like the language, although I don't like the license and stability of most JVM's. On http://www.leanedit.org, I run a Java free software project. I would like to join the Debian project to package Java applications, I have already mailed with Ola about that, he will be my sponsor.
I have read the proposed Debian Java policy and some of the discussions on the list in its archive. The biggest problems seem to be installing versioned libraries and dependencies between these. Sun has missed to specify a standard here, IMHO the Java Extension mechanism is totally useless. So it's our task to clean up this mess..
I agree to most ideas you wrote here. The following text describes my idea of a solution; much of it was already said, but I'll repeat it so the context is clear..
At first, what do we need (don't be confused that I'm using many examples without mentioning that):
- there are applications and libraries
- applications may also contain reusable code, maybe it is not useful to make a big difference here
- libraries depend on other libraries, e.g. Xalan depends on an XML parser
- some version of a library depends on a version of another library
- there are libraries which only contain interfaces, e.g. org.w3c.dom
- some of these interface libraries are shipped in another library, e.g. org.w3c.dom is shipped with Xerces, even within the same .JAR file
- so one .JAR file may contain not only the main library (Xerces), but also multiple secondary libraries, each one has its own version number (DOM Level 1 or 2 etc.)
- several incompatible versions of a library should be installed on a single system at the same time, e.g. Xalan1 and Xalan2, meaning Xalan1 will never be automatically upgraded to Xalan2
- Xalan1 may be automatically upgraded to Xalan2 if there is a compatibility library, meaning Xalan2 "provides" Xalan1
- Xalan 2.1 is compatible with Xalan2, so Xalan 2.1 "provides" Xalan2, or Xalan 2.1 has the package name Xalan2 (because interfaces are compatible), but has version number 2.1.0
- if library B provides A in its the file B.jar, application C can't put A.jar into its classpath because package A is not installed anymore, only package B. Application C must put B.jar in its classpath, but application C needs to know that.. but it can't since it was packaged before library B was written to replace library A. We cannot require everybody to repackage applications only because some library was rewritten.
- library D v1.0 may be in the same classpath as library D v2.0; but: library E v1.0 must not be in the same classpath as library E v2.0 because they would conflict. We need to deal with that.
- library F and library G may both provide the sub-library H, of the same version - they may both be installed, but not in the same classpath at the same time.
- much more complicated stuff I've forgotten to mention here
Quite a long list, isnt it?
My idea is: (only concept, I will make it more concret if you like the ideas)
We can't let applications build the classpath themselves, because they can't possibly know where to find libraries. So we have to provide a tool which constructs the classpath, either pre-constructed (when a new library is installed) or on-the-fly (every time the application is started). I'm not sure which one of both, although I think pre-constructed would improve performance a bit, which is what Java applications need badly ;) (well, maybe you say it's only 0.00005% of the time the application needs to start..)
At first, we need a library database (applications will be put there, too). Maybe just a path containing a description file for every library, whatever you may call database. Every Java library and every Java application installs its description (maybe XML, maybe Debian control file format, I don't care at this time). The description contains a list of all .JAR files with its full path (although we may specify that all libraries will be installed in /usr/share/java).
For every .JAR file, it contains a list of something like 'virtual packages' - sub-libraries, provides lines, whatever we may call them; with version numbers of course. Xerces-1.4.3.jar contains Xerces (1.4.3), DOM Level 2 (1.0), SAX 2. Xalan-2.1.0.jar contains Xalan2 (2.1.0), TRAX (1.0). Every .JAR file also describes which 'sub-libraries' it requires.
There will be a script which takes library (or sub-library) names, looks up dependencies in the .JAR database, and prints the required CLASSPATH to stdout. Now application packagers are able to write wrapper scripts for their applications, which do something like 'export CLASSPATH=`/usr/share/java/build_classpath MyApp3`; java MyApp'.
This build_classpath script takes care of all dependencies between libraries. In the case above, F and G both provide H, and the application only requires H - then the script must decide which one to use. Maybe set priorities, something like /etc/alternatives? If some library the application requires F, but another library requires G, this is a conflict because H cannot be in the classpath twice - hm, what to do now? build_classpath must be quite "intelligent" to find the best way through the dependency jungle.
It would also be cool if Debian package dependencies would try to reflect dependencies between Java libraries in the .JAR database. So there would be the dh_java helper application which reads a (not-yet installed because not-yet-packaged) .JAR description file and generates Debian package dependencies of it. Library names can be directly converted into Debian virtual package names. Example: if the library 'xalan2' (xalan2.jar) contains the library 'trax', the library libxalan2-java provides the virtual package libtrax-java. So if my Application 'MyApp3' required 'trax' to run, dh_java can automatically generate 'Depends: libtrax-java' from this information for MyApp3 and writes 'Provides: libtrax-java' in libxalan2-java.
In addition to dh_java, we need scripts in java-common which update the library database, and a script which generates the classpath.
One topic I havn't written anything about what to do with conflicts.. conflicts should not happen because developers should know what they develop and Debian package dependencies do the rest. Well, they 'should'... I want Debian-Java to become rock-stable, the user must not fear starting a Java application because there might be a IncompatibleClassChangeError or NoClassDefFoundError "again".
That was it for now, please let me know what you think of it. I hope you get the ideas I had, maybe my sentences were not clear, ask me if something was not understandable.
Regards,
Max
Reply to: