Google Summer of Code - Jigsaw - Report 5 (final)
-------------------------------------------------
The goal of this Google Summer of Code at Debian was to package Jigsaw
(the modularized JDK that will be shipped with JDK8). I tried to do
everything I can to get the job done. Sadly, I haven't completed the
packaging task. I have done several tasks such as compiling Jigsaw,
understanding how it works, writing examples, packaging 2 dependencies,
fixing and running tests and finally starting to package Jigsaw.
The beginning of this year's GSoC consisted in getting the source code
of Jigsaw and trying to compile it. I also rebuilt OpenJDK 6 and
OpenJDK 7 packages to get used to the way they are packacged. This was
not the hardest thing I had to do. After few hours, I was able to
compile Jigsaw and to know what is needed to do so. Jigsaw is still a
big piece of code so it takes time to compile. On my own machines it
takes more than 2 hours but Tom Marble gave me an access to a more
powerfull computer so I was able to compile Jigsaw in 15 minutes.
Being able to run the tests was a big part of the work. Running the
tests of Jigsaw is important because if the tests pass it means that
Jigsaw will work as expected once it is installed. I took a lot of
times to make sure that there are as many tests that pass as possible.
Running the tests is a very long process, even longer than the build
process but it also needs some piece of softwares. That is how I
started to work and JT Harness and JT Reg. JT Reg provides two binaries
"jtreg" and "jtdiff" which are used to run the tests. JT Harness is a
library used by JT Reg. There are also tests that require JUnit. The
hackers of IcedTea helped me a lot for this work. So I finally created
a patch to fix some tests that did not pass due to some unreachable
hosts. The IcedTea project was aware of that so they provide their own
hosts (which we can reach).
Creating JT Harness and JT Reg packages took me more time than I
thought. I had to learn what javahelper is and how it works. Packaging
JT Reg was more complicated because I had to generate man pages for the
2 binaries. I think that I spent too much time on this. Even if I had
already do some packaging work before this GSoC, I learnt several things
which I haven't see in Debian packaging before.
I also spent some time to write examples which use Jigsaw. This is how I
wrote an example of a module to see how to create and install it. It
helps to understand how things work and also how Jigsaw in general works.
It's also a good way to know that Jigsaw is working properly even if the
tests are made for this.
During a week, I had the pleasure to attend DebConf 11 at Banja Luka. It
was really fun to meet the Debian community and especially the GSoC mentors
and administrators. It was a good way to learn a lot about Debian, to work
with Tom Marble and Sylvestre Ledru and also to have fun. Meeting all those
smart people is really a great experience. DebConf 11 was also a place when
I was able to work (at one of the hacklabs or at the hotel). Tom, Sylvestre
and I gave a talk about our GSoC project[2]. This talk is about what I was
done so far and what Jigsaw is. The slides and a video are available so
don't hesitate to take a look at them.
- Here is what happened since the last report
After DebConf I started to work on packaging Jigsaw. This is a complex
task for me because it is a complicated piece of software. Sylvestre
created a Git repository[1] so we can track the progress of the packaging.
After studying the OpenJDK 7 packaging, I decided to create a new debian/
directory from scratch. I think that it is easier to re-think the packaging
of OpenJDK. Of course the current packages are a source of inspiration.
The build system of Jigsaw is supposed to generate Debian packages for each
modules when using the "modules" target of the main makefile. Actually, it
doesn't work with the current source code. So I decided to fix this before
trying to make my own packages. Only one line is the cause of the problem.
So I started by editing the file:
- jdk/src/share/classes/org/openjdk/jigsaw/cli/Packager.java
This file contains a method called translateVersion(String) which
returns a string. This methode is quite simple. It just replaces all -
characters by the _ character. This is bad. I tried a simple fix by replacing
all - characters by a dot and it worked. So here is what the method looked
like after being patched.
private String translateVersion(String v) {
return v.replaceAll("-", ".");
}
I have sent a mail to the jigsaw-dev mailing list to know why people
wanted to use the _ character, which is illegal in Debian but you know that,
in the version string. After recompiling the code building the Jigsaw modules
started to build things I had nerver see such as Debian package for each
modules and 2 new images called "jdk-module-image" and "jre-module-image".
These images are like the 2 images built when compiling the current OpenJDK but
the module images are as you can guess images when all modules are installed.
This gives me a good clue to know how the Jigsaw directory should look like
once modules are installed. Debian packages being built by the Jigsaw build
system, I took a look at them. I didn't not expect such a result. Everything is
installed in /usr/local/jigsaw/ and that's not want we want. But still,
studying these packages is interesting to know what Oracle wanted to do and it
can give us some pointers to what we have to do to install the modules. The
preinst, postint and prerm scripts are interesting to read.
The debian/ directory in the Git repository is a copy of the OpenJDK 7
debian/ directory package. There is a second directory called debian-wip/
which is the directory I am working on. The debian/rules file was the most
impressive file in all previous OpenJDK package. Mine is much more simple to
understand. There are only few environment variables which are set and then it
is just several overrides that call make as expected by the Jigsaw build system.
#!/usr/bin/make -f
# Some export and unexport not interesting here
dh_override_auto_configure:
$(MAKE) sanity
java -version 2>&1 | grep "java -version \"1.7" ; \
if [ $$? -eq 1 ] ; then echo "OpenJDK 7 needed to build" ; \
exit 1 ; fi
dh_override_auto_build:
$(MAKE) all
$(MAKE) modules
override_dh_auto_test:
xvfb-run -e xvfb-errors -a -s -ac $(MAKE) test -k
%:
dh $@
Basically, we run "make sanity" to ensure our environment is clean to
build Jigsaw, then we check that OpenJDK 7 is installed (because Jigsaw is now
in sync with JDK8). Then we build, we build the modules and after that we run
the tests using Xvfb which is required for graphical user interface tests.
The debian/control file lists a lot of Debian binary packages to build
(one for each modules plus few other). Creating the debian/ directory for
packaging Jigsaw consist in using all the previous work I have done before. So
there is several patches which are applied before trying to build Jigsaw.
These patches are about differents things
- 01_jpkg_fix.patch: this one is the fix so jpkg can generate .deb.
- 02_jtreg_binary.patch: this one is used to locate jtreg and
jtdiff.
- 03_hosts_tests.patch: this one changes the hosts to reach for
testing.
- 04_junit_jar.patch: this one is used to locate the junit JAR file.
- 05_disable_tests.patch: this one is used to disable tests that
don't pass
yet (about 15 tests).
All those patches have been made using all the work I have done before
trying to package Jigsaw.
This Google Summer of Code was not long enough for me to finalize the
package of Jigsaw. The code is public (of course) so anybody can help us to
conclude this work. I was already involved in Debian before doing this GSoC
and I plan to keep doing some work for Debian after. I am really motivated to
continue the work I started on Jigsaw. If you want to help, Sylvestre created
a IRC channel to talk about Jigsaw, feel free to join us (#debian-jigsaw on
irc.debian.org).
Finally, I would like to thank my mentors Tom Marble and Sylvestre Ledru
for their help and for answering to all the questions I have asked. I hope
to finish the Jigsaw packaging after this GSoC with them and maybe with other.
I'm still really interested in modularizing Java for Debian so I looking
forward to this.
See you around.
[1] http://anonscm.debian.org/gitweb/?p=pkg-java/jigsaw.git
[2] http://penta.debconf.org/dc11_schedule/events/718.en.html
Attachment:
signature.asc
Description: Digital signature