Re: [Soc-coordination] Jigsaw - Report 5 (final)
2011/8/22 Guillaume Mazoyer <respawneral@gmail.com>:
> Google Summer of Code - Jigsaw - Report 5 (final)
> -------------------------------------------------
>
> The goal of this Google Summer of Code at Debian was to package Jigsaw
> (the modularized JDK that will be shipped with JDK8). I tried to do
> everything I can to get the job done. Sadly, I haven't completed the
> packaging task. I have done several tasks such as compiling Jigsaw,
> understanding how it works, writing examples, packaging 2 dependencies,
> fixing and running tests and finally starting to package Jigsaw.
>
> The beginning of this year's GSoC consisted in getting the source code
> of Jigsaw and trying to compile it. I also rebuilt OpenJDK 6 and
> OpenJDK 7 packages to get used to the way they are packacged. This was
> not the hardest thing I had to do. After few hours, I was able to
> compile Jigsaw and to know what is needed to do so. Jigsaw is still a
> big piece of code so it takes time to compile. On my own machines it
> takes more than 2 hours but Tom Marble gave me an access to a more
> powerfull computer so I was able to compile Jigsaw in 15 minutes.
>
> Being able to run the tests was a big part of the work. Running the
> tests of Jigsaw is important because if the tests pass it means that
> Jigsaw will work as expected once it is installed. I took a lot of
> times to make sure that there are as many tests that pass as possible.
> Running the tests is a very long process, even longer than the build
> process but it also needs some piece of softwares. That is how I
> started to work and JT Harness and JT Reg. JT Reg provides two binaries
> "jtreg" and "jtdiff" which are used to run the tests. JT Harness is a
> library used by JT Reg. There are also tests that require JUnit. The
> hackers of IcedTea helped me a lot for this work. So I finally created
> a patch to fix some tests that did not pass due to some unreachable
> hosts. The IcedTea project was aware of that so they provide their own
> hosts (which we can reach).
>
> Creating JT Harness and JT Reg packages took me more time than I
> thought. I had to learn what javahelper is and how it works. Packaging
> JT Reg was more complicated because I had to generate man pages for the
> 2 binaries. I think that I spent too much time on this. Even if I had
> already do some packaging work before this GSoC, I learnt several things
> which I haven't see in Debian packaging before.
>
> I also spent some time to write examples which use Jigsaw. This is how I
> wrote an example of a module to see how to create and install it. It
> helps to understand how things work and also how Jigsaw in general works.
> It's also a good way to know that Jigsaw is working properly even if the
> tests are made for this.
>
> During a week, I had the pleasure to attend DebConf 11 at Banja Luka. It
> was really fun to meet the Debian community and especially the GSoC mentors
> and administrators. It was a good way to learn a lot about Debian, to work
> with Tom Marble and Sylvestre Ledru and also to have fun. Meeting all those
> smart people is really a great experience. DebConf 11 was also a place when
> I was able to work (at one of the hacklabs or at the hotel). Tom, Sylvestre
> and I gave a talk about our GSoC project[2]. This talk is about what I was
> done so far and what Jigsaw is. The slides and a video are available so
> don't hesitate to take a look at them.
>
> - Here is what happened since the last report
>
> After DebConf I started to work on packaging Jigsaw. This is a complex
> task for me because it is a complicated piece of software. Sylvestre
> created a Git repository[1] so we can track the progress of the packaging.
> After studying the OpenJDK 7 packaging, I decided to create a new debian/
> directory from scratch. I think that it is easier to re-think the packaging
> of OpenJDK. Of course the current packages are a source of inspiration.
> The build system of Jigsaw is supposed to generate Debian packages for each
> modules when using the "modules" target of the main makefile. Actually, it
> doesn't work with the current source code. So I decided to fix this before
> trying to make my own packages. Only one line is the cause of the problem.
> So I started by editing the file:
> - jdk/src/share/classes/org/openjdk/jigsaw/cli/Packager.java
>
> This file contains a method called translateVersion(String) which
> returns a string. This methode is quite simple. It just replaces all -
> characters by the _ character. This is bad. I tried a simple fix by replacing
> all - characters by a dot and it worked. So here is what the method looked
> like after being patched.
>
> private String translateVersion(String v) {
> return v.replaceAll("-", ".");
> }
>
> I have sent a mail to the jigsaw-dev mailing list to know why people
> wanted to use the _ character, which is illegal in Debian but you know that,
> in the version string. After recompiling the code building the Jigsaw modules
> started to build things I had nerver see such as Debian package for each
> modules and 2 new images called "jdk-module-image" and "jre-module-image".
> These images are like the 2 images built when compiling the current OpenJDK but
> the module images are as you can guess images when all modules are installed.
> This gives me a good clue to know how the Jigsaw directory should look like
> once modules are installed. Debian packages being built by the Jigsaw build
> system, I took a look at them. I didn't not expect such a result. Everything is
> installed in /usr/local/jigsaw/ and that's not want we want. But still,
> studying these packages is interesting to know what Oracle wanted to do and it
> can give us some pointers to what we have to do to install the modules. The
> preinst, postint and prerm scripts are interesting to read.
>
> The debian/ directory in the Git repository is a copy of the OpenJDK 7
> debian/ directory package. There is a second directory called debian-wip/
> which is the directory I am working on. The debian/rules file was the most
> impressive file in all previous OpenJDK package. Mine is much more simple to
> understand. There are only few environment variables which are set and then it
> is just several overrides that call make as expected by the Jigsaw build system.
>
> #!/usr/bin/make -f
>
> # Some export and unexport not interesting here
>
> dh_override_auto_configure:
> $(MAKE) sanity
> java -version 2>&1 | grep "java -version \"1.7" ; \
> if [ $$? -eq 1 ] ; then echo "OpenJDK 7 needed to build" ; \
> exit 1 ; fi
>
> dh_override_auto_build:
> $(MAKE) all
> $(MAKE) modules
>
> override_dh_auto_test:
> xvfb-run -e xvfb-errors -a -s -ac $(MAKE) test -k
>
> %:
> dh $@
>
> Basically, we run "make sanity" to ensure our environment is clean to
> build Jigsaw, then we check that OpenJDK 7 is installed (because Jigsaw is now
> in sync with JDK8). Then we build, we build the modules and after that we run
> the tests using Xvfb which is required for graphical user interface tests.
>
> The debian/control file lists a lot of Debian binary packages to build
> (one for each modules plus few other). Creating the debian/ directory for
> packaging Jigsaw consist in using all the previous work I have done before. So
> there is several patches which are applied before trying to build Jigsaw.
> These patches are about differents things
>
> - 01_jpkg_fix.patch: this one is the fix so jpkg can generate .deb.
> - 02_jtreg_binary.patch: this one is used to locate jtreg and
> jtdiff.
> - 03_hosts_tests.patch: this one changes the hosts to reach for
> testing.
> - 04_junit_jar.patch: this one is used to locate the junit JAR file.
> - 05_disable_tests.patch: this one is used to disable tests that
> don't pass
> yet (about 15 tests).
>
> All those patches have been made using all the work I have done before
> trying to package Jigsaw.
>
> This Google Summer of Code was not long enough for me to finalize the
> package of Jigsaw. The code is public (of course) so anybody can help us to
> conclude this work. I was already involved in Debian before doing this GSoC
> and I plan to keep doing some work for Debian after. I am really motivated to
> continue the work I started on Jigsaw. If you want to help, Sylvestre created
> a IRC channel to talk about Jigsaw, feel free to join us (#debian-jigsaw on
> irc.debian.org).
>
> Finally, I would like to thank my mentors Tom Marble and Sylvestre Ledru
> for their help and for answering to all the questions I have asked. I hope
> to finish the Jigsaw packaging after this GSoC with them and maybe with other.
> I'm still really interested in modularizing Java for Debian so I looking
> forward to this.
>
> See you around.
>
> [1] http://anonscm.debian.org/gitweb/?p=pkg-java/jigsaw.git
> [2] http://penta.debconf.org/dc11_schedule/events/718.en.html
>
> _______________________________________________
> Soc-coordination mailing list
> Soc-coordination@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/soc-coordination
>
Good work, keep it going. Java 8 is scheduled to launch sometime next
year. Maybe we will have Jigsaw in Debian by then.
--
Ioan Eugen Stan
http://ieugen.blogspot.com/
Reply to: