[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Updating Eclipse



Thanks for your quick replies.
I have spent some time reading the doc and poking at the code, and I must admit that I'm still confused :)
However I have not tried building the package locally.

Before more specific comments/questions on some of your answers, here are some more general questions?
- What is the root of the decision to build everything from source? In a situation where the upstream provider has a clear policy about what goes in and out, with a relatively simple and uniform build process (and the output is signed), I'm wondering if there can't be a trust system that gets established so everything does not have to be rebuilt.
- What is the limit of the actions that can be run in an install script? For example, could parts of the application being installed be used during the install script (e.g. call a utility to generate files)?
- What drives the decision to package certain Eclipse libraries as system jars and not others (e.g. swt and osgi, but not emf or jface whereas they can run on plain java as well)?
- Do we have stats about what the user is downloading? The reason I'm asking is to understand if all the packages are needed.

See also embedded questions below



[...]
I believe Markus already touched on some of these.  I only have an
incomplete list.  None that some of my remarks / follow comments here
are based on my time as Debian maintainer of eclipse (mostly active
during 2009-2011ish). Anyhow:

 * DFSG compatible licenses.  Here Eclipse general excels with its IP
   and rigid handling of copyright/licenses.
 * Rebuild things from source.  Some of it works, except for the
   bootstrap problem.  Namely to compile eclipse you (used to?) require
   eclipse.
    This was indeed the case in the past, and it is still the same. The difference is that now everything is built with Maven rather than PDE build. More specifically, it is built using a Maven plug-in called Tycho which embeds parts of Eclipse (mostly osgi, ecf and p2).

 * No / Avoid bundling of libraries and avoid static linking.
 * Rebuild everything *without internet*.  Mostly an issue with maven.
   In my time, Eclipse did not have an issue here, but I heard they
   considered moving to maven or so?
    Really? Without internet access? With Tycho in the middle of this, it means that we would first have to build all the Maven dependencies (which may already be handled), plus all the Tycho dependencies which themselves depend on Eclipse bits and pieces. Definitely not a piece of cake, but it seems that the ppl from Fedora have managed to get this working by using artifacts from a previous build.

 * Debian generally tries to avoid "= $version" dependencies, which are
   fairly common in the Java world (if I am not mistaken).
    Yes, this is true especially in the Maven world since it guarantees build reproducibility. Eclipse is a bit more flexible in how plug-ins specify their dependencies on other plug-ins, but control is happening higher up in features or in product since it is for us a way to make sure the user is running a combination of software that has been tested together (at least to some degree).

Again note that my remarks here are based from my experience and may
have changed since 2011.  Hopefully you will find it easier - your time
as Eclipse committer might make a lot of things easier for you.  Also
please note that some of these issues are not specific to Debian (e.g.
Fedora have similar concerns).

 * Eclipse upstream and Debian had different ideas on certain things
   like "how much should be rebuilt from source".
   - Debian: Everything *must* be rebuildable and everything *should* be
     be rebuilt during a package build.
   - Eclipse: Automatic rebuilds are /sometimes/ an afterthought or not
     done at all (e.g. because they "consume the official" binaries of
     XYZ).
    Originally (back in 2001) the source files were provided for "viewing pleasure" and to allow users to fix a plug-in by working in the IDE. Also the thing is that with the original build system (PDE), you could not simply download the dependencies to compile your plug-in, which made the inclusion of build scripts moot. However the eclipse platform provided a huge tar.gz files with all the source from the eclipse SDK along side with a bunch of ant scripts (some of which were generated). This was not ideal either but solved the purpose of rebuilding from source but this was only done for the SDK and not generalized to other plug-ins.
    Now with the massive adoption of Maven/Tycho (and some other wrapping scripts called CBI), it should be possible to build anything independently and have the right thing to happen. But, this does not resolve the more fundamental difference between the Java and Linux ecosystems of reusing vs rebuilding
.

[...]
 * Providing OSGi support for (non-eclipse) JAR files in other debian
   packages.
   - We spent a few month adding those back in 2009 and 2010.
     Hopefully most of it is still correct.  But it was a lot of work
     to migrate from eclipse 3.2 to 3.5.
    Sigh... Nowadays, more jars have manifest but work will still be required.
 * When build breaks, it is sometimes painful to figure out what went
   wrong.  Especially when the upstream build does not fail on first
   error (but instead continues and explodes later or creates broken
   binaries/jar files).
    With Tycho, this has changed since the build proceeds depth first (it compiles and produces the jar before moving to the next one) whereas PDE build proceeded breath first.

But in summary, have a look at the eclipse-build project[2].  This is
probably the best possible entry point to compiling a newer eclipse.
  It is definitely doable, but it required a larger commitment than I
was able to provide more than a couple of years (as a volunteer).
    Kudos to you and the other people for what you've done so far. Getting the package done and maintain it for so long was already a big task especially with how things were upstream.
    Now what strikes me with the current approach is the amount of effort required to maintain an individual package, and I would really hope that we can find a more systematic way of bringing various parts of Eclipse to debian, especially that we want to provide more eclipse based applications to the user (e.g. there is a standalone C debugger (https://wiki.eclipse.org/CDT/StandaloneDebugger), or trace compass (https://projects.eclipse.org/proposals/trace-compass)).


Again, thanks for your help and time. I will talk with the people funding the effort to see what are the next steps, so don't take it personally if you never hear from me :)

Regards,

Pascal


~Niels

PS: You will want a machine with sufficient RAM.  I suspect today's
average hardware have sufficient capacity, but it is the difference
between a 15 minutes and a 90 minutes build (and possibly your build or
desktop being OOM-killed).

[1] https://www.debian.org/doc/debian-policy/

[2] http://wiki.eclipse.org/Linux_Tools_Project/Eclipse_Build

[3] The debian/eclipse-build-generatedScripts.tar.bz2 variant was
because we needed a rebuilt after the one provided from eclipse-build
and I did not want to override a binary file in the "orig" tarball.

[4] Fedora uses /bin/bash as /bin/sh.  Debian migrated to /bin/dash, so
a lot of the "extra" features in bash are not present in our /bin/sh.
These bashism problems occur in many different places (even outside
eclipse-build).




Reply to: