Dealing with API/ABI incompatibilities (was: guava-libraries 18?)
* Emmanuel Bourg:
> Le 27/10/2014 16:07, Hilko Bengen a écrit :
>
>> 1. Testing whether a rebuild against the newer version still works is
>> clearly not sufficient: code can be loaded at run-time, for instance.
> True, but that's more the exception than the rule, and I don't think
> many projects load Guava classes dynamically.
Even though the problem has surfaced with one specific library, I am
really looking for a more general solution.
>From my point of view, in order to be able to curate large numbers of
library packages (like we do for Debian), one should aim for generating
as little extra work for maintainers as possible. If we can avoid
looking at reverse dependencies, that's a large amount of work that can
be saved. It may mean little to you as a seasoned Java developer, but
for me, being forced to look at reverse dependencies or having to
rebuild them in order to see if I broke anything is a non-starter.
I can imagine that having to do this extra work would be overwhelming
for novice Debian contributors.
>> 3. We (Debian) already know how to deal with ABI breakage in shared
>> libraries produced from (mostly) C and C++ code: Change the SONAME
>> (upstream's job) and adjust the name of the resulting library package
>> accordingly.
>
> I'm not familiar enough with shared C/C++ libraries to compare, but I'd
> like to highlight that in Java you can't have two incompatible versions
> of the same library on the classpath. So if elasticsearch pulls another
> library that depends on guava but is incompatible with the version 18,
> it's likely to break at runtime.
C/C++ libraries have a similar problem but I can't recall any case where
and how it can be resolved.
>> In the short run, this increases the number of packages in the
>> distribution, but different versions of a given library can co-exist.
>
> With some care to the Maven artifacts though. The package with a fixed
> version must not install Maven artifacts with the generic 'debian'
> version, otherwise it would conflict with the main versionless
> package.
Of course. libguava-18-java would get an 18.x tag.
>> This generally saves work because checking a potentially large number
>> of reverse dependencies is not needed right away and needs to be done
>> only when the old version is removed.
>
> I'd argue that this may also create some work to transition the reverse
> dependencies.
Correct. It's less work, though, and it can be deferred to a later point
in time -- it's not necessary to do this work right away when a new
version of a library is introduced.
> Well we can and we do when it's necessary, but I think we should
> evaluate if this is necessary first. For example ASM used to break the
> compatibility on every new major release, but it got better with the
> version 5 and we have been able to upgrade the libasm4-java package
> directly instead of creating a new package and triggering a transition.
I agree: If version 19 or 20 of the Guava libraries does not break
compatibility, there's no reason to introduce a new library name: It can
still be libguava-18-java -- and use the 18.x tag for Maven as proposed
above.
>> 4. I don't really care about most of the packages that may or may not
>> break with the changed API/ABI in guava-18.
>
> I rebuilt the 26 reverse dependencies of Guava with the version 18 and I
> got only two failures with gradle and closure-compiler. They both failed
> due to the removal of ByteStreams.newInputStreamSupplier(byte[]) and
> Files.newReaderSupplier(File, Charset). So adding back these two methods
> is all that's really needed to transition to the new version.
Forward-porting APIs is clearly some extra work that can't be performed
by everybody. And, what's worse, this is working actively against
upstream's wishes. (In the Guava case, APIs are generally deprecated
over the course of two years and then removed.) Tracking
security-relevant bugs is also made harder.
Cheers,
-Hilko
Reply to: