binNMU order of ObjC/GNUstep packages

[Yavor Doganov <yavor@gnu.org>]

[CCing -release because this is a FAQ during GNUstep transitions; hope
you don't mind.]

Mehdi Dogguy wrote:
> On 08/21/2010 05:28 PM, Yavor Doganov wrote:
> > BTW, talksoup.app must be binNMUed on all archs once a binNMUed 
> > gnustep-netclasses is available everywhere.
> 
> I'll schedule the necessary binNMUs. I saw it green on my monitor
> because it depends on new gnustep libs, but has been compiled with
> pre-binNMU gnustep-netclasses. I didn't really test… but is the version
> currently in the archive broken? (can be tested on amd64 and i386).

AFAICT it works, but usually these breakages are discovered
post-factum.

The reason why we insist all apps depending on additional ObjC
libraries to be rebuilt against the *binNMUed* lib is because of the
way ObjC subclassing works.

/* Pseudo code.  */

/* Public class in gnustep-netclasses.  */
@interface IRC : NSString
{
@private
  int _net_ivar;
}
@end

/* Class in the application.  */
@interface TalkSoup : IRC
{
  NSString *soup_ivar;
}
- (void) setNick: (NSString *) nick
@end

Now, consider that NSString in gnustep-base/1.19.3 has 2 ivars, but
the ivar layout is changed in 1.20.x, adding another one (which is not
unusual at all):

@interface NSString : NSObject
{
  unsigned char *_ivar1;
  NSUinteger _ivar2;
+ CGFloat _ivar3;
}
@end

When the app is being run, and new instances of the TalkSoup class are
created, the ObjC runtime determines the size of the class in order to
allocate it.  The size of the class is the size of the ivars of the
root class NSObject (the so called "isa" pointer) plus the size of the
ivars of NSString (_ivar1, _ivar2, _ivar3), plus the size of the ivars
of the IRC class (_net_ivar), plus the size of TalkSoup's own ivars
(soup_ivar).  The overall size may grow or shrink depending on the
changes in the ivar layout throughout the class hierarchy.  That's the
main reason for the frequent GNUstep transitions we have in Debian;
it's very easy to break the ABI of an Objective-C library.

When the application code then invokes the -[TalkSoup setNick:] method
on the newly created instance, it attempts to assign a valid NSString
value to its soup_ivar instance variable, but it was compiled with the
assumption that the ivar layout is

{
  Class isa;
  unsigned char *_ivar1;
  NSUinteger _ivar2;
  int _net_ivar;
  NSString *soup_ivar;
}

but the actual layout at runtime is

{
  Class isa;
  unsigned char *_ivar1;
  NSUinteger _ivar2;
  CGFloat _ivar3;
  int _net_ivar;
  NSString *soup_ivar;
}

because of NSString's newly added ivar.  The assignment would
overwrite _net_ivar, leading to obvious problem at runtime (which may
be grave and apparent, or, depending on the case, not at all obvious,
but causing some other harm to the user or user's data, or simply
unnoticed bug leading to irrational program behavior under certain
circumstances -- it entirely depends on the specific case/code).

Errors like these are extremely hard to track down and analyze; they
lead to memory clobbering and allocation failures which flow down to
libc (through libobjc).  Back in the dark times, before binNMUs were
possible, I remember I spent literally weeks in a debugger trying to
"fix" such a problem.  When I rebuilt the app with debugging symbols
to obtain a meaningful backtrace, the issue was magically gone, so my
only sensible conclusion (completely wrong, of course) at that time
was that something was messy in the environment of the DD who uploaded
the package.

Thinking aloud, some classes are almost never subclassed.  It doesn't
make sense to subclass some library classes (like renaissance), so
this particular issue is more or less rare.  It's possible that
talksoup.app does not subclass any of gnustep-netclasses' classes, in
which case there is no problem.  But because it is not feasible to
maintain up-to-date information for every affected app (redoing the
audit of the code at every new release), it is simpler and easier to
require rebuilds, even if they are not actually necessary in many
situations.  The extra time spent by the buildds is certainly less
(and less costly) than developers' time when analyzing any eventual
problems.

This is also the reason why all non-core GNUstep libraries have
patches to link against -base and/or -gui, enabling the release team
to determine the correct dependency information (in the past, we used
to spell explicitly the order how binNMUs should be scheduled).