[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Please reenable GCJ on mips

Nathanael Nerode wrote:
> >> Apparently the MIPS ABI is just plain broken.  It contains some sort of 
> >> impassable hard limit on relocation table size, breaking random packages at 
> >> random times with no possible fix.  Nobody can fix this without changing 
> >> the ABI.
> >
> Thiemo Seufer wrote:
> >That's wrong.
> OK.  Can somebody *describe* the damned bug?  Is it a hard limit on GOT size 
> which can't be exceeded (but breaks a predictable collection of packages)?  
> Is it a bug in the way ld constructs the GOT (say, not subdividing it 
> properly into multiple GOTs for 'multigot')?  Is it a bug in the way the gcj 
> Makefile *uses* ld, preventing ld from having the right information to 
> construct the GOT?  (If it's the latter, I can almost certainly fix it; I'm a 
> configury maintainer upstream for GCC.)
> Or is it a  mystical bug which nobody can actually describe which causes GOT 
> overflow in mysterious cases for mysterious reasons not to be questioned by 
> mortal men?  Because that's what it seems like, what with all these vague and 
> meaningless comments I'm hearing.  Does anyone actually know what the problem 
> is?

As far as I debugged it before sarge, it is a collection of bugs, or
insufficiencies which may qualify as bugs depending on the context.
Btw, I haven't seen anyone mentioning this problem except for Debian
and Gentoo, it seems the embedded mips vendors have not much use for
insanely large binaries. Debian is pushing the envelope here.

We have for MIPS:

  - The plain GOT mode, where a GOT has a maximum size of 2^16 byte,
    with 16k symbols.

  - The XGOT mode, with unlimited (2^32 byte) size, which increases
    code size by 15-20%, and reduces perfomance accordingly.

  - MultiGOT, which creates several GOTs in a single binary for the
    final link, but uses only one GOT for imports/exports. The object
    files still have only plain GOT.

MultiGOT is supposed to be the current best solution, and XGOT is
supposed to be obsoleted by it. Normally plain GOT is used, and the
linker switches to MultiGOT in the final link if the GOT grows too

The problems, as far as I know about them:

  - A too large object file can overflow plain GOT. This is not only
    MIPS-specific, it affects several architecture's toolchains, and
    was exposed pre-sarge (IIRC most virulently on sparc) by a
    broken/deficient libtool which relinked things into a single huge
    object file.
    libtool was fixed, and the remaining cases (like a huge blob of
    generated C code for python bindings) learned to split the C
    source to some smaller pieces, which also helped link speed.
    For MIPS, and if the need arises, this could be worked around by
    using XGOT, but see below. The real fix would be a MultiGOT
    extension to the object format, which is possible in a downward
    compatible way but not implemented yet.

  - MultiGOT works fine, until the limit of 16k _dynamic_ symbols is
    hit. A executable/library with larger exported GOT will build
    without warning but will cause ld.so to segfault. This is the main
    bug, and hard to debug (a statically built gdb may help here).
    This hits currently (at least) the gcj shared library runtime,
    the ghc executable, and libgklayout.so in mozilla*. A workaround
    involving XGOT is possible in some cases, and was done for the
    mozillae (and some others, grepping for -xgot in build logs seems
    to be the most reliable way to find them all). Dynamically linked
    executables/shared libraries with any of the different internal GOT
    models are freely mixable.

  - XGOT and MultiGOT are mutually exclusive, because the MultiGOT
    handling ignores XGOT relocations. This is arguably not a bug,
    since MultiGOT is supposed to supersede XGOT. The resulting binary
    will crash at the first XGOT relocation outside the plain GOT
    limit. The fact that the linker accepts XGOT when being in MultiGOT
    mode is a bug. Worse, upstream binutils always invoke MultiGOT
    linking, even when all object files are XGOT.
    Fixing it requires the addition of a XGOT flag to the object file.
    Currently there is a XGOT ELF header flag, but it was used by SGI
    for unknown purpose, reusing it may not be advisable. Obsoleting
    XGOT altogether may be more advisable...

The partial XGOT workaround involves:

  - switching ld from MultiGOT to XGOT linking once it sees more than
    16k exported symbols. This is, in the general picture, a step
    backwards, and thus a debian-specific binutils patch. It needs to
    be removed once ld.so is fixed, and all -xgot users (see below)
    will need to be changed at that time.

  - Add -xgot (or -Wa,-xgot) to the CFLAGS for objects which go in the
    big binary. Make sure everything else has _no_ -xgot flag, since
    this may break a MultiGOT link below the critical size. Make sure
    the same is true for static library objects.

The workaround fails as soon as it needs to link in non-xgot files,
typically the crt* startup files of gcc and glibc, and those startup
file's relocations happen to not end up in the first 64k of the XGOT.


Reply to: