Re: Please reenable GCJ on mips
Nathanael Nerode wrote:
> >> Apparently the MIPS ABI is just plain broken. It contains some sort of
> >> impassable hard limit on relocation table size, breaking random packages at
> >> random times with no possible fix. Nobody can fix this without changing
> >> the ABI.
> >
>
> Thiemo Seufer wrote:
> >That's wrong.
>
> OK. Can somebody *describe* the damned bug? Is it a hard limit on GOT size
> which can't be exceeded (but breaks a predictable collection of packages)?
> Is it a bug in the way ld constructs the GOT (say, not subdividing it
> properly into multiple GOTs for 'multigot')? Is it a bug in the way the gcj
> Makefile *uses* ld, preventing ld from having the right information to
> construct the GOT? (If it's the latter, I can almost certainly fix it; I'm a
> configury maintainer upstream for GCC.)
>
> Or is it a mystical bug which nobody can actually describe which causes GOT
> overflow in mysterious cases for mysterious reasons not to be questioned by
> mortal men? Because that's what it seems like, what with all these vague and
> meaningless comments I'm hearing. Does anyone actually know what the problem
> is?
As far as I debugged it before sarge, it is a collection of bugs, or
insufficiencies which may qualify as bugs depending on the context.
Btw, I haven't seen anyone mentioning this problem except for Debian
and Gentoo, it seems the embedded mips vendors have not much use for
insanely large binaries. Debian is pushing the envelope here.
We have for MIPS:
- The plain GOT mode, where a GOT has a maximum size of 2^16 byte,
with 16k symbols.
- The XGOT mode, with unlimited (2^32 byte) size, which increases
code size by 15-20%, and reduces perfomance accordingly.
- MultiGOT, which creates several GOTs in a single binary for the
final link, but uses only one GOT for imports/exports. The object
files still have only plain GOT.
MultiGOT is supposed to be the current best solution, and XGOT is
supposed to be obsoleted by it. Normally plain GOT is used, and the
linker switches to MultiGOT in the final link if the GOT grows too
big.
The problems, as far as I know about them:
- A too large object file can overflow plain GOT. This is not only
MIPS-specific, it affects several architecture's toolchains, and
was exposed pre-sarge (IIRC most virulently on sparc) by a
broken/deficient libtool which relinked things into a single huge
object file.
libtool was fixed, and the remaining cases (like a huge blob of
generated C code for python bindings) learned to split the C
source to some smaller pieces, which also helped link speed.
For MIPS, and if the need arises, this could be worked around by
using XGOT, but see below. The real fix would be a MultiGOT
extension to the object format, which is possible in a downward
compatible way but not implemented yet.
- MultiGOT works fine, until the limit of 16k _dynamic_ symbols is
hit. A executable/library with larger exported GOT will build
without warning but will cause ld.so to segfault. This is the main
bug, and hard to debug (a statically built gdb may help here).
This hits currently (at least) the gcj shared library runtime,
the ghc executable, and libgklayout.so in mozilla*. A workaround
involving XGOT is possible in some cases, and was done for the
mozillae (and some others, grepping for -xgot in build logs seems
to be the most reliable way to find them all). Dynamically linked
executables/shared libraries with any of the different internal GOT
models are freely mixable.
- XGOT and MultiGOT are mutually exclusive, because the MultiGOT
handling ignores XGOT relocations. This is arguably not a bug,
since MultiGOT is supposed to supersede XGOT. The resulting binary
will crash at the first XGOT relocation outside the plain GOT
limit. The fact that the linker accepts XGOT when being in MultiGOT
mode is a bug. Worse, upstream binutils always invoke MultiGOT
linking, even when all object files are XGOT.
Fixing it requires the addition of a XGOT flag to the object file.
Currently there is a XGOT ELF header flag, but it was used by SGI
for unknown purpose, reusing it may not be advisable. Obsoleting
XGOT altogether may be more advisable...
The partial XGOT workaround involves:
- switching ld from MultiGOT to XGOT linking once it sees more than
16k exported symbols. This is, in the general picture, a step
backwards, and thus a debian-specific binutils patch. It needs to
be removed once ld.so is fixed, and all -xgot users (see below)
will need to be changed at that time.
- Add -xgot (or -Wa,-xgot) to the CFLAGS for objects which go in the
big binary. Make sure everything else has _no_ -xgot flag, since
this may break a MultiGOT link below the critical size. Make sure
the same is true for static library objects.
The workaround fails as soon as it needs to link in non-xgot files,
typically the crt* startup files of gcc and glibc, and those startup
file's relocations happen to not end up in the first 64k of the XGOT.
Thiemo
Reply to: