[Bug libgcj/16122] gij - Incorrect result due to computations in extended precision on x86
------- Comment #4 from aph at gcc dot gnu dot org 2006-02-14 15:45 -------
A bit more explanation.
The problem is caused by the fact that 9007199254740994.0 + 0.9999847412109375
is carried out in extended precision, and the result is rounded to
9007199254740995. In double precision, the result of this calculation is
rounded to 9007199254740994.
When the extended-precision value is rounded to double for storing, it is
rounded (again) to 9007199254740996.
So, double rounding leads gij to return a value of d that is 2.0, whereas it
should be 0.0.
Note however, that the true accurate value for d, calculated at infinite
precision, is 1-(2^-16). So, the absolute error for gcj is 1+(2^-16) and the
absolute error with correct rounding is 1-(2^-16). (I'm not surprised this
hasn't been reported as a problem with any real applications!)
It might be worth setting the floating-point precision of gcj to double, but
that would only fix the double-precision case, and I presume we'd still have
the same double rounding problem for floats.
And in any case, I do not know if libcalls would be affected by being entered
with the FPU in round-to-double mode. We might end up breaking things.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16122
------- You are receiving this mail because: -------
You reported the bug, or are watching the reporter.
Reply to: