Re: And in 2019? Re: -flto to become more of a routine - any change in opinion since 2011?
>We just had SuSE embracing LTO
Not entirely.
With my DD hat doffed and wearing the mksh upstream developer hat,
I’ve been asking the package maintainers of mksh in all distributions
to remove the LTO flags, and will remove the built-in support for
using LTO in the next release.
Why?
mksh’s testsuite has proven to be pretty good in catching kernel,
libc, compiler and toolchain bugs. (I have commits in all libcs
for Linux except notably musl, fixing tons of bugs found this
way.)
However, the GCC developers have, time and time again, ignored
all but (I think) one of my bugreports regarding LTO, first
-fwhole-program --combine, then -flto with -fno-use-linker-plugin,
with -fuse-linker-plugin, or without, with or without -fwhole-program,
and lately -flto=jobserver with or without -fuse-linker-plugin.
Some of the code generation bugs are hard to find. mksh itself
is an interpreter language and the most recent bug, appearing
in GCC 6 IIRC and *still* pertinent in GCC 8 and I believe 9,
only shows up in one single testsuite failure of a program that
is written in the mksh Korn Shell scripting language, so, very
hard to reduce down (impossible for myself).
Nevertheless, all (except IIRC one) of those bugs got ignored
by the GCC developers, and while mksh’s testsuite caught some
of the codegen bugs I’m afraid of more LTO bugs lurking and
thus recommend people to NOT enable LTO for GCC in the general
case, only if they test the exact output with the exact com‐
piler version throughoutly before even attempting to use it.
I’ve removed -flto from the packages I maintain where I had
previously added it, happy for the benefits.
CI and tests do not help if nobody fixes the bugs. Worse, the
bugs will make it into production undetected.
Oh, and that’s not on an esoteric architecture. It’s on plain amd64.
mksh itself is not very large…
tg@blau:/usr/src/bin/mksh $ cat *.c *.h | wc
34477 119600 798703
and very portable code (it works on DEC ULTRIX with the
vendor C compiler, has an active OS/2 port, etc.) and I
regularily build it with whatever modern GCC is available
in Debian (even gcc-snapshot) with as many warnings en‐
abled as I can find, have had other people looking it
over and am pretty sure it does not fall into the category
of “design[ed] with knowledge of how compilers translate[…]”
that Ian writes about. (I do agree with his “The C standard
rules are obtuse, insanely complicated, and generally hard
to satisfy or even analyse in nontrivial cases.“ I rewrote
the arithmetic code to emulate signed arithmetic using only
unsigned C types, to avoid UB/IB.) mksh works with ANSI C89,
C99, C11 compilers and even some pre-ANSI ones. I literally
revived an entire Debian architecture in order to be able
to build and test mksh on another CPU architecture because
every port has let me find new bugs (or not, in the happy
case).
Therefore I state that the tooling is nowhere near working
enough for LTO, and note that the upstream developers of
said tooling are not interested in fixing it either, while
the developers of the to-be-compiled software don’t have
skills and tooling available in aiding them (e.g. to reduce
testcases which, in the LTO case, is extremely hard to im‐
possible anyway).
Ted, I agree, please drop LTO from e2fsprogs as well.
Thanks,
//mirabilos
--
11:56⎜«liwakura:#!/bin/mksh» also, i wanted to add mksh to my own distro │
i was disappointed that there is no makefile │ but somehow the Build.sh is
the least painful built system i've ever seen │ honours CC, {CPP,C,LD}FLAGS
properly │ looks cleary like done by someone who knows what they are doing
Reply to: