Re: Bypassing the 2/3/4GB virtual memory space on 32-bit ports
On Wed, Aug 14, 2019 at 5:13 PM Aurelien Jarno <aurelien@aurel32.net> wrote:
> > a proper fix would also have the advantage of keeping linkers for
> > *other* platforms (even 64 bit ones) out of swap-thrashing, saving
> > power consumption for build hardware and costing a lot less on SSD and
> > HDD regular replacements.
>
> That would only fix ld, which is only a small part of the issue. Do you
> also have ideas about how to fix llvm, gcc or rustc which are also
> affected by virtual memory exhaustion on 32-bit architectures?
*deep breath* - no. or, you're not going to like it: it's not a
technical solution, it's going to need a massive world-wide sustained
and systematic education campaign, written in reasonable and logical
language, explaining and advising GNU/Linux applications writers to
take more care and to be much more responsible about how they put
programs together.
a first cut at such a campaign would be:
* designing of core critical libraries to be used exclusively through
dlopen / dlsym. this is just good library design practice in the
first place: one function and one function ONLY is publicly exposed,
returning a pointer to a table-of-functions (samba's VFS layer for
example [1]).
* compile-time options that use alternative memory-efficient
algorithms instead of performance-efficient ones
* compile-time options to remove non-essentlal resource-hungry features
* compiling options in Makefiles that do not assume that there is vast
amounts of memory available (KDE "developer" mode for example would
compile c++ objects individually whereas "maintainer" mode would
auto-generate a file that #included absolutely every single .cpp file
into one, because it's "quicker").
* potential complete redesigns using IPC/RPC modular architectures:
applying the "UNIX Philosophy" however doing so through a runtime
binary self-describing "system" specifically designed for that
purpose. *this is what made Microsoft [and Apple] successful*. that
means a strategic focus on getting DCOM for UNIX up and running [2].
god no please not d-bus [3] [4].
also, it's going to need to be made clear to people - diplomatically
but clearly - that whilst they're developing on modern hardware
(because it's what *they* can afford, and what *they* can get - in the
West), the rest of the world (particularly "Embedded" processors)
simply does not have the money or the resources that they do.
unfortunately, here, the perspective "i'm ok, i'm doing my own thing,
in my own free time, i'm not being paid to support *your* hardware" is
a legitimate one.
now, i'm not really the right person to head such an effort. i can
*identify* the problem, and get the ball rolliing on a discussion:
however with many people within debian alone having the perspective
that everything i do, think or say is specifically designed to "order
people about" and "tell them what to do", i'm disincentivised right
from the start.
also, i've got a thousand systems to deliver as part of a
crowd-funding campaign [and i'm currently also dealing wiith designing
the Libre RISC-V CPU/GPU/VPU]
as of right now those thousand systems - 450 of them are going to have
to go out with Debian/Testing 8. there's no way they can go out with
Debian 10. why? because this first revision hardware - designed to be
eco-conscious - uses an Allwinner A20 and only has 2GB of RAM
[upgrades are planned - i *need* to get this first version out, first]
with Debian 10 requiring 4GB of RAM primarily because of firefox,
they're effectively useless if they're ever "upgraded"
that's a thousand systems that effectively go straight into landfill.
l.
[1] https://www.samba.org/samba/docs/old/Samba3-Developers-Guide/vfs.html#id2559133
[2] incredibly, Wine has had DCOM and OLE available and good enough to
use, for about ten years now. it just needs "extracting" from the
Wine codebase. DCOM stops all of the arguments over APIs (think
"libboost". if puzzled, add debian/testing and debian/old-stable to
/etc/apt/sources.lst, then do "apt-cache search boost | wc")
due to DCOM providing "a means to publish a runtime self-describing
language-independent interface", 30-year-old WIN32 OLE binaries for
which the source code has been irretrievably lost will *still work*
and may still be used, in modern Windows desktops today. Mozilla
ripped out XPCOM because although it was "inspired" by COM, they
failed, during its initial design, to understand why Co-Classes exist.
as a result it caused massive ongoing problems for 3rd party java and
c++ users, due to binary incompatibility caused by changes to APIs on
major releases. Co-Classes were SPECIFICALLY designed to stop EXACTLY
that problem... and Mozilla failed to add it to XPCOM.
bottom line: the free software community has, through "hating" on
microsoft, rejected the very technology that made microsoft so
successful in the first place.
Microsoft used DCOM (and OLE), Apple (thanks to Steve's playtime /
break doing NeXT) developed Objective-C / Objective-J / Objective-M
(dynamic runtime self-describing capabilities *built-in to the
compilers*) and built the Cocoa framework around it. that both
efforts used runtime dynamic self-descriptive capabilities and that
both companies were a runaway dominant success is not a coincidence.
[3] d-bus. *shakes head*. when it first came out (about 15 years
ago?) i did a side-by-side review of the d-bus spec and the DCE/RPC
OpenGroup spec. incredibly, the wording and technical capability was
*NINETY* percent identical. except that d-bus was providing about 25%
of the *functionality* of DCE/RPC. as you can see from the
XPCOM-vs-COM example above, it's that missing functionality - the lack
of strategic foresight - that causes the GNU/Linux community to miss a
golden opportunity.
[4] remember the hard lesson of OpenMoko. on an ARM9 processor which,
when it context-switched it would throw out the entire L1 cache,
whereas Nokia was doing single-threaded applications and
heavily-optimised real-time OSes, the OpenMoko team designed
everything based around d-bus ("because it was shiny and new").
some interfaces did not even exist and were still being called.
on receipt of a phone call, the X11 "answer call" application took so
long to appear on-screen, and was so unresponsive thanks to d-bus
*and* X11 hammering the poor device into the ground, that even when
the buttons appeared appeared you still couldn't actually answer the
call. some 30 seconds after the call had gone to voicemail, the OS
would "recover".
Reply to: