Re: Debian and ISO C++ package management conversations
On Sat, Nov 6, 2021 at 3:52 PM Thaddeus H. Black <thb@debian.org> wrote:
> Not an expert on the subject, I should be reluctant to reply on
> list, except that 24 hours have passed and I see no other
> replies on list yet; so here goes.
Thanks, your comments are appreciated.
> I did not know that to make modules behave like headers were to
> accept defeat! Is it?
[...]
> what would
> the drawback be in letting modules behave like headers at least
> for now, and then revisiting the question for C++26 or C++29 in
> view of accumulated experience?
The main concerns being raised are:
1. Forcing file system semantics on module names, when module names
are not meant to be hierarchical. The "." is just a word separator, it
doesn't imply hierarchy.
2. Impact of having to traverse the file system when trying to get to
module files (although I'm personally skeptical on that front).
3. Potential issues caused by case-insensitive file systems as well
as on the mapping from unicode identifiers to file names.
But perhaps more importantly, if we had support at a higher level, the
package manager would be able to communicate to the build system the
complete mapping from module names to interface files, and no lookup
of any kind would be necessary. That's the context where making
modules behave like headers is "admitting defeat".
> Another way to ask the question is: wouldn't you rather
> implement something like your paper [2] for now, and then
> gradually expand upon that later to move cautiously in the
> direction of paper [1]?
> Or am I missing the point?
The main thing that may be causing confusion is that I seem to be
presenting contradictory views here. Let me explain.
I presented both papers 1 and 2.
The point of the first paper was to raise the issue that the current
implementation of modules in Clang and g++ would *not* work for
environments like the Debian GNU/Linux distribution, for instance.
It's written from Bloomberg's perspective since that's where we're
coming from, but we actually have pretty much the same requirements as
Debian has.
The second paper presents a very simple implementation that addresses
most of the issues raised by the first paper. In other words: I,
personally, would be fine if we commit to the "behave like headers"
approach.
However, there has been some opposition to that approach, and I took
it to myself to do an honest evaluation of what it would take to do a
higher-level solution, where the package manager would be able to
communicate to the build system what is the mapping between module
names to interface files without any expectation of how the files are
named. That's what the document linked from the other thread tries to
address.
> Except for the part on discovery tooling, which admittedly
> looks to me as though it might be slightly overengineered
> for C++23, paper [2] looks pretty good to me at first glance.
The reason why it needs to be a bit more over-engineered is that
parsing modules must be done in topological-sort order of
dependencies. Meaning you need to produce a full dependency graph of
all modules, and make sure you process the module interfaces in
dependency order. Only after the module interfaces are processed can
you proceed to the normal "embarrassingly parallel" step of producing
the object files. And the dependency discovery needs to be done by
reading the module interface files with preprocessing capabilities.
> what would happen to the preprocessor? Admittedly, I have not
> directly used the preprocessor to do anything interesting since
> the C++98/03 era, but the thought of eliminating the
> preprocessor entirely seems unsettlingly un-C++-like to me.
> Doesn't it, to you, too?
The preprocessor doesn't really go away with this. Modules actually
raise another problem that you need not one set of preprocessor flags
for your translation unit, but you also need to know the preprocessor
flags required for every module interface you consume.
But the point of modules, to a large extent, is to offer a
non-preprocessor way of reusing code. And at that, the spec is very
successful.
> Meanwhile, it occurs to me that, in Debian, C++20 has only
> reached sid a few months ago;
Ok, that is a point that bears clarification. Modules are a feature in
C++20. It's already part of the language, and there are already usages
of modules in some specific organizations. However, we don't yet have
any established convention on how anyone would be able to ship a C++
library that exports modules.
In fact, nothing that we're discussing in SG15 will actually change
the spec of the language. What we're trying to do is create guidelines
for interoperability between compilers, build systems, package
managers, organizations shipping pre-built libraries, organizations
consuming pre-built libraries.
In that sense, we're actually very much behind, because Modules are
not actually expected to change in the language in any meaningful way,
but the tooling is nowhere near where it needs to be for modules to
actually be useful.
The worst case scenario is that C++ libraries that export modules will
not be interoperable across build systems, and you'll end up in a
situation where we can't meaningfully ship C++ libraries pre-built.
Daniel
Reply to: