Bug#1050001: Unwinding directory aliasing [and 3 more messages] [and 1 more messages]
On Wed, 30 Aug 2023 at 12:57, Ian Jackson
<ijackson@chiark.greenend.org.uk> wrote:
>
> On the burden of proof and the correctness of software:
>
> I'm afraid I see a pattern, where blanket statements are made which
> are only "mostly" or "roughly" or "generally" true. But the
> discrepancies and details matter. When we make computer systems, it's
> not good enough to if they're only "mostly" right.
>
> Every program is an unproven conjecture whose truth we end up relying
> on. We need simple axioms, and rigorously correct reasoning.
Yes, and fact-free and evidence-free threads such as #1050001 go in
the exact opposite direction.
> Aliased usrmerge, when deployed, was a conjecture that was disputed:
> "usrmerge via directory aliasing functions correctly".
And it demonstrably does: it works beautifully and flawlessly on an
uncountable number of Debian, Ubuntu, Mint, Fedora, RHEL, CentOS,
Arch, SUSE, Yocto, Gentoo, etc. installations, for a decade in some of
those cases.
> Helmut Grohne writes ("Bug#1050001: Unwinding directory aliasing [and 3 more messages]"):
> > In order to prefer Debian over something else, we want it to be similar
> > enough to make switching feasible while making it differ from others
> > enough to make switching worthwhile. Not having aliasing symlinks hardly
> > seems like an aspect that makes Debian more suitable to people. I guess
> > that you disagree with this and that is fine.
>
> Debian has historically been simply much more reliable. That
> reliability doesn't come from one particular decision. It comes from
> an aggregate of, on the whole, making better decisions. That
> reliability is very valuable to our users and downstreams. It's
> one of our our most compelling advantages.
It comes in the largest part from having reliable upstreams that can
use common, stable, reliable and proven interfaces, such as the
usr-merged filesystem layout which has been the de-facto standard
across the vast majority of Linux distros for the past decade.
> > My impression has always been that the disagreement on the end
> > state was involving a minority.
>
> Well, if you disagree with aliased usrmerge and say you don't want it,
> you get abuse. Even here, many abusive messages have been posted with
> impunity by persons with considerable status.
Sure, insofar as asking for evidence for outrageous and unproven
claims can be considered 'abuse'
> > > This argument is basically drawing together the common factor in the
> > > DEP-17 problem list.
> >
> > And that's precisely why I don't understand your argument. All of the
> > DEP-17 problems are supposed to go away. So it appears as if you cite a
> > list of problems that I expect to not be problems for much longer as a
> > reason for changing the end goal.
>
> DEP-17's list of *problems* forms a category: directory-aliasing-
> induced malfunctions. But the DEP-17 *solutions* are ad-hoc, and many
> are only applicable to to malfunctions that occur during migration.
No, it's a list of potential bugs caused by dpkg being hopelessly
broken, and effective and demonstrably correct temporary workarounds
for them.
> > > Affected tooling includes not just our .debs, but also remote
> > > configuration management systems like Ansible, image construction
> > > (Dockerfiles), and 3rd-party software installation progrmas (including
> > > both 3rd-party .debs and 3rd-party script-based installation systems).
> >
> > I don't follow the reasoning. [...]
>
> Sam has posted a helpful set of examples of things that one might do,
> whose consequences with aliased directories are unclear.
>
> These are of course only examples.
Yes, they are only examples, more specifically evidence-free examples.
It would be trivial to demonstrate that Ansible/Puppet/whatever are
broken and don't work on Debian/Ubuntu/Fedora/RHEL/CentOS/Arch/etc,
but somehow evidence remains scarce. I wonder why.
> > > I would be much more convinced that all of this isn't a problem, if
> > > there existed, and had been formally adopted (eg by existing in some
> > > manual somewhere) a short and clear set of rules about what is and
> > > isn't allowed - not just rules for us within Debian, but rules for
> > > everyone, everywhere, referring to and modifying files.
> >
> > It appears to me that the empty set of rules (outside packaging) is too
> > simple for you to consider.
>
> "Packaging" is doing a lot of work there. I find your comment rather
> tendentious, I'm afraid.
>
> I think we need a short and clear set of rules for what you can do
> with ansible, rsync, docker, etc. etc. etc., as well as just apt and
> dpkg.
Again, the rule is simple and short: vendor files go under /usr,
ignore legacy locations. Doesn't get any easier than that.
> > > > We already have the Debian Usr Merge Analysis Tool available at
> > > > https://salsa.debian.org/helmutg/dumat and its output at
> > > > https://subdivi.de/~helmut/dumat.yaml. As explained on d-d@l.d.o, I want
> > > > to turn those findings into automatic RC bugs. Does that alleviate your
> > > > concerns to some extent?
> > >
> > > That's certainly helpful for the transition now. Are we going to
> > > maintain this or something like it indefinitely ?
> >
> > I don't think so. Once packages have moved to /usr, we can simply have a
> > lintian-checkable policy of not shipping stuff in aliased paths. What's
> > the benefit of continuing this crazy approach then?
>
> Not just not *shipping*. All management operations (eg that inspect
> non-filesystem databases that mention files), even some read-only
> ones, must use physical paths. So we need lintian *at least* to
> search maintainer scripts etc. (And see also my ideological comments
> about privileging Debian.)
No, we don't, it's just dpkg that is broken. Of course you are free to
provide evidence to the contrary at any time.
> > > But, a goal of the directory aliasing is to be compatible with other
> > > systems, so that 3rd party software is *allowed* to refer to, and,
> > > presumably, ship files in /bin (because "there's no difference now,
> > > right?").
> >
> > If they install without dpkg, yes.
>
> Many upstreams make a .deb by running their ad-hoc installation shell
> script (which in this brave new world will talk about /bin, probably)
> and stuffing the result in a .deb.
>
> What engeineered safeguard are we planning to prevent the
> installation of usrmerge-broken .debs on end user systems ?
> I think the current answer is "we hope ISVs will run lintian" ?
The same that we provide against third party packages linking against
the wrong libraries, or doing rm -rf / in a maintainer script, or any
other broken and unsupported combination: none whatsover. We have a
set of documented interfaces that cover a myriad of aspects and
facets, and those who step outside of them get to keep the pieces.
And yet nobody reported any issue in the past 6 years or so that this
has been a thing on Ubuntu and Debian, which means they'll be ok
regardless, because the chances of a third party shipping in the
legacy hierarchy is tiny, and the chances of a third party moving that
across at the same time as a moving across yet another third party
package are so small there are millions of more important things to
worry about.
> > > Our downstreams (of all kinds) are are more likely to use other
> > > tooling (of all kinds).
> >
> > Tooling that does not have some kind of inventory of the filesystem is
> > mostly unaffected. Given the wide adoption of the aliasing approach, I
> > think it is fair to ask for evidence at this time.
>
> Tooling that *uses* an inventory of the filesystem can be affected,
> even if it doesn't modify that database. So "has" isn't right. It's
> not tolls that *have* such a database. It's any program that
> exchanges filenames with a tool that uses such a database. Also, note
> "mostly".
Nah, it's just dpkg. Otherwise: evidence please.
> > And if we actually try to have a symlink flower patch rather than a
> > symlink farm, we are left with the pain of where things live differing
> > between distributions and releases in a distribution.
>
> I think it's worth pointing out that any software which is trying to
> be portable to Unix systems other than just Linux (which includes the
> BSDs and MacOS) will need to avoid assuming directory aliasing.
Which is a dwindling minority, and for good reasons. Not only tons of
upstream software these days doesn't want to be constrained to what
some commercial unix derivatives were doing in the 1980s, and want to
use the full power of Linux instead, but even Debian recently dropped
two of the three non-Linux ports.
> > On the other hand, symlinks have been around throughout the lifetime of
> > Linux, and I think sysadmins do have a feel for how they work.
>
> Directory symlinks have always been troublesome. Sometimes they're
> the best answer, but IMO not here.
The vast majority of the rest of the Linux world disagrees with that
opinion as a matter of fact. But I'm sure that they will all change
their minds as soon as they get a chance to peek at the mountain of
incontrovertible evidence that has been provided so far in this thread
by the proponents of symlinks-farm.
Reply to: