[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#1050001: marked as done (migrate from merged-/usr to new alternative filesystem layout)

Your message dated Fri, 22 Sep 2023 23:42:25 +0200
with message-id <20230922214225.gplwhmdua7jrbfbt@mail.gaussglocke.de>
and subject line migrate from merged-/usr to new alternative filesystem layout
has caused the Debian Bug report #1050001,
regarding migrate from merged-/usr to new alternative filesystem layout
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
1050001: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1050001
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: tech-ctte


Background and current status

In 2019 the TC decided[1] that Debian would achieve the largely-agreed
goal of having only one place to put most files, /usr, by using
symlinks to alias from /bin etc. to e.g. /usr/bin.

At the time, concerns were raised that package management systems and
other software would malfunction but the set of malfunctions was not
enumerated; proponents of the aliasing approach characterised them as
FUD.  In the absence of the results of the research work which has now
been done, that characterisation seems to have been implicitly
accepted as true by the then TC.

Thanks to work funded by Freexian we now have a list of many of these
malfunctions[2] (although new ones keep popping up (eg [3]).

The most serious malfunction is the disappearing files bug, which has
prompted a seriously inconvenient file move moratoriam which has now
been in place for many years.  After 4 years, we still don't have a
clear path forward to resolving that and other problems [4].

It should now be clear to most observers that the decision to go down
this path was a mistake.[5]


Fixing the mistake

I think we can probably fix it by backing out this change, and then
doing usrmerge the traditional Debian way by making changes to
debhelper, so that we move the files package by package, in the .debs.

But given the atmosphere, such a proposal would need clear political
backing from the TC.  It wouldn't be worth anyone's time or emotional
energy to attempt to make a concrete and detailed plan if the TC is
not minded to consider it.

So I would like to pose the following questions for the TC:

 * Given the information that the Committee has now, that it didn't
   have in 2019, does the Commitee agree that the decision in 2019
   was questionable ?

 * Is the Committee open to the idea of a plan being developed which
   reverses the mistake, rather than merely "mitigating" the problems ?
   Would such a plan be considered on its merits ?

I appreciate that I'm asking the TC to revisit a previous and
controversial decision.  That this isn't a step we should take
lightly, but I think in this case it's clearly warranted.


Timeline for a fix

Any plan to solve this would probably take a few releases (ie, many
more years) to sort out, sadly.  We would probably need to wait with
shipping packages that move files in a conventional-for-Debian way,
until we have our userbase's systems restored to a non-aliased state.
So I think we would need trixie to undo the aliasing, and in trixie+1
we could move the files.

This delay is unfortunate, but - unlike pressing forward - it has
relatively low levels of risk, most of which is front-loaded.  I think
we can develop tools which will reliably de-alias a system; and, once
users' systems have been de-aliased, the actual file movement is
routine work that Debian knows how to do.

I can see that there could possibly be ways of going straight to the
desired state (un-aliased systems but with nothing much in /), but I
haven't given them much thought them through because they seem risky
to me and involve some grievous hacks.


Protecting my mental health

I will try to avoid regularly reading this thread.  I hope that now
that I have made the suggestion, others will be able to carry the
conversation.  I will be configuring my mail client to disregard my
personal copies of messages sent to this bug; when I want to read
the thread I will look at the mailing list.

Also, if you disagree with my decision to raise this now, please don't
email me.  If you feel I have overstepped a boundary, please contact
the Community Team or DAM.

If the Comittee gives a positive indication, I will be happy to
re-engage at the level of technical work to try to make it happen.


Thanks,
Ian.

[1] https://lists.debian.org/debian-devel-announce/2019/03/msg00001.html
[2] https://subdivi.de/~helmut/dep17.html
[3] https://lists.debian.org/debian-devel/2023/05/msg00311.html
[4] https://lists.debian.org/debian-devel/2023/06/msg00353.html

[5]
  Perhaps merged-usr via directory aliasing worked well in some other
  distros with less sophisticated package management systems.

  But we obtained almost all the practical benefits of abolishing the
  distinction between / and /usr very early, by deciding that the
  initramfs would mount /usr too.  We have inflicted all this pain and
  confusion on ourselves simply to do some tidying up.  The result has
  been the opposite.

  If we had just moved the files rather than trying to rush things
  with the directory alias symlinks, we would have been finished by
  now.

--- End Message ---
--- Begin Message --- 
Dear Ian,
thank you for reaching out to the Technical Committee and engaging in a dialogue despite the very flammable nature of the topic.
We believe the major points of your argument are as follows: Directory symlinks ("aliasing symlinks") are inherently less robust than file symlinks. Aliasing symlinks violate a key assumption of dpkg, namely that every managed filesystem object can be uniquely determined by its path name. Given the pervasive nature of this assumption in the dpkg code base, aliasing symlinks will remain a persistent source of difficult-to-track bugs, which will negatively impact the robustness of dpkg and Debian as a whole. Given that only a small fraction of binaries actually need to be reachable through both / and /usr, the desired level of compatibility with other distributions can also be achieved by using a carefully curated list of file-to-file symlinks.
Therefore, you have asked the TC if it has changed its stance on the 2019 decision and whether or not it would be open to alternative technical solutions.
The TC recognizes your standing as subject matter expert on dpkg and agrees that aliasing symlinks expose a limitation of dpkg, which needs to be dealt with. However, the TC disagrees with your risk assessment and the conclusions you draw from DEP-17. Given the scope of your proposed change, the current transition state, and the general fatigue that has set in with regard to merged-/usr discussions, we find it justified to set a high standard of proof before we impose more change on the project. The presented arguments were not strong enough to let us conclude that continuing on the present course will result in a much worse bug situation. Formally, your questions do not propose any immediately actionable steps either; on those grounds, the TC declines to rule, which effectively upholds previous rulings on the matter.
Informally, I believe it is fair to say that we have gained a significant understanding in the past few months on how to complete the merged-/usr transition. While DEP-17 does enumerate several issues, none of them are show-stoppers; they can be mitigated with temporary workarounds for the trixie release and do not create a permanent maintenance burden. Many TC members currently believe that the benefits of a filesystem layout with aliasing symlinks outweigh its drawbacks. Therefore, it is very unlikely that the TC will overturn the current transition plan, let alone revert to the old split-/usr layout, unless confronted with concrete and compelling evidence that the plan results in irrecoverable breakage. Any remaining pain points, if they persist beyond the trixie release and cannot be resolved by tweaking the transition itself, should rather be addressed under the assumption that the forky release cycle begins in a post DEP-17 world.
You have probably anticipated this response (and maybe hoped for a different one). Although the TC may disagree with your conclusions, we do appreciate your technical insight and are very thankful for your continued constructive participation in the discussion. Personally, I think this is a situation where the Perfect is the enemy of the Good. I hope that despite our disagreements, you will continue to help turning Good into Better. 


Cheers
Timo
for the Technical Committee


--
⢀⣴⠾⠻⢶⣦⠀   ╭────────────────────────────────────────────────────╮
⣾⠁⢠⠒⠀⣿⡁   │ Timo Röhling                                       │
⢿⡄⠘⠷⠚⠋⠀   │ 9B03 EBB9 8300 DF97 C2B1  23BF CC8C 6BDD 1403 F4CA │
⠈⠳⣄⠀⠀⠀⠀   ╰────────────────────────────────────────────────────╯

Attachment: signature.asc
Description: PGP signature


--- End Message ---
Reply to: