Re: Second take at DEP17 - consensus call on /usr-merge matters
Hi Sam,
On Thu, Jul 06, 2023 at 01:51:04PM -0600, Sam Hartman wrote:
> BUT I don't think it matters.
> If we have a consensus we're unwilling to wait for a patch, it doesn't
> matter whether that's because:
Indeed, this is not how I looked at it. It also is a view that I don't
subscribe to, because I think commercial backing of the issue changes
the equation. So if I were to see changing dpkg as a viable way now (I
did earlier), I would be willing to wait for it, because we have
recently made significant progress on defining what those semantics
should be. While you try to remove the reasoning for this point of view
from the consensus, the "willing to wait" language implies a reason of
"too slow" already.
> 1) Some of us think the patch would be a bad idea
Additionally, people disagree on why it is a bad idea.
> 2) Some of us think the patch will not happen because of politics
>
> 3) Some of us think the patch won't happen because no one cares enough
> to write it
>
> 4) Some of us think the patch will eventually get done
>
> 5) Some of us think the problem is too constrained and if we really
> wanted to make progress we could incrementally move toward it.
We also have quite some disagreement on what "the patch" is in terms of
what semantics would help here.
> Helmut effectively asked us to agree with 1.
I disagree here. For reference, I am quoting the proposed consensus
item:
| The primary method for finalizing the /usr-merge transition is
| moving files to their canonical locations (i.e. from / to /usr)
| according to the dpkg fsys database (i.e. in data.tar of binary
| packages). dpkg is not augmented with a general mechanism
| supporting aliasing nor do we encode specific aliases into dpkg.
I carefully avoided adding reasoning to the proposed consensus as I was
seeing our disagreement on reasoning. I now see how that second sentence
could be read as precluding a dpkg patch in general. Would it help to
add "+For the purpose of finalizing the transition,+ dpkg is not
augmented ..." to leave open the possibility to add such a change but
still implying that we are not waiting for it to happen?
> And I don't think there is a consensus on this.
Even though I disagree on that's what I asked for, I agree that we don't
have consensus on patching dpkg being a bad idea in general.
So how do we get towards an agreeable consensus item? Evidently, the one
I proposed does not work out and the "willing to wait" variant you
proposed garners rough consensus, but not more. Would someone else be
able to propose wording that passes muster?
> ----------------------------------------
>
> I have reviewed the DEP 17 draft at
> https://subdivi.de/~helmut/dep17.html
>
>
> Helmut asked for consensus on the problems and mitigations or at least
> I think he did.
> I think we don't need that.
> I think we need consensus on decisions and confirmation that everyone
> feels heard.
Heh. I see how you get there. I agree with that latter part and tried
to use the agreement on problems and mitigations as a vehicle for
ensuring that everyone feels heard. Evidently, that does not work out
either.
In any case, the rough consensus on moving forward without major changes
to dpkg (for reasons we disagree about) paves the road for selecting a
subset of mitigations and proposing that as decision. The major missing
piece to perform this selection is an agreement on how we protect
aliasing symlinks from deletion (aka P9), because we still have quite
strong disagreement on the mitigations selected there.
> WRT the problems, I confirm that the list of problems does (in my
> reading) accurately describe the problems people have brought up.
Thank you!
> I don't think we have (or should try to get) a consensus on which
> problems need to be fixed except in so far as that affects our consensus
> on a proposal.
I was trying to imply that we need to address (more or less) all of
these nine problems. I say address rather than fix, because we may
choose to only fix them in certain environments and skip others (e.g.
derivatives, addon repositories, backports, skip upgrades etc.).
> I will admit that even though I've followed the discussion fairly
> closely, I don't have a good feeling about the mitigations.
>
> I think that once a reasonable set of the mitigations have been applied,
> we'll be in a reasonably good place.
>
> My concern is about upgrades and about unstable.
> I would like to see a set of instructions that I could follow for moving
> files in my packages in the data.tar to their canonical locations.
To me, that set of instructions is a later step, because those
instructions strongly depend on the selection of the mitigations and the
selection varies wildly with our disagreement of symlink protection. If
I were to present instructions now (one way or another), people would
rightfully disagree (different ones depending on the selection).
> I'd like instructions that clearly allowed me to reason about
> upgrades, and about how my packages interact with other packages during
> the transition.
I agree that this is where we want to be. This is not where we are.
> Because several of the problems look kind of serious, and my reading of
> the mitigations is that hey, by the time we get done, it'll all be okay.
> Implicit is the idea that because of the moratorium these problems are
> rare.
> Except during the transition, there will be no moratorium, so perhaps
> these problems will not be rare.
Exactly.
> There are two many mitigations, and the interactions matter for me to
> think about safety.
>
> Proposed way to address this concern:
>
> A) Develop a proposal assuming no dpkg changes for moving files too
> canonical locations in data.tar.
> Assume bootstrap protocol 3B (assume bootstrap creates the initial
> symlinks). I will talk about expanding to bootstrap protocol 3C in a bit.
>
> B) Develop an argument for the safety of that proposal.
>
> C) Let's review and agree to that.
>
> D) Then think about bootstrap protocol 3C; see below.
As much as I'd like to do that, D strongly influences A. Beyond that,
Doing D later removes the benefit of doing D as debbisect and debootsnap
will be eternally broken then. I am arguing that we cannot defer that
question for two main reasons. For one thing the selection of symlink
protection implies a bootstrap protocol and we do need a form of symlink
protection to facilitate upgrades. For another, doing 3B first and 3C
later removes much of the benefit of doing 3C in the first place.
> ----------------------------------------
>
> Regarding bootstrapping.
> I do not see an argument that 3C is safe though.
I am surprised that you see it this way. I asked Luca twice about the
involved risks doing a risk analysis of my own in
https://lists.debian.org/20230629123336.GA237611@subdivi.de and really
nothing substantial came back. Would you be able to quantify that
supposed unsafety in any way?
> Luca is right that if it breaks unstable or breaks upgrades, it will be
> really bad.
The problem with that point of view is that the cause for such breakage
is most probably not 3C, but the file move. So while that would be
really bad, it would happen without 3C as well. In order for this to
become an argument, we need to look at the additional risk incurred by
3C and not attribute the risk that we are taking anyway to it as an
argument against it.
As for shipping the symbolic links in base-files (or some other
package), I thought we had quite strong consensus:
* Simon Richter, https://lists.debian.org/669234b3-555b-4e2a-ccc7-dd5510b6e9c1@debian.org
* Luca Boccassi, https://lists.debian.org/CAMw=ZnQcKcuUaV-_k+NSrt2PuMss8Cmjp_nXog26Nd21hDgkmw@mail.gmail.com
* Johannes Schauer Marin Rodrigues (repeatedly)
* Russ Allbery, https://lists.debian.org/87edmigdtp.fsf@hope.eyrie.org
If you assume this to happen, implementing 3C becomes a matter of
changing debootstrap and nothing else. So we are looking for a case
where installing or upgrading the debootstrap binary package would break
unstable or upgrades. It is implausible that you see this is a major
risk.
Therefore, my premise of us agreeing on shipping the symlinks in
base-files likely is wrong despite the number of vocal proponents.
> So I think that to argue for 3C you need a specific proposal about what
> happens.
https://lists.debian.org/20230517093036.GA4104525@subdivi.de
> And you need a really strong argument that implementing that proposal is
> safe for upgrades and safe during the transition.
I am sorry, but I fail to come up with that strong argument, because I
fail to understand the supposed unsafety despite having asked for
clarification several times. I do not see how we can move this forward,
because there no longer is anything technical about it and it has become
one belief versus another.
> It sounds like part of the argument of safety during transitions is that
> you will coordinate with ftpmaster to do some essential packages in
> lock-step.
I now guess that you have a misconception here. There is an aspect of
unsafety about 3C that will be helped by uploading essential packages in
lock-step. That aspect is not upgrades however. That aspect is being
able to bootstrap unstable. So from the point in time where we start
uploading those essential packages until the point in time where the
last of it is built, bootstrapping will fail. So that safety issue is
not about upgrades.
> If you can show why that can be implemented and is safe, that's fine.
Thus far, my impression was that temporarily (<1week, preferably <1day)
breaking the ability to debootstrap was an acceptable risk and is
something we experience every now and then anyway (with adduser most
recently). For instance, Luca Boccassi explicitly mentioned this as an
acceptable risk earlier.
> But because we have another option (3B) that is viable, the safety
> argument needs to meet a high bar. If people like Luca are saying they
> are not convinced after reading it, even if they cannot articulate
> specific concerns, I would consider that blocking.
Well, we have this both ways, right? Other people are concerned about 3B
breaking long-term usability and we have another option (3C) that is
viable. The major difference here is that those other people can
articulate their concerns. I am unsure why we would not block 3B in the
same way.
> But for the safety argument for bootstrapping 3C, having someone who is
> familiar with the issues, has read the safety argument, and says they
> have a bad feeling should be blocking.
In essence, you are saying that this comes down to trust. We tried
having that safety argument, but nothing substantial came from it. We
evidently disagree about whom to trust here.
> My current reading of bootstrap consensus is:
>
> * I think everyone now agrees 3B is viable. Helmut and Josh explain why
> they do not prefer that option.
Yes.
> * Several people have been convinced by Josh's message that 3C is a nice
> to have.
Yes.
> * Luca thinks 3C is too risky to support. I think he's close to in the
> rough on going that far.
I note that Luca could not articulate the supposed risk after having
been asked twice about it.
> * I agree with Luca that 3C has risk and complexity, but am willing to
> consider a safety argument.
I strongly suggest that you analyse that risk on your own. You may use
the mail I referenced earlier as a starting point.
Helmut
Reply to: