A hack to alleviate transitions in Britney; now what?
Hello,
this mail is to talk a bit about the current situation regarding
transitions in unstable. In my opinion, it is unfortunate that the
Release Team has had to insist on semi-serializing them, because that’s
not the kind of development you want to have in unstable right after a
release.
Executive summary
=================
We’ll manage to get Britney to be less of a PITA when dealing with
transitions, which will hopefully enable us to do more of them
concurrently if that’s what we want.
This will be done by letting the old libraries stay in testing for a
while after introducing the new one.
Please still wait for an ACK from the Release Team before starting a
transition. We’ll be handing them out more quickly now, but it’s very
important that somebody checks whether reverse dependencies will rebuild
fine or need patches, as to minimize the time core build infrastructure
remains broken.
Please find below the longer version.
Part 1: explanations
====================
Let me start by explaining why it has been that way, the semi-serializing
of transitions (you can skip the next two paragraphs if you already know
how britney handles transitions). When a library package bumps its
SONAME, it normally does so by creating a new package and dropping the
old one from the same source package. When britney tries to migrate this
change to testing, it wants to migrate at the same time both the
creation of the new package, and the removal of the old one. But removal
of the old one means rendering packages uninstallable (those depending
on the old name), so this effectively means that all packages that
depend on the old library in testing must get a new version of
themselves migrated at the same time that the library itself migrates
(they can’t migrate one at a time, because the new library would not be
in testing yet). For obvious reasons it’s a bit complicated to get all
packages in shape at the same time, but it’s certainly doable.
The problem is when you have several transitions open, because they are
bound to get tied via packages that depend on two or more of the
disappeared libraries, which means that to update package P for
transition T, you must do transition T2 at the same time, which brings
in a whole lot of more packages to migrate at the same time. And that’s
what increases the complexity of things, the ties between transitions.
I’ve been thinking what to do about this, because I sincerely hate to be
the guy that is blocking people from doing the work they’ve been waiting
for so long already. You must believe me when I say I’m very unhappy
about doing that. Should it have occurred to me earlier, I would
probably have gone for making squeeze “hidden” from the public (i.e., no
Packages.gz files), so that unstable could get very entangled and slowly
converge. The thing is that a hidden testing allows you to break it, and
a testing with permission to get broken converges much more quickly than
a testing that must remain installable.
But it’s now too late for that, and has a lot of disadvantages of its
own, so I’ve revisited a way of doing transitions that has always been
in the mind of the Release Team, but that britney does not support out
of the box: allowing the old library package to stay in testing for a
while. This means that not all packages have to migrate at once, because
the ones that do not migrate won’t be uninstallable, since the old
library stays.
Now, this has its own set of problems and caveats as well, since if you
don’t pay attention and take care of later cleanup, you end up with
packages in testing that do not belong to any source in testing, which
is bad. These partial migrations can also be problematic (AFAIK) if eg.
a recompiled library against the new transition T migrates, and a binary
depending on it and the old library of T doesn’t, whereas both the old T
and the new T will get loaded into the process namespace.
These concerns notwithstanding, I’ve decided to start using this method
as a common procedure this early in the release cycle, but only as a
means to break *ties* between transitions, and some hand-picked
exceptions. I don’t want to systematically (ab)use it, because it’d be
hard to clean afterwards.
The consequence is that individual transitions will take approximately
the same time to get ready themselves, but that once that happens it’ll
be possible to migrate them without having to wait on some other
transitions.
I’ve already done that with libdvdread, which was tied to libraw1394 and
ffmpeg via vlc and a couple others. This has been relieving, because
having a transition ready in unstable but blocked for migration means
that it will inevitably go back to “unready” after a while of waiting.
As hinted above, britney does not support this procedure out of the box,
and it has to be done with a gross hack and manual poking, but I’m
willing to do that because I think it makes things significantly better.
Additionally, britney2 does support it, and I’m hoping that we can start
using that codebase soon.
Part 2: opportunity for feedback
================================
With this procedure in place, it is now possible to hand out transition
tickets at a faster pace, and I intend to do that. This means unstable
is probably going to see more breakage, but at the end it’s going to
allow us to get the first post-release set of transitions done faster,
which I believe is important.
If somebody has good reasons why it shouldn’t be that way, or
consequences about the procedure explained above that I haven’t
foreseen, please mention them now.
However, please do not take this mail as a call for starting any pending
transtions at once. We still need to hand out tickets, but we’ll be
handing them more quickly. It is, for example, very important to
determine if the rebuilds of reverse dependencies will succeed, or if
contrary source changes are needed for them, and whether patches exist
or not. It’s one thing for a transition to take its time to migrate, but
it’s another to effectively prevent other developers from doing their
work, as it happened with poppler/texlive.
Finally, I also want to thank all the developers that have shown their
willingness to cooperate with the Release Team and wait their turn in
the transition queue, whilst seeing other developers get theirs started
by not coordinating with us. Thanks!!, it’s much appreciated.
Cheers,
--
- Are you sure we're good?
- Always.
-- Rory and Lorelai
Reply to: