Bug#983087: sbuild-createchroot misses usr/libexec/qemu-binfmt/ directory
On 25 Feb 2021, at 12:00, Christoph Biedl <debian.axhn@manchmal.in-ulm.de> wrote:
>
> Roger Leigh wrote...
>
>> I was having a think about this last night. To be completely
>> realistic, schroot maintenance is very low on my list of my
>> priorities. Work on it is sporadic at best. My interest in it is
>> also fairly low. I’ve moved on to other things.
>
> That's sad to hear.
Well, yes and no. Yes, in that I was quite proud of the impact and utility this small tool had, and didn’t want to let it go. But no, in that everything has its time, and schroot was obsoleted quite a few years ago now. It’s time to move on.
schroot was designed as a single-purpose tool. It’s original name was “sbuild-chroot-helper” (-> “schroot”). Its purpose was virtualising build environments solely for sbuild’s benefit, and it did that fairly well. The only reason it was separated was because Perl scripts can’t be setuid root. But the simple fact here is this: Docker exists. It’s better, it’s more flexible, and it’s backed by funding and resources I don’t have. I did consider making schroot a daemon with a socket interface and doing a number of other changes to bring it up to a level with what Docker offers. But, again to be quite realistic, I don’t have the time, the resources or the desire to expend that effort when there’s a much simpler option: switch to Docker. systemd has also broken some of the core assumptions we rely on through use of basic standard POSIX interfaces. That also requires time and effort I don’t have to work around. schroot will continue to be less usable on a systemd-based system.
The other uses for schroot were implemented as niceties after the fact. They were never part of the core requirements. Some were heavily used at various points, for example running 32-bit code on 64-bit systems, but now we’re all migrated to 64-bit systems that’s much less common.
Don’t get me wrong. I don’t actually like Docker particularly. But it’s here, it does the job, and schroot can’t match its functionality. schroot users should migrate to Docker where possible.
>> I don’t think that something which is not maintained or well supported
>> should be used as a critical part of the Debian build infrastructure,
>> and honestly, | think you should take steps to remove it entirely. It
>> was a nice tool, it served its purpose, but it is dated and limited,
>> and has been completely eclipsed by Docker. If you can switch over to
>> using Docker, I think you should make an active effort to do so.
>
> And those who cannot or don't want to because of the consequences?
When I integrated schroot support into sbuild I created an abstraction to leave the original (privileged) chroot support in place. Adding a third implementation of this interface to enable docker support would be relatively simple. Just a few dozen lines of Perl to tell it how to create, run commands in, and tear down a container.
> The highly convenient feature of schroot are simplicity, maturity and
> therefore reliability in its usage. So it's a tool that "just works",
> something people value once they've grown into a greybeard and/or want
> to be sure the workflow around a tool doesn't break every now and then.
> I can live with some shortcomings otherwise, or might even try to work
> on them.
I sent this email as a courtesy simply to explain what the situation is, similarly to when I left the Debian project six years ago. You are of course welcome to continue to use the tool. But when it comes to being a part of Debian, that’s a different matter, and I am encouraging you all to think about and expedite its removal. Having unmaintained and abandoned tools hang around past their sell-by date will cause problems, and schroot hasn’t been maintained actively for at least that long. It would be for the best if you removed it.
I do not have any written or unwritten obligations to anyone when it comes to free software maintenance. My time is limited, and maintaining a niche tool is very costly, particularly when I derive zero benefit from it myself, and it has been obsoleted by other tools. It’s time to retire it completely.
> Also, there are more usecases than just package building: I use schroot
> to manage interactive chroots where I can check the behaviour of other
> distributions and (to some extent) other architectures.
You can do most of that in Docker as well.
The only thing which is unique is the support for non-native architectures with qemu-user. But that was just discovered as a side-effect of the command-prefix feature. It worked, but it was never a requirement or designed-in feature. You can always use qemu-user via binfmt-misc. I don’t think this one super-niche aspect of the tool is a reason to keep it. There are alternatives.
>> Now that you have GitLab on salsa, how much work would it be to:
>
> There is life outside salsa, and gitlab itself really is a textbook
> example of how software should not be. Huge and complex, and certainly
> not easy to set up, client/server, bad content presentation for limited
> clients, yada yada. While I just want to maintain a few packages that
> are not part of Debian for various reasons.
I didn’t write this email to address your *personal* needs. I wrote it more for consideration for *Debian’s infrastructure*, and I’d like you to think about it seriously from that point of view.
Back in the mid-to-late ‘90s, when buildd and sbuild were written, there was no other choice. No software existed to meet Debian’s needs, and so we wrote our own services, our own databases, and our own tools to support building, archiving and distributing our packages, mainly driven by the m68k porters. These tools are old, custom and somewhat fragile. They also require continual hand-holding and ongoing maintenance. They are complex and opaque, written in ancient Perl, and are difficult to maintain. The pool of capable maintainers is small and declining. This is costly in multiple ways. I didn’t account for the hours I spent on these tools over the years, but it’s going to be many *thousands* of hours. We need to be realistic about sbuild and buildd, too. They aren’t maintained. Other than myself, and very occasional contributions from a small number of others, they are essentially unmaintained since the early 2000s.
If you were to redo the infrastructure from scratch, in 2021, you would find that there is no need to write these tools *at all*. We have many existing software projects to choose from which do everything required.
buildd is a “simple" job scheduler and package uploader. sbuild is “just" a wrapper around “apt-get build-dep” and “dpkg-buildpackage”. Today, we have several really high quality tools to choose from. There’s Jenkins, which is used the world-over to automate stuff. And there’s GitLab CI, again used by hundreds of thousands of projects. And several others.
Think about how much work it would be to replace sbuild. It’s literally a docker container, a .gitlab-ci.yml totalling a few dozen lines, and a dozen-line shell script. That’s it. And buildd? It’s literally a single GitLab project or Jenkins job and a webhook. You could replace the whole buildd/sbuild/schroot setup for the *entire* Debian project in a few hours of effort. And it would be a vast improvement!
Think about it.
Consider also the accessibility and maintainability this would bring. Using GitLab CI or Jenkins makes all of the job scheduling, build logs, build artefacts, and the logic controlling it all visible to all. Any developer (or general user) could browse the lot, open merge requests to update the configuration etc.. Both systems have support for secure “secrets” to control use of signing keys. And it means that you no longer need dedicated “buildd admins” to manage all of the creaking infrastructure. Just admin to set up the runners. Yes, these are “huge and complex” beasts. But don’t underestimate the value that they bring. They bring *a lot* of value. And they don’t require active and ongoing maintenance effort in the way all our *custom* infrastructure does. We already have GitLab deployed. Why not make serious use of it? That’s it’s job.
I’m no longer involved, so take or leave the suggestion as you will. But just think about what schroot is. It’s a support tool, written for a support tool (sbuild), which is a wrapper for another support tool (dpkg-buildpackage). The complexity here is both staggering and unnecessary. Now there are widely used generic alternatives which could be used, you should think about using them instead. Debian has a 25-year legacy and “sunk cost” in these tools. But the cost of replacing them is much less than you might think. It can’t be understated just how much the barrier has been lowered by tools like GitLab and Jenkins.
Regards,
Roger
Reply to: