Rust policy for bcachefs-tools in Debian
Hi all, we're looking at getting bcachefs-tools back into Debian (we
just re-opened an ITP). Was just speaking with you guys on IRC, where
the possibility of a team meeting was raised, and I was directed to this
list.
(Thank you capitol; when the previous efforts were happening I had no
luck with getting connected to the right people, fingers crossed this
time :).
Background: bcachefs-tools is a mixed C/Rust package, and the dependency
unbundling requirements are particularly problematic for us. I don't
think it's realistically possible to ship and support without at least a
partial exemption.
Rational:
- It's a critical system package (bugs in this package will cause your
system to fail to boot), so the testability/maintainability issues
that unbundling causes have a much bigger impact. As far as I know,
bcachefs-tools is the first package with this level of criticality to
make use of Rust.
- To make matters worse, it's mixed C/Rust; that means we have lots of
FFI, and that's a particularly sensitive area (very high potential for
bugs that won't be caught by the compiler compared to typical Rust
code).
We've had some painful experiences with these sorts of issues, which I'd
prefer not to repeat. Our mount helper is written entirely in Rust
(unlike btrfs, bcachefs does multi-device discovery in userspace, so
this is required for the system to function), and with the original
bcachefs-tools Debian package an unfortunate chain of events led to a
lot of users unable to mount in degraded mode after a device died - it
was an FFI bug and the process issues meant that we weren't able to get
them a proper fix.
There's two specific reasons that unbundling is problematic:
- Swapping out dependencies means the entire package needs to be
retested. Retesting with the full test suite absolutely has to happen
for bindgen changes - like I mentioned, FFI is just too sensitive.
Other Rust dependencies are less sensitive, but given that (even in
Rust) semantic versioning is a promise with no actual verification,
we'd want those retested too. Consider the possibility of a clap
version change that subtly changes option parsing; it's easy to say
"that would never happen", but as a filesystem developer, I've seen
and had to debug lots of things that never should have happened :)
Given the already high cost of testing and the size of our test suite,
testing overhead is not something I want to add to. Cargo's locking of
dependency versions solves a lot of problems for us.
- The other issue that came up before is: for a critical system package,
we absolutely need to be able to get out hotfixes in a timely manner
(just like e.g. the kernel).
This was the major issue that led to bcachefs-tools being orphaned in
Debian the last time around, when conflict over dependency unbundling
led to us not being able to ship out a bugfix for mount option
parsing. (The bug was fixed upstream in a matter of days, and AFAIK no
one on other distributions was affected). Discussions and resolving
the issue got sidetracked by "well, it's experimental, it's fine if we
break this" - so I have to reiterate that it's upstream's policy that
supporting users and making sure they have a working filesystem comes
first. It's my policy that the experimental label is simply a warning
to users that they may run into issues that we haven't discovered yet,
we still strive to deliver the same level of support (triaging and
quickly responding to bug reports) that any other filesystem would,
and having our priorities in the right place from the start is
critical to shipping a bulletproof, reliable, trustworthy filesystem.
So, that's things from my end, and why I'm asking for an exception to
the unbundling requirements. I'd like to hear things from your side, and
see what we can do to address any issues this might cause on the Debian
side. I'm aware of concerns over security updates; Fedora folks also
raised build server overhead when we had this discussion with them. Like
I mentioned we never managed to raise this discussion to the appropriate
places in the past, so if there's other issues at play I'm looking to be
educated :)
Cheers,
Kent
Reply to: