Bug#932795: Ethics of FTBFS bug reporting
On Tue, 23 Jul 2019 at 13:54:10 +0200, Santiago Vila wrote:
> Ethics of FTBFS bug reporting
I don't think framing this as a question of ethics is necessarily
helpful. When people disagree on a technical question, a recurring
problem is that both "sides" end up increasingly defensive, arguing
from an entrenched position, and unwilling to be persuaded. Using terms
that the other "side" is likely to interpret as an accusation of being
unethical seems likely to exacerbate this.
> I reported this bug:
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=907829
>
> and it was downgraded on the basis that the official autobuilders
> are multi-core.
Do I understand correctly that you are asking the TC to exercise our
power to overrule developers, in order to overrule the maintainer's
and/or the release team's judgement about the severity of (bugs like)
#907829?
Or are you asking the TC for advice, or are you asking us to use a
different one of the TC's powers?
> * The informal guideline which is being used, "FTBFS are serious if
> and only if they happen on buildd.debian.org", is not written anywhere
> and it's contradictory with Debian Policy, which says "it must be
> possible to build the package when build-essential and the
> build-dependencies are installed".
I had always interpreted the informal guideline as: FTBFS on the official
buildds of release architectures are always serious, because they mean we
can't release the package; FTFBS anywhere else (non-release architectures,
unofficial buildds, pbuilder, etc.) *might* be serious, but might not,
depending on how "reasonable" the build environment is.
There are many aspects of a build environment that might be considered
reasonable and might not, and they are generally evaluated on a
case-by-case basis. A working build environment needs "enough" RAM (a
lot more for gcc or WebKit than for hello); it needs "enough" disk space
(likewise); it needs a writeable /tmp; it needs a correctly-installed
Debian toolchain (I hope you wouldn't argue that it's RC if a package
FTFBS with a patched gcc in /usr/local/bin); it needs to be on a
case-sensitive filesystem (I hope you wouldn't argue that FTBFS on
FAT/NTFS/SMB would be release-critical); it needs to not have weird
LD_PRELOAD hacks subverting its expectations; and so on.
We also have packages that FTBFS (usually as a result of test failures)
when built as uid 0, when built as gid 0, when built with not enough
CPUs, when built with too many CPUs (a lot of race conditions become more
obvious with make -j32), when built in a time zone 13 hours away from UTC,
when built on filesystems that don't provide the FIEMAP ioctl, when built
on filesystems that don't have sub-second timestamp resolution, and many
other failure modes. Clearly these are bugs. However, not all bugs are
equally serious. For example, we've managed to release the glib2.0 package
for years, despite it failing to build when you're uid 0 (a test fails
because it doesn't expect to be able to exercise CAP_DAC_ADMIN), because
we consider building as uid 0 to be at least somewhat unreasonable.
If build-time test failures are always RC, however usual or unusual the
build environment, then one rational response would be for all maintainers
to disable build-time tests (if they are particularly conscientious,
they might open a wishlist bug, "should run tests", at the same time
as closing a RC bug like "FTBFS due to test failure when gid == 0"). I
don't think that is a desirable outcome. We are not building packages for
the sake of building them, but so that they can be used - which means we
should welcome efforts like build-time tests that improve our confidence
that the package is actually usable in practice, and not just buildable,
and try to avoid creating incentives to remove them.
For the specific question of whether a single CPU core is a "reasonable"
build environment, my answer at the moment is "I don't know".
> * Because this is a violation of a Policy "must" directive, I consider
> the downgrade to be a tricky way to modify Debian Policy without
> following the usual Policy decision-making procedure.
The wording of the serious severity is that it is a "severe" violation
of Debian Policy, which is qualified with "(*roughly*, it violates a
"must" or "required" directive)" (my emphasis). This suggests that there
can exist Policy "must" violations that are not RC.
The release team are the authority on what is and isn't RC: the fact that
serious bugs are normally RC is merely convention. However, I suspect
that the release team would not welcome being asked to add -ignore
tags to serious bugs that describe non-severe Policy "must" violations,
and would ask the package's maintainer to downgrade the bug instead.
> To illustrate why I think this guideline can't be universal, let's
> consider the case (as a "thought experiment") where we have a package
> which builds ok with "dpkg-buildpackage -A" and "dpkg-buildpackage -B"
> but FTBFS when built with plain "dpkg-buildpackage".
>
> Are we truely and honestly saying this package would not deserve a
> serious bug in the BTS just because it builds ok in the buildds?
In this case I think I'd open the bug as serious, but if the maintainer
downgraded it with the justification that a straightforward workaround
exists, I think that would be reasonable. For the end user to be able
to exercise their software freedom, they need to be able to build the
package *somehow*, but I don't think that can imply that every possible
build configuration must be supported - there are innumerable ways to
construct an unreasonable build environment, and we can't support them
all. We have no shortage of bugs already!
> Whenever someone dares to report a bug like this as serious, following
> both Debian Policy and Release Policy (or at least the letter of it),
> we lambast them, we make mock of their building environment, we call
> them a fool, and we quote informal guidelines which are not written
> anywhere.
Developers should not lambast bug reporters or call them fools, but I
think we have to have a sense of priorities. A package failing to build
under some circumstances is not *necessarily* a serious Policy violation:
if it was, every FTBFS on the reproducible-builds infrastructure would
be RC, including those that only happen because the reproducible-builds
infrastructure is specifically trying to cause odd situations.
If a maintainer has a couple of hours to spend on Debian, then I think
it would often be of more benefit to our users and Free Software to
spend that time fixing an important, normal or even wishlist bug (for
example "application crashes when I do foo" or "no way to do foo using
public API") than to spend it fixing a FTBFS that only occurs in unusual
circumstances. If our concept of bug severity doesn't reflect that,
then our concept of bug severity needs to be improved: bug severity,
and the BTS in general, should be a tool to make Debian as good as it
can be, not a mechanism to coerce maintainers.
(If developers are resorting to personal attacks like calling you a
fool, then that is not acceptable, and I would suggest contacting the
anti-harassment team if you feel that it is appropriate. However, that's
orthogonal to whether that developer is prioritizing bugs correctly.)
smcv
Reply to: