Use and abuse of the unreproducible tag

[Santiago Vila <sanvila@unex.es>]

Greetings.

I have seen already several maintainers tagging bugs as unreproducible
without actually trying to reproduce the bugs at all. Ok, sometimes it
happens by accident, and sometimes by sloppiness, but sometimes it
happens deliberately too (recent examples: #837614 and #837622).

If a report said "ls" segfaults, we would never tag it unreproducible
after checking that "ls -l" or "ls --color" does not segfault.

If a report said some package "FTBFS when LANG=C" and not just "FTBFS",
we would not tag it as unreproducible after checking that it builds ok
when LANG is any other thing, even if such other thing is what the
maintainer has in his system.

If a report said some package "FTBFS when TZ=UTC" and not just "FTBFS",
we would not tag it as unreproducible after checking that it builds ok
when TZ is any other thing, even if such other thing is what the
maintainer has in his system.

Similarly, if a report says "FTBFS in testing" and not just "FTBFS",
we don't tag it as unreproducible without checking in testing first.

Otherwise "unreproducible" becomes a synonym for "don't want to reproduce it",
which would be quite sad.

This is the official definition for the tag:

unreproducible

    This bug can't be reproduced on the maintainer's system.
    Assistance from third parties is needed in diagnosing the cause of the
    problem.

The second phrase is quite clear about what this tag is about: It's
for bugs which we don't know how to reproduce (i.e. we don't have a
recipe, step by step, to reproduce the bug, or we don't know the
circumstances under which it happens).

If the reporter explicitly says that the problem happens when LANG=C,
or when TZ=UTC, or it happens specifically in testing, this is not the
case at all.

Otherwise, if we ignore completely the "assistance is needed" part and
also interpret "the maintainer's system" with the narrow meaning of
"whatever the maintainer is using, regardless of it being the same as
it's being reported or not" then it would follow that a maintainer
having "ls" aliased to "ls --color" would be unable to reproduce a bug
saying "ls" segfaults (when ls --color does not) and could similarly
tag a bug like that as unreproducible.

I know, the example is completely absurd, but so is the usage of the
"unreproducible" tag I see from time to time.

Is this really what we want?

Now I ask, dear Sebastiaan: Why do you consider acceptable to change
the distribution from the one it's being reported and tag as
unreproducible anyway, and not, for example, the LANG variable or the
TZ variable in the examples above?

Please don't tell me that that "we build our packages in unstable", as
we have a release policy clearly saying "packages in stretch must be
buildable in stretch".

You can't reproduce it, or you don't want to reproduce it?

Thanks.