Re: security-tracker: A proposal to significantly reduce reported false-positives (no affected-code shipped)
Hello everyone,
I've done some small updates to the proposal, mostly improving readability and
making my suggestion more clear.
v2 below:
I would like to propose something which will lower the amount
of reported false-positive CVEs to our users by about 20%.
# tl;dr
We don't have a unique way of stating that a CVE does not affect us when we
don't build the affected package's feature or hardening blocks exploits.
This leads to our users being required to manually distinguish which CVEs
affect them and which don't.
I propose we mark those cases as not-affected.
Alternatively, I mention an option to create a new state to indicate that the
resulting package is not affected due to the build options. I also explain why
that's not my prefered approach.
# Problem statement
The possible outcomes of a CVE assessment in our security-tracker are[0]:
> <no-dsa> | <unfixed> | <undetermined> | <not-affected> | <itp> | <ignored> | <postponed>
We also have the following severity levels [0]:
> SEVERITY_LEVEL : (unimportant) | (low) | (medium) | (high)
"unimportant" being defined as:
> unimportant: This problem does not affect the Debian binary package, e.g., a
> vulnerable source file, which is not built, a vulnerable file in
> doc/foo/examples/, PHP Safe mode bugs, path disclosure (doesn't matter on
> Debian). All "non-issues in practice" fall also into this category, like
> issues only "exploitable" if the code in question is setuid root, exploits
> which only work if someone already has administrative privileges or similar.
> This severity is also used for vulnerabilities in packages which are not
> covered by security support.
We have a problem in the way we assess CVEs when the generated package is not
affected (but the source code contains the vulnerability). Our current process
is to set "no-dsa" and lower the severity to "unimportant", although it's also
possible that in some cases people are making use of "ignored", which
represents "won't fix".
The result is that "unimportant/no-dsa" CVEs can mean two things:
1) We are affected but we the severity is too low, eg.: packages not covered by
security support, the CVE is considered a non-issue by our security-team but we
are still affected...
2) We are definitely not affected since we don't build that feature of the
software or we have hardening in place which prevents this from being
exploited.
This leads to our users, who are interested in knowing which CVEs affect their
systems, having to check the notes of every CVE on security-tracker to
filter-out the false-positives.
# Proposed solution
I propose that we start setting CVEs to not-affected also when the following is
true for all officially supported architectures:
* We don't ship the affected source package.
* We don't build the affected feature.
* We have hardening which makes the exploit impossible (only in the cases when
there's no doubt about it).
If we still want to flag the cases where a build with different flags might
change that assertion, we can use the "(free text comment)" section of the NOTES[0] to
mention it.
Effectively this proposal means I would push an MR updating the documentation
at [0] and start changing those CVEs to not-affected. I'm not asking for anyone
to do the work.
# Stats
As a way of sampling the impact of this issue, I've done a high-level check on
how many sets of affected package-CVE we have in our debian:stable docker
image[1].
Out of the 82 affected package/CVE pairs, 15 were clear cases of our packages
not being affected.
Out of the rest of those, the majority are other cases where we are reporting
non-issues, but those require a deeper investigation so I don't want to assume
they also fall under this case.
So 18% of the reported affected packages are false-positives. Based on what
I've seen, I believe this is a fair estimate to extrapolate.
I've listed some examples to this issue at [2].
# Alternative solution
If using the "free text comment"[0] is not a good enough way of stating that
only the source contains the vulnerable code:
## A1) Add a new sub-state "only-source-vulnerable", to be used in addition to "not-affected"
## A2) Add a new mutually exclusive state to the set: "not-affected-build-artifacts"
I don't like these approaches because they increase the complexity of our process
(a new state is more costly than a free text mention) where there's not a clear
benefit/motivation. What's the value in saying the sources carry the vulnerable
code? If someone does their own modified build of a package, all bets are off
and that's not an official package.
It should also be mentioned that identifying cases where only the source-code
is vulnerable will never be done perfectly due to how easy it is to miss a
bundled library which is not used. For example, rsync bundles zlib and we do
not set rsync as affected for all zlib CVEs (rsync does not use the bundled
lib), would we like otherwise to be the case?
Coming up with a new state is confusing as systems/people reading that might
end up parsing it as "affected". So I prefer A1 over A2.
This being said, the non-preferred alternatives are still better than the
current situation IMHO.
[0] https://security-team.debian.org/security_tracker.html#summary-of-tracker-syntax
"ignored" and "postponed" are sub-states, supposed to be used together with "no-dsa".
[1] $ grype debian:stable
[2] https://security-tracker.debian.org/tracker/CVE-2011-3374
[2] https://security-tracker.debian.org/tracker/CVE-2022-0563
[2] https://security-tracker.debian.org/tracker/CVE-2017-18018
[2] https://security-tracker.debian.org/tracker/CVE-2019-19882
[2] https://security-tracker.debian.org/tracker/CVE-2023-28320
Cheers,
--
Samuel Henrique <samueloph>
Reply to: