Re: Guidance for CVE triage and listing packages in dla-needed.txt
Hi Ola,
Thank you for putting thought into the matter of issue severity.
On Sun, Apr 07, 2024 at 11:19:08PM +0200, Ola Lundqvist wrote:
> Hi Roberto
> After first some thinking on what "constitutes a minor issue?" I did some
> research and realized that there is in fact a good classification in the
> Debian Security team list here:
> [1]https://security-team.debian.org/security_tracker.html#severity-levels
> We have "unimportant", "low", "medium" and "high".
While I agree that the security tracker documentation does give four
possible priority levels, in practice it seems that only 'unimportant'
is currently in use.
Using grep to search data/CVE/list, the most recent CVEs carrying tag
priorities of high/medium/low are from the following years.
high (371 occurences) - 2016
medium (1223 occurences) - 2017
low (4509 occurences) - 2021
The counts are slightly over the actual numbers, because in some
instances the issue title or one of the notes may contain a string of
the form "(high/medium/low ....)" but which is not meant to be a
priority indicator. I did not put the effort into perfecting the
expression to filter these out and the numbers seem "close enough".
The 'unimportant' tag is still in regular use and there are quite a
number of current CVEs with that tag.
For context, there are a total of 728,466 lines in the file, tracking a
total of 295,240 distinct CVEs.
> The ones classified as "high" are clearly things that warrant a DSA/DLA,
> so they should never be postponed in LTS.
> We can discuss medium, I think they should in general give a DSA/DLA but
> with lower prio.
> Then we need to discuss how to handle all the "low" severity issues we
> have. We are fixing a lot of them. I think these should in general be
> postponed or ignored. But that is my view. Sure fixing such can be good
> but I'm not sure the risk of regression outweighs the security gain by
> fixing them and also we can discuss whether it is worth fixing.
> Related to this I think we should in fact start to use this severity
> classification more because the severity level will definitely help with
> the decision on which packages to fix first.
All of that said, I think that the 'unimportant' tag may continue to be
useful for us for LTS work, but I am not convinced that high/medium/low
are still useful. And the contents of data/CVE/list quite convincingly
show that the security team is not really using high/medium/low any
longer.
The high/medium/low scheme would be potentially useful if we were
overwhelemed with CVEs to the point where we had to routinely decide to
fix only some subset of outstanding fixable CVEs because we lacked a
sufficient number of contributors to fix all 100%. However, in practice
we have enough contributors that we regularly fix all, or most, of the
fixable CVEs (i.e., those which can be backported and for which the risk
is not too high, etc). It seems to me that we are in a state where if we
elect to not fix a CVE, it is a technical matter (i.e., fix is too
intrusive, risk of regression is too high, etc.) and in those cases we
would simply mark the CVE as ignored in the applicable suites.
The remaining CVEs are either fixed right away, or we defer them (either
explicitly by marking the CVEs as 'postponed', or implicitly becuase
they were already marked 'no-dsa' by secteam and we choose to treat that
the same as 'postponed' for LTS purposes). It seems to me that in such a
case, trying to then apply high/medium/low priorities will require
additional work but not produce any significant additional benefit. I
also don't think that the "Minor issue" comment seen on many CVEs is
particularly relevant for LTS.
That is, the tracker requires a comment. To tag a CVE as simply
'<no-dsa>' but not put a comment there is a syntax error. I can
certainly imagine that many occurences of "Minor issue" are simply there
to make the tracker happy and that all the relevant meaning is conveyed
by '<no-dsa>'. I.e., this should get fixed at some point, but it is not
especially urgent and doesn't warrant a DSA all on its own.
========================================
So, breaking from that let me try to return for a moment to the original
topic of the thread, "Guidance for CVE triage and listing packages in
dla-needed.txt".
In essence, there are two basic scenarios that we have:
1. a particular Debian release passes from the responsibility of secteam
to the LTS team; with it the LTS team inherits a load of CVEs with
previous triage decisions from secteam
2. for the active LTS release(s) some new CVEs are assigned which
(potentially) have to be fixed
In neither of those scenarios does there seem to be a compelling reason
to try to grade CVEs high/medium/low. In the first scenario, a whole
bunch of packages will get added to dla-needed.txt and contributors will
pick them, perform more in depth triage of the open CVES (which are most
probably already marked 'no-dsa' and 'postponed' by secteam), and then
make appropriate decisions. In some cases the CVEs will be fixed, and in
other cases they will re-triaged (as either 'postponed'* or 'ignored').
* The re-triage of 'postponed' may be a 'no-dsa' that is not fixed right
away, but which we simply leave as 'no-dsa' to avoid unnecessary churn
in the security tracker. If there is another change that is needed,
e.g., adding or updatign a comment, then changing the 'no-dsa' to
'postponed' would be OK.
The second scenario works in much the same way, but just deals with CVEs
as they arrive. The key difference is that we are more likely to be
making a first pass decision on the CVE as it pertains to LTS, rather
than inheriting a previous decision from secteam when the release was
not yet under LTS.
So, unless I miss something, we can accomplish everything we need, by
simply discontinuing new assignments of the type 'no-dsa'. This is what
was described in last month's LTS meeting. Here is the propsal that I
made:
Proposal: discontinue use of "no-dsa" in LTS (and ELTS) triage
- Instead, any CVE markings added by FD or another (E)LTS contributor
must either be "postponed" or "ignored"
- Contributors are encouraged to review "no-dsa" entries when beginning
new work on a package and deciding if those CVEs can be fixed as well
(which is something that many contributors, perhaps all, already do)
- To avoid unnecessary churn in the security tracker, we will not batch
update (e.g., marking all LTS "no-dsa" as "postponed" instead); rather
individual CVEs will be reviewed and either fixed or (if appropriate)
updated to "ignored", "not-affected", etc.
Additionally, our practice is to make comments that explain the triage
decision and those comments are always more verbose than "Minor issue".
So, if we are applying a 'postponed' or 'ignored' tag and making an
explanatory comment with it, that seems to me like just what we need to
be doing.
Do you think that this would be sufficient? Or did I miss something that
would make the use of high/medium/low priorities potentially beneficial?
This proposal came out of the discussion from when I first started this
thread nearly a month ago. That is to say, it was after a substantial
discussion had already developed. I am inclined to prefer a simpler
approach and the fewer triage states that we have to manage, the better.
Regards,
-Roberto
--
Roberto C. Sánchez
Reply to: