Re: Guidance for CVE triage and listing packages in dla-needed.txt
Hi Raphael
You can see corrected statistics in a separate email.
Now to comment a few things below.
On Wed, 10 Apr 2024 at 10:49, Raphael Hertzog <hertzog@debian.org> wrote:
>
> Hello,
>
> On Tue, 09 Apr 2024, Ola Lundqvist wrote:
> > Let me use some data from CVEs for last year 2023.
> > I used the following method to extract the data
> > grep -B 5 '\[buster\]' list | grep -A 5 "^CVE-2023-" | grep '\[buster\]'
> > and then grepped for the end-of-life, not-affected (and so on to count further).
> > It is not a perfect method, but I think it is good enough to crunch the numbers.
> >
> > 1260 CVEs with tag [buster]
> > 382 end-of-life
> > 410 not-affected
> > 281 no-dsa
> > 71 postponed
> > 58 ignored
> > 58 fixed (most in the linux kernel)
>
> Those numbers are quite surprising. I hope there's some error somewhere
> otherwise I wonder what has been done in the 2400+ hours paid each year to
> work on LTS... I'm pretty sure we have fixed more than 58 CVE. The average
> month has 20 to 30 updates (see
> https://lists.debian.org/debian-lts-announce/2024/03/threads.html for
> example).
Yes I was surprised as well. When I checked yesterday I could not find
the fault but today with fresh eyes it was rather obvious. :-)
Sorry for the confusion. See the other email about that.
> > A large portion of the CVEs are marked as no-dsa for a long time. We
> > still have 200 of them from year 2019 for example.
> >
> > And I think this is a good practice. Security is always a balance
> > between a lot of things. Fixing everything will likely result in a lot
> > of tools breaking.
> > Also following the regular Debian Security team is also a good
> > practice. They do a good job of analyzing things and we should
> > normally not fix issues in buster when it will not be fixed in later
> > releases. So this makes a lot of sense.
>
> This comment still conflates "no-dsa" with "doesn't need to be fixed". We
> clearly told you many times that it doesn't mean that at all.
Did I write that? I cannot see it it what you comment. But after
reading the whole email I think I understand why you think so.
If you read my proposal I suggest we postpone all no-dsa and then wait
for point release updates as one way to handle it.
That is basically the same thing as what you write below, that no-dsa
means no need for an urgent fix.
But then you commented that we should do our own judgement. Yes it
would be good if we have a guilde. More about that below.
> "no-dsa" means "doesn't need to be urgently fixed, and will not be fixed
> by the security team". It basically defers the question to the package
> maintainer. It's up to the package maintainer to decides between
> "to fix via spu", "postponed" or "ignored".
I agree to this.
> Some package maintainers will typically decide to fix it via a point
> release. But they rarely update the triaging to document "postponed" or
> "ignored". So that's why it's up to the LTS team to make that call
> when we are (alone) in charge of a given release.
This is a good point. I had missed that package maintainers do not
update the security tracker. Can we have tools to help us with this?
> And regarding "we should normally not fix issues in buster when it will
> not be fixed in later releases", we have already explained that it goes
> the other way around: if we decide that something is worth to be fixed
> in buster, then we should help to get it fixed in later releases.
I do agree with that approach. The question then is when we should do that.
I think we need some type of guidance. It is very easy to fall into extremes.
One extreme is to only follow secteam.
The other extreme is to try to fix all no-dsa no matter how unimportant it is.
> Obviously if the sec-team triages it as unimportant/ignored, then we are
> unlikely to fix it. But when the sec team opts for no-dsa, then we are
> entitled to decide to fix it and we can also prepare the SPU to fix newer
> releases.
Yes, then the question is what rule/guide to follow. We can't fix all.
That is clear from the statistics.
> > 2) Another way is to use the definition we have of
> > unimportant/low/medium/high classification and use that as judgement.
>
> I wanted to comment about this classification. While it's not in active
> use by the security team, I think it would be nice if we could provide
> such a classification:
Yes, I think so. Or some other method we choose to follow.
But I think we need to clarify it, because it is a little fuźzy at the moment.
> 1/ it can help to drive attention to the the most pressing CVE
> 2/ it lets us monitor how well we are performing for different severities
> (in terms of delay to provide a fix)
>
> Clearly the LTS sponsors who are funding security work would like to see
> some things like "90% of the high-priority CVE are fixed within one week".
> But we are not able to provide any data.
Precisely.
> Also we are regularly telling customers/sponsors that we are doing on own
> assessment of the importance of CVE and that it may diverge from the
> standard CVSS score that they use as reference. In the ideal world, we
> would provide our own CVSS score with explanations but we are quite far
> from this. In the mean time, it would already be great if we had more
> reliable low/medium/high scoring...
Yes it would be really good. If we add this as part of the triaging
work it would be good.
In a way we need to do that already because we say that we should not
simply tag no-dsa, but rather postpone or not.
There are a few things we need to decide on:
- What severity levels warrants the package to be in dla-needed? Is
low one of them?
- Should all medium be in dla-needed?
If you ask me I think the line is somewhere in medium, but I'm not
sure of what definition to use yet.
Cheers
// Ola
>
> Cheers,
> --
> ⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hertzog@debian.org>
> ⣾⠁⢠⠒⠀⣿⡁
> ⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/
> ⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS
--
--- Inguza Technology AB --- MSc in Information Technology ----
| ola@inguza.com opal@debian.org |
| http://inguza.com/ Mobile: +46 (0)70-332 1551 |
---------------------------------------------------------------
Reply to: