Re: Guidance for CVE triage and listing packages in dla-needed.txt
Hi again
I have started with a document that clarify the severity levels. I
also introduced the level "critial" but I'm not sure it adds any
value.
https://inguza.com/document/debian-security-severity-levels
This is just a first draft. It is not final. But comments are welcome.
// Ola
On Wed, 10 Apr 2024 at 22:03, Ola Lundqvist <ola@inguza.com> wrote:
>
> Hi Raphael
>
> You can see corrected statistics in a separate email.
>
> Now to comment a few things below.
>
> On Wed, 10 Apr 2024 at 10:49, Raphael Hertzog <hertzog@debian.org> wrote:
> >
> > Hello,
> >
> > On Tue, 09 Apr 2024, Ola Lundqvist wrote:
> > > Let me use some data from CVEs for last year 2023.
> > > I used the following method to extract the data
> > > grep -B 5 '\[buster\]' list | grep -A 5 "^CVE-2023-" | grep '\[buster\]'
> > > and then grepped for the end-of-life, not-affected (and so on to count further).
> > > It is not a perfect method, but I think it is good enough to crunch the numbers.
> > >
> > > 1260 CVEs with tag [buster]
> > > 382 end-of-life
> > > 410 not-affected
> > > 281 no-dsa
> > > 71 postponed
> > > 58 ignored
> > > 58 fixed (most in the linux kernel)
> >
> > Those numbers are quite surprising. I hope there's some error somewhere
> > otherwise I wonder what has been done in the 2400+ hours paid each year to
> > work on LTS... I'm pretty sure we have fixed more than 58 CVE. The average
> > month has 20 to 30 updates (see
> > https://lists.debian.org/debian-lts-announce/2024/03/threads.html for
> > example).
>
> Yes I was surprised as well. When I checked yesterday I could not find
> the fault but today with fresh eyes it was rather obvious. :-)
> Sorry for the confusion. See the other email about that.
>
> > > A large portion of the CVEs are marked as no-dsa for a long time. We
> > > still have 200 of them from year 2019 for example.
> > >
> > > And I think this is a good practice. Security is always a balance
> > > between a lot of things. Fixing everything will likely result in a lot
> > > of tools breaking.
> > > Also following the regular Debian Security team is also a good
> > > practice. They do a good job of analyzing things and we should
> > > normally not fix issues in buster when it will not be fixed in later
> > > releases. So this makes a lot of sense.
> >
> > This comment still conflates "no-dsa" with "doesn't need to be fixed". We
> > clearly told you many times that it doesn't mean that at all.
>
> Did I write that? I cannot see it it what you comment. But after
> reading the whole email I think I understand why you think so.
> If you read my proposal I suggest we postpone all no-dsa and then wait
> for point release updates as one way to handle it.
> That is basically the same thing as what you write below, that no-dsa
> means no need for an urgent fix.
>
> But then you commented that we should do our own judgement. Yes it
> would be good if we have a guilde. More about that below.
>
> > "no-dsa" means "doesn't need to be urgently fixed, and will not be fixed
> > by the security team". It basically defers the question to the package
> > maintainer. It's up to the package maintainer to decides between
> > "to fix via spu", "postponed" or "ignored".
>
> I agree to this.
>
> > Some package maintainers will typically decide to fix it via a point
> > release. But they rarely update the triaging to document "postponed" or
> > "ignored". So that's why it's up to the LTS team to make that call
> > when we are (alone) in charge of a given release.
>
> This is a good point. I had missed that package maintainers do not
> update the security tracker. Can we have tools to help us with this?
>
> > And regarding "we should normally not fix issues in buster when it will
> > not be fixed in later releases", we have already explained that it goes
> > the other way around: if we decide that something is worth to be fixed
> > in buster, then we should help to get it fixed in later releases.
>
> I do agree with that approach. The question then is when we should do that.
> I think we need some type of guidance. It is very easy to fall into extremes.
> One extreme is to only follow secteam.
> The other extreme is to try to fix all no-dsa no matter how unimportant it is.
>
> > Obviously if the sec-team triages it as unimportant/ignored, then we are
> > unlikely to fix it. But when the sec team opts for no-dsa, then we are
> > entitled to decide to fix it and we can also prepare the SPU to fix newer
> > releases.
>
> Yes, then the question is what rule/guide to follow. We can't fix all.
> That is clear from the statistics.
>
> > > 2) Another way is to use the definition we have of
> > > unimportant/low/medium/high classification and use that as judgement.
> >
> > I wanted to comment about this classification. While it's not in active
> > use by the security team, I think it would be nice if we could provide
> > such a classification:
>
> Yes, I think so. Or some other method we choose to follow.
> But I think we need to clarify it, because it is a little fuźzy at the moment.
>
> > 1/ it can help to drive attention to the the most pressing CVE
> > 2/ it lets us monitor how well we are performing for different severities
> > (in terms of delay to provide a fix)
> >
> > Clearly the LTS sponsors who are funding security work would like to see
> > some things like "90% of the high-priority CVE are fixed within one week".
> > But we are not able to provide any data.
>
> Precisely.
>
> > Also we are regularly telling customers/sponsors that we are doing on own
> > assessment of the importance of CVE and that it may diverge from the
> > standard CVSS score that they use as reference. In the ideal world, we
> > would provide our own CVSS score with explanations but we are quite far
> > from this. In the mean time, it would already be great if we had more
> > reliable low/medium/high scoring...
>
> Yes it would be really good. If we add this as part of the triaging
> work it would be good.
> In a way we need to do that already because we say that we should not
> simply tag no-dsa, but rather postpone or not.
>
> There are a few things we need to decide on:
> - What severity levels warrants the package to be in dla-needed? Is
> low one of them?
> - Should all medium be in dla-needed?
>
> If you ask me I think the line is somewhere in medium, but I'm not
> sure of what definition to use yet.
>
> Cheers
>
> // Ola
>
>
> >
> > Cheers,
> > --
> > ⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hertzog@debian.org>
> > ⣾⠁⢠⠒⠀⣿⡁
> > ⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/
> > ⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS
>
>
>
> --
> --- Inguza Technology AB --- MSc in Information Technology ----
> | ola@inguza.com opal@debian.org |
> | http://inguza.com/ Mobile: +46 (0)70-332 1551 |
> ---------------------------------------------------------------
--
--- Inguza Technology AB --- MSc in Information Technology ----
| ola@inguza.com opal@debian.org |
| http://inguza.com/ Mobile: +46 (0)70-332 1551 |
---------------------------------------------------------------
Reply to: