[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Tracking related source packages (new tool)


I went through the several discussions and attempts that happened over
the past few years: we have several similar problems, typically:

- tagging CVEs for renamed packages in Debian
- tagging CVEs for renamed packages in Debian LTS or ELTS
- tagging CVEs for related package sets (branched/versioned/forked)
- suggesting CVEs for packages with embedded code copies
- etc.

but each warrant a different workflow/actions, e.g.:
- reporting for human validation
- updating data/CVE/list
- adding entries to data/CVE-<extended>/list

I worked on a tool that is flexible enough for us to experiment with
these use cases (see examples below).

I pushed it to salsa at:

Your comments and testing would be appreciated.
I can submit a proper merge request if we decide that the current
direction is OK.

A couple notes about our previous exchanges:

- The tool decides between <removed> and <unfixed> by checking the
  list of sid packages in data/packages/ (auto-refreshed)

- The tool cannot decide whether a missing entry is an oversight or a
  conscious decision (e.g. php7.0 missing for a CVE that affects
  php>7.3). As Salvatore noted, we currently do not clutter
  data/CVE/list but this means this implicit "not-affected" triage is
  not documented and hence the script cannot use it.  This can be
  mitigated by one-way relationships and/or a year-based threshold.

Here are a few use cases:

# Report CVE entries that may have been missed for old renamed packages in Debian
$ bin/related-cves.py --transitions data/renamed-packages --report

# Also report CVE entries that may have been missed for newly branched packages in Debian (e.g. the golang-1.xx set)
$ bin/related-cves.py --transitions data/renamed-packages --report --two-way

# Automatically add entries renamed packages in ELTS, in the extended CVE list file, restricting to supported packages
# (can complement the script for automated <end-of-life> entries)
$ bin/related-cves.py --transitions data/renamed-packages.elts --extcve data/CVE-EXTENDED-LTS/list --extadv data/ELA/list \
    --start-year 2019 \
    $(awk '{print $1}' < ../extended-lts/packages-to-support)

# Check issues that might affect krfb and vino in Debian (due to embedding e.g. libvncserver)
$ bin/related-cves.py --transitions data/embedded-code-copies --format embedded-code-copies --report krfb vino

# Check issues that might affect php5 in ELTS (due to embedding various libraries)
bin/related-cves.py --transitions data/embedded-code-copies --extadv data/ELA/list --format embedded-code-copies --report php5

The 'renamed-packages' file look like:
# Renamed/branched/forked packages, probably affected by the same CVEs
# Format: old-package... < new-package...
allegro4.4 < allegro5
cfengine2 < cfengine3
fltk1.1 < fltk1.3
golang-1.7 golang-1.8 golang-1.11 golang-1.15 < golang-1.16 golang-1.17

Last, the current tool usage:

$ bin/related-cves.py --help
usage: related-cves.py [-h] [--report] [--transitions TRANSITIONS]
                       [--format FORMAT] [--two-way] [--extcve EXTCVE]
                       [--extadv EXTADV] [--exttransitions EXTTRANSITIONS]
                       [--start-year START_YEAR]
                       [package [package ...]]

positional arguments:
  package               restrict action to these packages (default: all)

optional arguments:
  -h, --help            show this help message and exit
  --report              output related CVEs, grouped by package (default:
                        modify CVE list file)
  --transitions TRANSITIONS
                        package transitions file (default: data/renamed-
  --format FORMAT       transitions file format (renamed-packages or embedded-
  --two-way             add missing CVEs to both sides of package transitions
                        (default: one-way)
  --extcve EXTCVE       extended CVE file to load, also becomes changes target
                        (e.g. data/CVE-XXXX/list)
  --extadv EXTADV       extended advisory file (e.g. data/XXX/list)
  --exttransitions EXTTRANSITIONS
                        additional package transitions file (default: none)
  --start-year START_YEAR
                        ignore CVEs prior to that date, e.g. the start of an
                        extended support release (default: 0)

What do you think?


On Fri, Feb 26, 2021 at 06:32:00AM +0100, Salvatore Bonaccorso wrote:
> Hi Moritz,
> Thanks for CC'ing.
> On Thu, Feb 25, 2021 at 08:01:42PM +0100, Moritz Mühlenhoff wrote:
> > Am Thu, Feb 25, 2021 at 05:30:05PM +0100 schrieb Sylvain Beucler:
> > > - This problem is similar/related to tracking embedded code copies.
> > >   See https://salsa.debian.org/lts-team/lts-extra-tasks/-/issues/2
> > >   With one difference: there's no reference source package.
> > 
> > Not reallly, embedded code copies has a very poor s/n ratio and
> > would require manual assessment whether actually affected.
> > 
> > For renamed source packages this isn't the case (and if they turn out
> > to be not vulnerable, they should be marked not-affected anyway)
> > 
> > > - This is hard / doesn't make sense to fully automate.
> > >   Security Team expressed opposition to such automation in the past.
> > 
> > Quite the opposite, there's even a bug for it :-) This is #738172.
> In any case though such an automatism should not start clutering the
> list, because usually if one has such say a mapping of source packages
> A renamed from B renamed from C, and know an issue was introduced
> after the B rename, then we usually can skip C.
> > 
> > > - Approaches:
> > 
> > 1. Add a new file to the tracker with active mappings, e.g.
> > - golang-1.15,golang-1.11,golang-1.8,golang-1.7
> I remember we discussed that some years ago indeed at the last sec
> team mmeeting, which resulted in the above bug for tracking :)
> this soulds like an idea, this means we would have an explict list of
> 'actively tracking' packages which is considered with a mapping.  It
> is yet unclear to me how such a mapping would need to be constructed,
> something both human readable easy but clearly parsable for a helper
> script is needed (and with small dependency set ideally from python or
> perl core). In "metalanguage" maybe it could be something describing
> the following
> Case 1. Package is really renamed in unstable, and original package is
> removed (e.g. tiff, tiff3), so this should record as well that tiff3
> is removed.
> Case 2. Package is "superseeds" the original package, examples of
> those might be the list of python versioned source packages python3.9
> -> python3.8.
> For case 2 see as well the above comment, maybe the comment should be
> invalidated and in all circumstance add the entries from the active
> mapping. That means though as well we should then regularly after
> supports end for something clean up the active list that later on
> entreis are not cluttered anymore for stuff which is even not anymore
> in the LTS release (the script might need an override possiblity so
> that other projects like ELTS can manage their own active mapping
> lists).
> Side note: Embedded code copies though should still be handled
> separately, they need to be chekced on case to case basis, first if
> souece code is present, second if the security impact is actually
> given.
> > 2. Write a script which parses the CVE/list and creates a diff which
> > adds "foo <unfixed>" (or "foo <removed>") records if a CVE entry lists
> > one of the source packages of an active mapping, but not the others.
> Once such a lis then is know which is missing to be added with
> CVE/source package addition to data/CVE/list, one needs to know if it
> is <unfixed> or <removed>, this information probably needs to be in
> the mapping itself? Once the package is removed from unstable suite it
> should be in any case <removed> instead of <unfixed> initially.
> For the merging the helper function Emilio implemented (or even the
> script merge-cve-list) can be used to merge the entries properly in
> the list.
> > 3. Run the script manually for a while, to see if it all works well
> > 
> > 4. If it works fine in practice, set up a hook/systemd timer to
> > run it automatically and commit the result to the tracker.
> Agreed on the manual part, integration of automatic updates though
> only once we would be confident to work in practice in almost all
> cases withouth we  need additional fiddling later afterwards because
> it added in case X a bunch of entries wich are wrong.
> Regards,
> Salvatore

Reply to: