Bug#1001451: Candidate script updates
Hi Neil,
On Thu, Jan 06, 2022 at 02:13:38PM +0000, Neil Williams wrote:
> Hi Salvatore,
>
> On Mon, 20 Dec 2021 12:30:36 +0100 Salvatore Bonaccorso
> <carnil@debian.org> wrote:
> > Hi Neil,
> >
> > Btw, if you want "real" cases why not to do automatic merges:
> > https://tracker.debian.org/news/1287752/accepted-epiphany-browser-412-1-source-into-unstable/
> > typo in CVE, and
> > https://tracker.debian.org/news/1287534/accepted-bluez-562-1-source-into-unstable/
> > where one CVE was actually not really fixed.
>
> So for that epiphany tracker, there is a typo in the d.changelog -
> the automated link for CVE-2021-4508 is a 404.
>
> I've updated the script to catch this and report the error. From the
> security-tracker source-package page for epiphany, it looks like the
> d.changelog entry should be CVE-2021-45088 - a simple typo to omit the
> final repeated digit.
>
> Currently, I'm handling this by advising that the script is re-run
> using the offline support and a corrected list of CVE IDs.
>
> This also adds a --force-version option to the offline support, in case
> sid has moved ahead of the fixed version by the time the CVE list is
> updated.
Ack, this works indeed in this case as the CVE-2021-4508 does not
exist (yet). But in other cases typos are e.g. just wapping a number
or misstype the year, or other typos, which lead to an existing CVE.
So basically this all really boils down to, people working on
security-tracker trying as much as possible, in the human limits :),
to do a diligent work as possible.
But if you spot an easy or obvious to detect case which the script can
warn of, then this can be helpful! After all you are trying to write
up scripts which should help the workflow.
> Should the script also advise that a (normal|minor) bug is filed
> against the source-package to fix the d.changelog (retrospectively or
> in the next changelog entry?) or is it sufficient that the
> security-tracker has the correct information?
it is suficient thath the tracker will have correct information, IMHO.
Basically there are as well different policies in teams on "touching
old changelog entries". For instance for the kernel-team, now
switching hat, we almost never touch an old entry. What has been
written, has been written (we e.g. do not add retrospecively for
instance a CVE id, or change a CVE id which was rejected, and assigned
a new CVE).
> For the bluez case, the 5.62-1 upload would update both CVEs. My
> expectation is that finding out that CVE-2021-41229 was not fixed would
> result in a separate change to data/CVE/list to remove the fixed
> version of 5.62-1. Then when 5.62-2 as uploaded, CVE-2021-41229 would
> be marked as fixed in 5.62-2 as normal. In case a separate change is
> not made to data/CVE/list, I've adjusted the script so that if the fix
> version is older than the version from the retrieved changes entry, the
> CVE is updated to show the newer version. i.e. 5.62-1 would be changed
> to 5.62-2. This introduces a dependency on python3-apt to be able to
> get the correct version comparison support, hopefully that is OK.
> (python3-debian doesn't have dpkg --compare-versions type support.)
Hmm not sure. In if we can say that in general. In the particular
bluez case I spotted that while reviewing the entry, because I had in
the back of my mind still some information about the triage of the
bluez issues, knowing that one was fixed in a new upstream version and
the other not yet, which puzzled me then that both were mentioned, and
so i double checked the source again, confirming that no cherry-picked
patch was applied. So probably yes, if I would not have had this
information then *probably* i would have initially marked wrongly both
CVEs as fixed in 5.62-1. But again this boils down in people doing
this review work to be as carefull as possible, still acknowledging we
do a lot of mistakes which hopefully can be spotted by peer review of
commits (at least jmm/Moritz reviews most of the commits done by
another team member, so we have some backup of each other).
What does that mean for the above: having that support to adjust the
version would be ok, but i expect here that hopefully the 5.62-1 wrong
marking would live in the tracker only for a very short time after a
peer review has spotted a possible mistake.
> It's hard to see how to best test these scripts properly on an ongoing
> basis. The "correct" output of each script depends entirely on the
> current state of data/CVE/list and the setup_paths support gets in the
> way of using a static CVE list in a test directory.
>
> I yet may push all of the non-argparse code into a lib.python.sectracker
> module to at least have some way to test individual functions with
> carefully mangled static copies of data/CVE/list content.
In general I would find it perfect if we can have a sort of testsuite
or CI for the code part indeed of the security-tracker. In fact the
situation now without any tests has potential for many breackages when
one want to fix something, as the code has grown over decades almost
:). If you have a solution for that, and if splittiong out code into
libs can help, then that would be a welcome improvmenet.
(Note: I expect that the security-tracker will need some refactoring
in the 2022 year, as MITRE will change e.g. how data is exported and
which formats are supported, in particular this will affect our
automatic update scripts currently relying on downloading the
allitmes.html file, where it's planned now that this will change and
there will be only a JSON export of the data, we will need to adapt
the security-tracker on that regard, and possibly you will be
interested here :))
> For now, I'll mirror the real changes in data/CVE/list, trying to use
> the scripts to make some of the same changes. (Not all operations on
> data/CVE/list are to be covered.)
>
> Should bin/merge-cve-files also clean up the .xpck files that are
> created? On success only?
Will be good to not have them lying around afterwards yes.
Thanks a lot for your contribution and actions on the tracker! If you
like to contribute to it, then please do not stop :)
I'm still trying on concrete use cases during day to day work with
your scripts.
Regards,
Salvatore
Reply to: