Bug#761945: fixing links for DLAs in the security tracker
On 2017-03-30 08:38:07, Salvatore Bonaccorso wrote:
> Hi Antoine,
>
> On Wed, Mar 29, 2017 at 03:49:31PM -0400, Antoine Beaupré wrote:
>> On 2017-03-29 17:02:44, Salvatore Bonaccorso wrote:
>> > Hi Antoine,
>>
>> Hi!
>>
>> > If you want to look at this part: There is a ./parse-dla.pl script in
>> > the webwml CVS, which is used to import the DLAs (this is an
>> > analogeous script to parse-advisory.pl which is used to import the
>> > DSAs).
>>
>> I see... The scripts are in /english/security for anyone looking. And if
>> people are (like me) thinking "... wat.. CVS?" then yes, we are still
>> using this:
>>
>> https://www.debian.org/devel/website/using_cvs
>>
>> My cvs commandline finger memory is *definitely* still there though, so
>> that works for me. :)
>>
>> > The "manual" steps one would perform are roughly:
>> >
>> > ./parse-dla.pl $message
>> > cvs add $year/dla-$nr.{wml,data}
>> > cvs commit -m '[DLA $nr] $source security update'
>>
>> Is this something the security team performs as part of the DSA release
>> process? Or is this something the debian-www people do? I guess you need
>> write access to the repository and I see that *you* do, but is this
>> expected from everyone working on releasing public advisories, the same
>> way we need access to the security tracker?
>
> No it's not something we do as part of a regular DSA releasing
> process, and as well not expected to do so, as the websites are under
> debian-www "domain" (and btw, they do a great job!). But often, when I
> have time I do as well the import (but as you will see from cvs log,
> not always). For the security team the current process is: preparing
> the DSA (packages, tracker work, text, releasing packages), send out
> the advisory (at this stage our work is basically done).
Okay, so this is just something the www team needs to catchup with then...
>> And to import older entries, we'll need the original templates, which we
>> deliberately did *not* commit anywhere, so they are basically available
>> only as mailing list archives, and thus hard to find automatically.
>
> But given the debian-lts-announce is archived, shouldn't it be
> relatively easy to frist grab all announces from
> https://lists.debian.org/debian-lts-announce/ then check which one
> need to still be imported, extract the mail and do the import? Or do I
> missunderstand you?
I was assuming I was just web-browsing, but it's true I can probably
access the mailboxes and grep through this directly. It's still a pain
in the butt. ;)
>> I foresee difficulties in importing the missing data...
>>
>> Here's the bits that are missing:
>>
>> * the last DLA on the website is DLA-445-2, which is basically the last
>> DLA before squeeze support ended and wheezy was handed over
>>
>> * among those 445 DLAs, there are actually 31 missing:
>>
>> webwml$ cd english/security/; find -name 'dla-*.wml' | wc -l
>> 424
>>
>> * even worse, it seems there are at least 20 advisories missing from
>> the website because regression uploads hide advisories, because our
>> naming convention differs from DSA ("DLA-XXX-N", where XXX is the
>> original advisory and N are regression updates)
>
> I do not understand this point. What do you mean by hinding? For DSA's
> as well only one https://www.debian.org/security/$year/dsa-$nr is ever
> visible as well (and it depends if the text has been then updated
> according to a regression update or not, and in DLAs cases I guess
> just only the last iteration might has been imported, not the initial
> -1 one).
Here's an example. DLA-445-1 was a Squid upload to squeeze, announced
here:
https://lists.debian.org/debian-lts-announce/2016/02/msg00037.html
it caused a regression, which was fixed in DLA-445-2, announced here:
https://lists.debian.org/debian-lts-announce/2016/03/msg00001.html
You can see both DLAs in the sectracker:
https://security-tracker.debian.org/tracker/DLA-445-1
https://security-tracker.debian.org/tracker/DLA-445-2
(Note, BTW, that the regression update doesn't refer to the previous DLA
or the CVE, you just need to know the convention to figure that out.)
On the website, you only see the regression update:
https://www.debian.org/security/2016/dla-445
That is the equivalent of DLA-445-2. I am not sure that DLA-445-1 is
anywhere.
I guess that's another bug to report as well?
[...]
>> > having something on the debian-wwww side which does this
>> > automatically, once a DSA or DLA arrives would help surely the
>> > debian-www team who then "only" have to do the translations and fix
>> > obvious mistakes. OTOH keep in mind: When the debian-wwww team imports
>> > a DSA or DLA they may need to do some adjustments so, I'm not sure if
>> > it's liked to have the automatism, since sometimes before cvs commit
>> > some changes need to be done on the .wml file.
>>
>> It looks like this is something that should be discussed with the www
>> people... Maybe a bug against www.debian.org?
>
> I think yes on looking further at this with a bug against
> www.debian.org and the debian www team. In particular to find out why
> DLA imports ended, and if someone is willing to help doing the
> remaining task. And the other aspect is if DSA and DLA imports should
> be automated (and problems in the wml fixed later on manually, which
> will be detected since they might cause cron errors mails to the
> debian-www team).
I have filed the following bug reports about this:
* #859122: about 500 DLAs missing from the website
* #859123: automate import of DLAs and DSAs in www.debian.org
Let's see how that goes.
>> This begs the question, however - wouldn't it be simpler to import those
>> advisories in the security tracker directly?
>
> Feel free to, say for example data/DLA/advisories/ (or some other
> directory, but below the data/DLA "namespace"); As the
> testing-security team did for a while, for historical view look at
> data/DTSA/advs.
I'll let others weigh in. At this point, i'd be tempted to avoid
information duplication and instead use a message ID to refer to mailing
list archives instead... but maybe that would create additionnal
complexity again...
> This is my opinion: But for displaying the advisories
> well integrated I think the right place is the Debian websites.
After more reflexion: I agree. The website provides a third-party source
of information that is very fast to render (yay static HTML vs boo
dynamic python site ;). It also provides RSS feeds, if my memory serves
me right, which is useful in all sorts of situations as well. And for
people that prefer email, we still have that mailing list...
> Furthermore at least from Debian Security team view, our process in
> releasing a DSA should not be increased by the need to commit the
> advisory text as well in some place additionally to send the DSA (as
> said above, that I import the DSAs in webwml is purely on "voluntary"
> basis, the team is not required to do so; and the current debian-www
> team is good on picking them up after the mail ist out).
Well, either the burden is on us in the secteams to commit the file that
we already have (which is a very small burden, honestly) or the burden
is on the web team to convert that email into a webpage (which is also
small, but a tad more complicated, i would say).
Really, I can't think of a reason why this couldn't be automated... but
this should be discussed in the other threads. :)
A.
--
They say that time changes things, but you actually have to change
them yourself. - Andy Warhol
Reply to: