Fix and improve reference links (was: GSoC status: classification, output format and more)

[Jordà Polo <jorda@ettin.org>]

(I sent this message yesterday, but it looks like it hasn't reached the
list yet. I'm sending it again because other mails sent later have been
processed, and also because the original mail included "suspicious"
attachments - a patch with ~900 URLs - which I will simply link this
time instead.)

On Sat, Jul 26, 2008 at 10:45:59AM -0700, Russ Allbery wrote:
> Marc 'HE' Brockschmidt <he@ftwca.de> writes:
> > This is probably the right time to note something which has been bugging
> > me for some time now. The developer's reference uses actual names for
> > the section internally, so not much effort is spent on keeping the
> > section number stable. OTOH, lintian references it, so this could at
> > some point lead to broken links. I'm not sure what the best solution is
> > in this case, but iff we are redesigning the Ref: use anyway, we should
> > keep this problem in mind.
> 
> I'm all in favor of moving to more stable references.  The main constraint
> from my perspective is to maintain our ability to create working
> hyperlinks on lintian.d.o (which we're probably not doing that well at
> already).

The status of links on lintian.d.o has been bothering me for a while as
well. lib/manual_refs is terribly outdated and manual_refs_update.pl is
broken, so I tried to fix the problems (and implement some new features
as well).

Feel free review patches[1]. A summary follows.

I rewrote manual_refs_update.pl to make it more flexible and collect
references from all kinds of manuals. This also means that it is now
possible to use IDs instead of chapter/section numbers.

To provide an example, I converted FHS references to IDs. So, for
instance, "fhs 5" has been converted to "fhs thevarhierarchy". This is
an interesting example because previously lintian.d.o didn't even
provide links to FHS sections (there aren't section numbers in the HTML
version, only in the PDF), but it is now possible to do so.

(Note that Debian Developer's Reference hasn't been converted yet but it
shouldn't be a problem.)

Additionally, chapter/section titles are provided as well, which is
interesting for the command-line lintian too. So, instead of:

  Refer to Policy Manual, section 11.8.6 for details.

  Refer to FHS, chapter 5 for details.

It displays:

  Refer to Debian Policy Manual section 11.8.6 (Application defaults
  files) for details.

  Refer to Filesystem Hierarchy Standard (The /var Hierarchy) for
  details.
 
Finally, I'm trying to handle updates to manuals in a more automated
way. It is still a work in progress (and thus not included in the
attached patches) but basically, the idea is that titles can be used to
detect "relocated" sections. For example, if we have an old and a new
manual_refs files:

Old manual_refs:
  policy::3.1::The package name
  policy::3.3::The maintainer of a package
  policy::3.4::The description of a package
  policy::3.4.1::The single line synopsis
  policy::3.4.2::The extended description

New manual_refs:
  policy::3.1::New title for "The package name"
  policy::3.3::The description of a package
  policy::3.3.1::The single line synopsis
  policy::3.3.2::The extended description
  policy::3.4::Completely new section

The script is able to detect the following "problems":
  Relocated: policy 3.4 -> 3.3 -- The description of a package
  Relocated: policy 3.4.1 -> 3.3.1 -- The single line synopsis
  Relocated: policy 3.4.2 -> 3.3.2 -- The extended description
  Renamed: policy 3.1 -- The package name -> New title for "The package name"
  Removed: policy 3.3 -- The maintainer of a package
  Added: policy 3.4 -- Completely new section

Unfortunately not everything can be automated, but at least it could be
used to have an idea of which references need to be reviewed and which
are probably "safe".

  1. http://ettin.org/tmp/lintian/manual-refs/