Converting and unifying policy into a single formatting language?

[Guillem Jover <guillem@debian.org>]

Hi!

I think at the time of the DebianDoc-SGML to DocBook-XML conversion
and then from that to reStructuredText, there seemed to be agreement
(AFAIR) among the editors that unifying (ideally) into a single
formatting language would be best?

For reference there are currently still three languages in use:

  * MultiMarkdown:
    - README.md
    - autopkgtest.md
  * DocBook-XML:
    - (fhs)
    - copyright-format-1.0
    - debconf_specification
    - menu-policy
    - perl-policy
  * reStructuredText:
    - policy

Although the fhs does not seem to count as we do not modify it and
neither build it from source.

This multitude of formatting languages has bothered me for a while,
every time I take a peek at the sources. :) So this time around I
pondered how hard could it be to do that unification, and started
looking into this, which didn't look like that much work TBH. Before
proceeding though, I'd appreciate confirmation that this is desired,
and if so then the process surfaced multiple questions of preferences,
that I'd have for the editors.

I've got a sketch now with everything converted by hand, but the build
system is not fully migrated, so this all needs polishing, and I just
found Hideki Yamane's conversion scripts in the rst-conversion branch,
which I might try to reuse to split the menu-policy and perl-policy
documents into chapters. My plan, like with the original DebianDoc-SGML
conversion would be to create a report in the BTS to track the progress,
and then to send a set of preparatory patches (several of which I have
and seem independent of this conversion anyway) to make the conversion
easier. Then create/adapt a script to ideally handle the switch
automatically, and apply any necessary fixups, so that if this takes time
or the documents evolve in the mean time, then no manual work needs to be
done.

For the questions to the editors:

  - The policy/Makefile seems to be unused and a leftover from the
    original conversion? The policy build seems to be driven entirely
    by the source root Makefile, so I guess I could remove it?

  - There was an issue with duplicated translation directories some
    time ago, where Sean seems to have ended up unifying them in the
    next branch into the source root directory (I was expecting the
    reverse to happen :). I think this could be problematic, because
    then all documents will get dumped there, where they might end up
    sharing the same .po file if they match filename. Which from a
    gettext PoV should be just fine, but that means translators cannot
    easily decide to only translate specific documents based on their
    priorities or target audience for example. Should I move this back
    under policy/(locale|locales)/?

  - While going over the build system I found several seemingly unused
    or obsolete things which I've removed, such as dblatex support,
    and support for the html.tar generation. While neither seem to be
    used, I'm not sure whether you want to preserve the support for
    the latter in case the intention was to enable this again in the
    future?

  - With sphinx it looks like the preferred/expected structure is to
    keep each document under their own subdirectory. I've moved those
    all under one each, except for README.rst because that looks to be
    more helpful if it's in the source root (which I think will require
    special handling in the build system as it does not follow the same
    pattern), but I can move it under a subdir too if you prefer the
    uniformity. I was a bit undecided how to call the subdir for that
    file, and initially had devel/, but then settled on simply README/,
    which might also make unifying the make rules easier. I guess another
    option could be to simply ship the README.rst with no HTML, which
    then removes the entire problem. But let me know what's your
    preference here.

  - I've found the debconf_specification name a bit cumbersome in the
    past, both for its length and its underscore use, and now having
    debconf_specification/debconf_specification.rst and similar
    pathnames all over the place makes it a bit worse. There's also a
    mix of terms used all over the place in git for this for directories,
    make variables and doc-base support, including:

      debconf_specification
      debconf-spec
      debconf_spec
      debconf

    I was thinking about, at least for the sources to unify and rename
    all of these into debconf-spec to make the build system more uniform,
    then rename it when installing it into the .deb. (I could see also
    a proposal to rename it on the .deb, while preserving backward
    compat symlinks, but that might be more disruptive, and might require
    changes on the website for example, which I'd be prepared to handle
    as well, but there might be other potential unintended breakage).
    Let me know as well your preference here.

  - While moving things around, I also wondered whether it might be
    cleaner to move all fhs stuff under a new fhs/ subdir.

  - Should I work off master or next?

Thanks,
Guillem