Re: Removing long descriptions / english from Packages files

[Nicolas François <nicolas.francois@centraliens.net>]

Hi Jörg,

(I'm adding dpkg. See the original mail there:
http://lists.alioth.debian.org/pipermail/debian-l10n-devel/2009-August/000507.html)

On Thu, Aug 13, 2009 at 12:47:46AM +0200, Joerg Jaspert wrote:
> [...]
> It will also mean changes to apt-ftparchive, so lets get to the apt
> maintainers:
> It would mean changes to apt-ftparchive, so it can (at least in generate
> mode) output the description in a different file. That files layout is
> simple 822 format of lines as
> 
> ---+++---
> Package: $PACKAGE
> Description-md5: $MD5
> Description-en: $DESCRIPTION, short and long
> +++---+++
> 
> which should be easy enough to output. When such an output file is
> configured, the descriptions should no longer get written to the
> Packages files.

I'm not sure that will be sufficient, unless the Description-md5 field is
added to the Packages file.
The system needs a way to detect when a description changes.

> (Sidenote: Can we PLEASE DROP MD5 when we are going to work on it?)

Now, if we want to remove the MD5, we need to introduce another way to map
descriptions to actual packages.
This could be done with the package's version.

The only drawback would be that when a new version of the package is
available, and the Translation file is not updated, the description would
not be available.

On the other hand, if the goal is to integrate in the archive the
distribution of the Descriptions and their translations, it might be easier
to change the format of the Translation files (for users), and have an
intermediate file format, which can be used by a dak script to create the
final Translation files, based on (Packages, Translation-en).

>From an user point of view, having the version would be more logical. Even
if from a translator point of view, the version is not relevant and only
the content (MD5) counts.

Note: a change of the format need a change in APT (i.e. probably more than
just apt-ftparchive).

> Everyone:
> Do you see any problem with us simply going this way and throwing away
> descriptions, without providing backward "compatibility" files for one
> release? I just did a short test and neither apt-get nor aptitude nor
> apt-cache did fail on a missing long description (note that we keep the
> short one!). So the only bad point I see from a users POV is that on
> dist-upgrade time they will, for a short timeframe, see packages without
> a description, unless they use a translation anyways.

Users should not experience the "missing long descriptions" if they use
the recommended upgrade process from the release notes (but do they use
the recommended upgrade process?).
If needed, an "aptitude update" could be recommended in the release notes
after APT has been updated.

There could be more "missing long descriptions" issues for users using
pinning with different releases (even more if the Translation files format
changes).
We can check if supporting the two formats is possible.

> [...]
>  - Modify the l10n side (and here its partly guess-work from me what
>    needs to happen) to use the new file for the english descriptions as
>    input, and adapt the sync process l10n<->ftpmaster. Also who of you
>    is/will be behind this?

I think we need to:
 * Be prepared to change the input source for the long descriptions.
   (The transition should not be a big issues, dak already support delays
   from l10n)
   Michael, can you handle this?

 * Test if the English translation file is downloaded when the user's
   locale is set to English
   This should be the case, unless there is a specific exception for
   English.
   I can check this.

 * Test if the English translation file is used as a fallback (e.g. if the
   current locale is C, or if the language of the current locale dos not
   have the translated description)
   I can check this.
   This might require a change in APT to specify the translation files to
   be downloaded (currently, English is always available, and the
   translation file of the locale used during the apt update is
   downloaded)
   I can try to do this if the APT team is overloaded.

 * We may also need to decide what should be the user interface when a
   description translation is not available. If the English string is
   always used, this would probably force to always download the English
   translation. Another solution could be to have a default description:
   "This long description is not available" translated in the user's
   language.

dselect might be impacted.
(I do not know if it currently uses the Translation files, I do not know
if it will support empty long descriptions)

I do not know if dpkg is impacted (currently the available and status
files contain the long descriptions; dpkg -p will output those
descriptions, but I'm not sure dpkg would notice if a long description is
missing).

Cheers,
-- 
Nekral