Bug#663174: Bug#481129: Bug#671503: general: APT repository format is not documented
On Fri, May 18, 2012 at 04:34:29PM +0200, Bernhard R. Link wrote:
> * Julian Andres Klode <jak@debian.org> [120518 14:43]:
> > A working draft could be something like the following. It mostly
> > describes the current format for Release, Packages, and Sources
> > files. It's thus missing Contents and Translations, pdiffs, and
> > stuff, but it's a beginning.
> >
> > It specifies different requirements for servers and clients,
> > in order to have clients be backwards compatible with more
> > repositories, and forcing servers to be stricter. Don't know
> > how good that is, though.
> >
> >
> > ============================
> > = Debian Repository Format =
> > ============================
> >
> > This documents a subset of the Debian repository format. This is a work
> > in progress.
> >
> > "Release" files
> > ===============
> > The file "dists/$DIST/Release" shall contain meta-information about the
> > distribution and checksums for the indices. The file "dists/$DIST/Release.gpg"
> > shall be an GPG signature of the "Release" file, compatible with the format
> > used by the GPG options "-a -b -s". The file "dists/$DIST/InRelease" shall be
> > the "Release" file with a GPG clearsign signature compatible with the format
> > used by the GPG options "-a -s --clearsign".
>
> I think it does not make much sense to document all legacy information
> when starting to standardize. I'd opt for only "InRelease".
InRelease is mostly unused by 3rd parties, so better not. It's not even
used by Ubuntu yet, and most clients do not support it yet. So, I'd say
that all three must be available (or actually, Release and Release.gpg
shall be, and InRelease may be) - whereas Clients may assume that either
Release or InRelease is available.
>
> > The following fields might be available:
> >
> > - Origin
> > - Label
> > - Suite
> > - Codename
> > - Date
> > - Valid-Until
> > - Architectures
> > - Components
> > - Description
> > - MD5sum, SHA1, SHA256
>
> In *Release files it is MD5Sum not MD5sum.
OK.
>
> > - NotAutomatic and ButAutomaticUpgrades
>
> That sounds like nothing all is allowed. I think a client should ignore
> everything it does not know about instead.
I don't really understand what you mean.
>
> > Servers shall provide the Release file, and its signed counterparts with at
> > least the following keys:
> >
> > - SHA256
> > - Origin
> > - Suite and/or Codename
> > - Architectures
>
> Isn't Components also required currently?
Well, APT does not use it AFAIK, but all official servers create it. Making
it required seems OK.
>
> > Clients shall accept missing Release files, and Release files without the
> > fields required for servers. They might reject Release files that do not
> > contain at least one of the fields defined herein.
>
> I think a client should not be forced to accept a server without a
> Release.
Yes
>
> > Architectures
> > -------------
> > Whitespace separated unique single words identifying Debian machine architectures
> > as described in Architecture specification strings, Section 11.1.
>
> That section does not specify much, except the output of "dpkg-architecture -L",
> which makes no sense for servers (as a server should not need a dpkg
> with a architecture like that to support clients having it, and a client
> should not need to know about what other clients have).
>
> > Origin
> > ------
> > Shall indicate the origin of the repository.
>
> s/Shall/Can/ to make more clean it is optional?
>
> > Codename
> > --------
> > The Suite field shall describe the codename of the release. This is mostly
> > a free-form string used to give a name to a release.
>
> As this is the primary information, some data about how it can be formed
> would be nice.
The Codename field contains a name given to the release. In Debian systems,
this field contains a name from a Toy Story character. Like Suite, it may
have an optional suffix.
>
> > MD5sum, SHA1, SHA256
> > --------------------
> > Those fields shall be multi-line fields containing multiple lines of whitespace
> > separated data. Each line shall contain
> >
> > (1) The checksum of the file in the format corresponding to the field
> > (2) The size of the file (integer >= 0)
> > (3) The filename relative to the directory of the Release file
> >
> > Each datum may be seperated by one or more whitespace characters.
>
> must be instead?
Somehow, yes. It's a bit redundant with the initial sentence, so
they have to be merged.
>
> > Client behaviour:
> >
> > Any file should be checked at least once, either in compressed or
> > uncompressed form, depending on which data is available. If a file
> > has no associated data, the client shall inform the user about this
> > under possibly dangerous situations (such as installing a package
> > from that repository). If a file does not match the data specified
> > in the release file, the client shall not use any information from
> > that file, inform the user, and might use old information (such as
> > the previous locally kept information) instead.
>
> How about suggesting that a client should not download any files not
> listed in there or listed in other files already downloaded?
That's a good idea.
>
> > If "NotAutomatic: yes" is specified, the client should prevent installation
> > of packages from this repository unless explicitely requested (APT will assign
> > priority 1 to that repository).
>
> There are more clients than installers and there is no point in saying
> what a client should do there. Better describe it's meaning.
>
> > If "ButAutomaticUpgrades: yes" is specified in addition to "NotAutomatic: yes",
> > the client should cause upgrades to packages from that repository to be
> > installed automatically (APT will assign priority 100 to that repository).
>
> dito
>
> > If both are either missing or set to "No", the repository should behave like
> > any other repository (APT will assign either priority 500 or 990 by default,
> > depending on whether the release is it's target release).
>
> Again, that is for the client to decide...
>
> > "Packages" Indices
> > ==================
> > The files dists/$DIST/$COMP/binary-$ARCH/Packages are called Binary Packages
> > Indices. They consist of multiple paragraphs, where each paragraph has the
> > format defined in Policy 5.3 (Binary package control files -- DEBIAN/control),
> > and the additional fields defined in this section.
>
> Here it would be nice to state that uncompressed Packages files do not
> need to exist even if listed in InRelease.
>
> > MD5sum, SHA1, SHA256, SHA512
> > ----------------------------
> > Checksums for the package. They shall be represented in hexadecimal
> > notation. The SHA512 field is not in active use prior to this
> > specification, the MD5sum and SHA1 fields should be considered
> > deprecated, but should still be provided.
>
> Has SHA512 any sense? Perhaps wait till there is something more secure
> than SHA256 is around?
It should not have been in the draft.
>
> > Examples:
> >
> > MD5sum: 2519c8c1afd27e70cf4ac10a5fa46e32
> > SHA1: 646eda5b6d51190181c15f5537428161f6f04c1d
> > SHA256: 3183eff291d1e9d905e78a6b467bbfb90b20fc2808d50b5e91bf55158b4c18be
> >
> > Server requirements: SHA256 shall be available
> > Client requirements: Shall accept files without any such fields, should warn
> > if those fields are missing and a package is used.
>
> If there is none of them, I'd rather suggest a client to reject the
> package.
The current behavior is to request confirmation from the user in APT,
I don't know what the others do.
>
> > Description-md5
> > ----------------
> > An md5sum of the english description. This will be used to lookup the
> > translations in the translation indices. If this field is not defined,
> > the md5sum shall be calculated from the Description field.
> >
> > Server requirements:
> > Either Description or Description-md5 shall be specified.
> > Client requirements:
> > If neither Description, nor Description-MD5 is defined, the result shall
> > be the same as if an empty description was specified for all languages.
> > If Description-md5 is defined, the long description shall be looked up
> > via translation indices if requested.
> > Example:
> > Description-md5: 9fb97a88cb7383934ef963352b53b4a7
> >
> > Description
> > -----------
> > The Description field shall contain the complete package description, if
> > Description-md5 is not defined; or only the short description of the package,
> > if Description-md5 is defined.
>
> I'd suggest that a client should also support files without either of
> those.
Yes. The statement for Description-md5 contains that, they could be
merged.
>
> > "Sources" Indices
> > =================
> > The files dists/$DIST/$COMP/source/Sources are called Sources indices. They
> > consist of multiple paragraphs, where each paragraph has the format defined
> > in Policy 5.5 (5.4 Debian source control files -- .dsc), with the following
> > changes and additional fields. The changes are:
> >
> > - The "Source" field is renamed to "Package"
> >
> > - A new mandatory field "Package-List"
>
> That field is quite new. Not sure if it makes sense to make it
> mandatory.
I don't know from when this field is, I just saw it.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Reply to: