Re: dpkg-source v2
On Sun, 2002-10-06 at 13:47, Anthony Towns wrote:
> That's trivially worked around by having:
[...]
> where you replace <foo>_*.orig.tar.gz with <src>_<foo>_*.orig.tar.gz
> when <foo> != <src>. So that doesn't concern me at all.
OK, I agree.
> Well, the easy answer would be that you don't. You waste your
bandwidth
> and upload it, then the archive software notices that it's a
duplicate,
[...]
Yeah, that should work for now. It won't be worse than the current
situation at least.
So, I've modified my format proposal to accomodate your suggestions
(along with most of Wichert's suggestions). In particular, support for
patch-control.xml (and with it, patch dependencies and arch-specific
patches) is dropped. Wichert thought it was too complex, and I think I
agree; at least, we can add it back if it turns out we really really
need it.
So, could you take a look at this proposal?
(It's written in Docbook XML, but for the lazy people out there who
don't want to read it directly and can't run docbook2x, I've put up a
HTML version here:
http://people.debian.org/~walters/files/dpkg-source-v2/index.html
)
<?xml version="1.0"?>
<!-- format.xml: Documentation on dpkg-source format
Copyright © 2002 Colin Walters <walters@debian.org> -->
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
"http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"; [
<!ENTITY Debian "<productname>Debian</productname>">
<!ENTITY dpkg-source-prog "<command>dpkg-source</command>">
<!ENTITY dpkg-source "<productname>dpkg-source</productname>">
<!ENTITY dscfile "<filename>.dsc</filename>">
<!ENTITY unpack-dir "<replaceable><unpack-directory></replaceable>">
<!ENTITY debian-pkgname "<replaceable><debian-packagename></replaceable>">
<!ENTITY upstream-ver "<replaceable><upstream-version></replaceable>">
<!ENTITY full-version "<replaceable><full-version></replaceable>">
<!ENTITY source-version "<replaceable><version></replaceable>">
<!ENTITY archive-suffix "<replaceable><suffix></replaceable>">
<!ENTITY unpack-dir-file "<filename>&debian-pkgname;-&source-version;</filename>">
<!ENTITY patch-control "<filename>patch-control.xml</filename>">
]>
<book id="reference">
<bookinfo>
<title>&dpkg-source; Reference Manual</title>
<authorgroup>
<author>
<firstname>Colin</firstname>
<surname>Walters</surname>
<affiliation>
<address>
<email>walters@debian.org</email>
</address>
</affiliation>
</author>
</authorgroup>
<copyright>
<year>2002</year>
<holder>Colin Walters</holder>
</copyright>
<legalnotice>
<para>
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.1 or any
later version published by the Free Software Foundation. There are
no invariant sections. A copy of the license is included in the section
entitled "GNU Free Documentation License".
</para>
</legalnotice>
</bookinfo>
<preface>
<title>Introduction</title>
<para>
&Debian; source packages are actually made up of several
different components, and there are two different versions of
source packages available. Multiply this by the fact that
there are several different forms of packages, depending on
factors such as whether or not the package
is native<indexterm><primary>native</primary></indexterm>, and
you have a somewhat complex system. To say nothing of the
fact that it has recently (before the advent of the version 2
source format) been quite popular to subvert the source format
and instead do most of its work as part of the package build
process.
</para>
<para>
This manual attempts to address this complexity, by
documenting both version 1 and version 2 source formats, the
design decisions involved in them, as well as giving some
practical advice for using them in the real world.
</para>
<para>
Finally, it should be noted that this manual is not intended
to be a complete reference for
the <command>dpkg-source</command> implementation itself; for
that, see FIXME.
</para>
</preface>
<chapter id="intro">
<title>&dpkg-source; Overview</title>
<para>
Why have a special &Debian; source package format at all? Well,
&Debian; packages need to include a fair amount of metadata, in
order to be well integrated with the &Debian; system; e.g.,
dependency information. In addition, the &Debian; source package
must include commands for building the package. Also, in practice
it turns out that it is very useful for the source format to have
additional features, such as the ability to keep a directory of
patches to be applied after the "upstream" source is unpacked.
</para>
<para>
Therefore, the following are goals for the &Debian; source package
format:
<itemizedlist>
<listitem>
<para>
Contain &Debian;-specific metadata, such as relations
to other packages, a changelog, rules for creating binary
packages from the source package, and pre- and post-installation
scripts.
</para>
</listitem>
<listitem>
<para>
Support both "native" and "non-native" packages.
</para>
</listitem>
<listitem>
<para>
Support any changes the &Debian; maintainer has to
make to a non-native package, including updating binary files
such as graphic images.
</para>
</listitem>
<listitem>
<para>
Allow for multiple, flexible ways of applying these changes;
in particular, a directory of patches to be applied is very
useful.
</para>
</listitem>
<listitem>
<para>
Support different kinds of compression/archive types upstream
maintainers use to distribute sources; in particular
<productname>gzip</productname>
and <productname>tar</productname>, as well as
<productname>bzip2</productname> and
<productname>zip</productname>.
</para>
</listitem>
<listitem>
<para>
Ideally, be flexible enough so that the original archive
doesn't have to be modified for non-native packages.
</para>
</listitem>
</itemizedlist>
</para>
<para>
Now, there are several important characteristics of a &Debian;
source package. The foremost is its name. This name can (and
often is) different from that of any binary packages it may
produce. Secondly, every &Debian; source package has a version;
this is taken from the <filename>debian/changelog</filename> file.
</para>
<para>
There are two major different types of source packages;
&Debian;-native and non-native. In a non-native package, the
source archive is separated into upstream and &Debian; components.
The upstream components are archives of whatever the author of the
program distributes. The &Debian; component is all the
&Debian;-specific metadata mentioned previously.
</para>
<para>
In a &Debian;-native package, there is only one archive, which
contains both the &Debian; metadata and the source code itself (if
any).
</para>
<para>
Note that this is only a minor technical distinction at the level
of &dpkg-source;; it is provided for the convenience of the
maintainers of &Debian; packagers.
</para>
<para>
As implied previously, there are two versions of the &Debian;
source format. The version 2 archive format was recently designed
to meet the goals stated above, and the authors believe it
achieves this. The older version 1 format, which is now
deprecated, has trouble with a few of them. In general, the
version 2 archive format is a superset of that of the version 1
format.
</para>
</chapter>
<chapter id="version2">
<title>Version 2 Format</title>
<para>
The version 2 &Debian; source package format is a major redesign
of the (now deprecated) version 1 format.
</para>
<section>
<title>Source components</title>
<para>
A &Debian; source package is composed of at least two files.
All &Debian; source packages will have a &dscfile; file
associated with them. This section describes the source
components, as well as other bits of metadata associated with
the source package.
</para>
<section>
<title>Native Packages</title>
<para>
Besides the &dscfile; file, native packages consist of just
one other component, the source archive. This file will have
a name of the form:
<filename>&debian-pkgname;_&full-version;.&archive-suffix;</filename>,
where &archive-suffix; can be any
of <filename>.tar.gz</filename>, <filename>.tar.bz2</filename>,
or <filename>.zip</filename>. For example, suppose that the
source package is named <literal>foo</literal>, and the full
version is <literal>1.2</literal>, and the maintainer has
chosen to compress it with <command>tar</command>
and <command>gzip</command>. In that case, the filename
would be:
<filename>foo_1.2.tar.gz</filename>.
</para>
</section>
<section>
<title>Non-native Packages</title>
<para>
These types of packages are more complex than their native
counterpart. First of all, there are one or more upstream
components, as well as a &Debian; component. Most
non-native source packages will consist only of a single
upstream component.
</para>
<para>
All non-native packages include a file
named <filename>&debian-pkgname;_&full-version;.debian.&archive-suffix;</filename>,
where the possible values of &archive-suffix; are the same
as defined above for native packages. This is an archive of
the <filename>debian/</filename> directory, which contains
all relevant &Debian; metadata. Again supposing the source
package is named <literal>foo</literal>, with upstream
version <literal>1.2</literal>, and &Debian;
revision <literal>-1</literal>, the file would be of the
form:
<filename>foo_1.2-1.debian.&archive-suffix;</filename>.
</para>
<para>
For packages which represent just a single upstream source,
the &Debian; source package will include a component
named <filename>&debian-pkgname;_&upstream-ver;.orig.&archive-suffix;</filename>,
which represents the original upstream distribution archive.
Going with our previous example, this would look like:
<filename>foo_1.2.orig.&archive-suffix;</filename>.
</para>
<para>
Finally, source packages which represent multiple upstream
distributions are the most complex. Besides
the &dscfile; and the &Debian; component,
there are two or more upstream source components.
</para>
<variablelist>
<varlistentry>
<term>Normal component</term>
<listitem>
<para>
These components have a filename which looks
like <filename>&debian-pkgname;_&upstream-ver;.orig.&archive-suffix;</filename>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Sub-component</term>
<listitem>
<para>
These components have a filename which looks like
<filename>&debian-pkgname;_<replaceable><subcomponent-name></replaceable>_&upstream-ver;.orig.&archive-suffix;</filename>.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
The two types of components are unpacked differently; see the following section.
</para>
<para>
Suppose that the maintainer of our previous example decides
to add a new upstream source <literal>baz</literal>, with
version <literal>3.9.7</literal> to the source package. In
that case, the source components would look like:
<filename>foo_1.2.orig.&archive-suffix;</filename>, and
<filename>foo_baz_3.9.7.orig.&archive-suffix;</filename>.
</para>
</section>
</section>
<section>
<title>Unpacking</title>
<para>
Unpacking a version 2 Debian source archive proceeds in
several stages. In the following steps, the variable
&unpack-dir; will refer to the name of the subdirectory where
the source package will be unpacked. It has the form
&unpack-dir-file;. For example, the value of &unpack-dir; for
the <literal>foo</literal> native package example would
be <filename>foo-1.2</filename>.
</para>
<orderedlist>
<listitem>
<para>
The &dscfile; is examined for a cryptographic signature; if it
exists, it should be verified. If a signature does not exist,
or verification fails, implementations of &dpkg-source; should
by default exit with an error, unless an overriding option is given.
</para>
</listitem>
<listitem>
<para>
The <literal>Format</literal> field is checked for
compatiblity. It must be at least <literal>2.0</literal>
and below <literal>3.0</literal>.
</para>
</listitem>
<listitem>
<para>
The name of the source package, and the version, are
determined from the <literal>Source</literal>
and <literal>Version</literal> fields, respectively. The
filenames of the components, as well as their size and
SHA1 sum, are determined from the <literal>Files</literal> field.
</para>
</listitem>
<listitem>
<para>
The sizes and SHA1 sums of the components are verified.
</para>
</listitem>
<listitem>
<para>
At this stage, each individual archive component will be
unpacked. The archive is unpacked differently depending
on whether the package is native or not.
</para>
<variablelist>
<varlistentry><term>native</term>
<listitem>
<para>
This type of package is unpacked simply by first
creating the directory &unpack-dir;, then extracting
the contents of the archive into it.
</para>
</listitem>
</varlistentry>
<varlistentry><term>non-native</term>
<listitem>
<orderedlist>
<listitem>
<para>
First, the directory &unpack-dir; is created.
</para>
</listitem>
<listitem>
<para>
The contents of each <literal>Normal component</literal>
archive are unpacked into the unpacking directory
&unpack-dir-file;.
</para>
</listitem>
<listitem>
<para>
Next, the contents of
each <literal>Sub-component</literal> are unpacked into
a subdirectory of the unpacking directory; the
subdirectory will be given the same name as the
>subcomponent< name.
</para>
</listitem>
</orderedlist>
</listitem>
</varlistentry>
</variablelist>
</listitem>
<listitem>
<para>
Finally, any patches from
the <filename>debian/patches</filename> directory (if it
exists) are applied. This process is fully documented in
the next section.
</para>
</listitem>
</orderedlist>
</section>
<section>
<title>Patches</title>
<para>
The version 2 &dpkg-source; format supports separate files of
patches to be applied when a package is unpacked. In the
older source format, any changes necessary to the upstream
source either had to be managed as one large patch, or the
source system had to be subverted.
</para>
<para>
The &dpkg-source; patch system supports both regular text
patches and <command>xdelta</command> patches.
</para>
<section>
<title>Patch Application</title>
<para>
After all the components of a package are unpacked, any
patches found are applied. The patches are applied in
lexicographic order (as defined by Unicode). Both text and
xdelta patches are applied in the working directory of the
unpacked source tree. Text patches are applied as if
the <command>-p1</command> was passed
to <command>patch</command>.
</para>
<para>
Patches whose filename end in <filename>.patch</filename>
or <filename>.diff</filename> are assumed to be regular text
patches (i.e. suitable input for <productname>GNU
Patch</productname>). Patches whose filename end
in <filename>.xdelta</filename> are assumed to
be <productname>XDelta</productname> patches.
</para>
</section>
</section>
</chapter>
<chapter id="version1">
<title>Version 1 Format</title>
<para>To be finished. FIXME.</para>
</chapter>
</book>
<!--
Local Variables:
compile-command: "SP_CHARSET_FIXED=YES SP_ENCODING=XML nsgmls -wxml -mdeclaration/xml.soc -gues format.xml"
End:
-->
Reply to: