Bug#250202: "debian/README.source" file for packages with non-trivial source
This is the last Policy bug I had tagged as wording. It started with a
proposal for a README.source file documenting how to do things with a
package that uses a non-trivial source format, and then expanded into
standardizing debian/rules targets for doing various things.
Having reviewed the entire bug thread, I think there are a few
misconceptions and a few different cases mingled together, so I'm going to
attempt a summary.
The original concern was being able to unpack a Debian source package and
immediately start patching it, with a goal of creating a modified source
package (for local modifications, security fixes, RC bug fixes, or what
have you). Among the source package formats in Debian, I know of two
possible situations:
dbs-style systems
A dbs-style system is one in which the .orig.tar.gz file contains an
archive of the upstream source, rather than an unpacked copy. After
extracting a source package that uses one of these systems, one will
generally have only an upstream source archive and a debian directory.
In order to start making any modifications, one has to run some
debian/rules target to unpack the upstream source and apply any
patches that were shipped in the debian directory (or sometimes
upstream patches shipped alongside the upstream source in the
.orig.tar.gz).
It may be possible in some cases to modify the resulting source and
then build a binary package without doing any further. However, in
order to generate a source package, and probably even to generate a
binary package unless one uses dpkg-buildpackage -nc, the
modifications to the upstream source usually have to be saved in the
form of a patch and put somewhere else in the package to be included
in the source package and applied properly on build.
Patch systems
A patch system is less comprehensive in its reorganization of the
package. The upstream source is handled in the conventional manner
and unpacking the Debian source package results in the pristine
upstream source and a debian directory that contains all the patches
to be applied (generally in a debian/patches directory).
*If* the modifications that one wants to make to the package don't
overlap with any of the changes made by the patches, one can make
direct modifications to the resulting source, build binary and source
packages, and everything will work. The Debian *.diff.gz will contain
a mixture of direct modifications and patch-system-managed
modifications, which will be somewhat confusing going forward and may
result in the problems explained in the next paragraph, but it often
works in simple cases.
If, however, the patches overlap, one generally has to use the patch
system to add a new patch. This will be the only way to get the
sequencing right, let the patch system cleanly apply and unapply
patches, and generally work as expected.
There is one possible exception worth noting: Bill Allombert mentioned in
the bug log that it's possible to make the clean target depend on the
patch target when using dpatch, everything will work, and people can
unpack the source and make direct changes on top of all the applied
patches. Presumably if there are overlapping changes, this will break
further use of dpatch until the changes have been extracted as a regular
patch, but normal package builds may continue to work. I don't see any
mention of this in the dpatch documentation as a recommended option. A
similar technique could be used with quilt, although with quilt it would
bloat the diff even further due to the contents of the .pc directory.
Now, I think that everyone (possibly except Marco) agrees with the basic
idea of providing a README.source file explaining how to deal with a
package that has a non-trivial source layout. In particular, I think
someone needs to know how to do all of the following:
1. Generate the fully patched sources, the sources in the state in which
they'll be built, so that one can check to see if problems have been
patched, look at the source, and so forth.
2. Add a new patch to the build process (including any special techniques
to use to generate the patch if there's an easier tool than recursive
diff from an unmodified package and the like).
3. Remove an existing patch from the build process.
4. Ideally, explain how to upgrade the package to a new upstream version,
although this should probably be optional.
However, when we get into standardizing targets, this gets a lot murkier.
I think the discussion above makes it clear that the only thing that we
can really address with a target is point 1 above. For all of the
systems, in the general case of modifications that overlap with existing
patches, I don't think we can provide targets that do 2. 3 and 4 are
obviously outside what a target can do.
Accordingly, I think moving forward with specifying a README.source file
that explains the above three or four points is something we can reach
consensus on. I'm not as sure about standardizing a target for 1 (setup,
unpack, and patch are all currently in use), but I suppose that we could
standardize on patched. This does raise insta-buggy issues since existing
packages don't provide that target.
However, I don't think it's going to work to say that after running some
target, people should be able to make modifications and run
dpkg-buildpackage and expect to have that work. In each case, it looks
like there are cases where that isn't going to work, and I don't think
it's viable to say that those patch systems can't be used.
So, finally, I propose the following patch:
--- orig/policy.sgml
+++ mod/policy.sgml
@@ -1926,6 +1926,19 @@
possible is a good idea.
</p>
</item>
+
+ <tag><tt>patched</tt> (optional)</tag>
+ <item>
+ <p>
+ This target performs whatever additional actions are
+ required to make the source ready for editing (unpacking
+ additional upstream archives, applying patches, etc.).
+ It is recommended to be implemented for any package where
+ <tt>dpkg-source -x</tt> does not result in source ready
+ for additional modification. See
+ <ref id="readmesource">.
+ </p>
+ </item>
</taglist>
<p>
@@ -2076,6 +2089,50 @@
the file to the list in <file>debian/files</file>.</p>
</sect>
+ <sect>
+ <heading>Source package handling:
+ <file>debian/README.source</file></heading>
+
+ <p>
+ If running <prgn>dpkg-source -x</prgn> on a source package
+ doesn't produce the source of the package, ready for editing,
+ and allow one to make changes and run
+ <prng>dpkg-buildpackage</prgn to produce a modified package
+ without taking any additional steps, creating a
+ <file>debian/README.source</file> documentation file is
+ recommended. This file should explain how to do all of the
+ following:
+ <enumlist>
+ <item>Generate the fully patched source, in a form ready for
+ editing, that would be built to create Debian
+ packages. Doing this with a <tt>patched</tt> target in
+ <file>debian/rules</file> is recommended; see
+ <ref id="debianrules">.</item>
+ <item>Modify the source and save those modifications so that
+ they will be applied when building the package.</item>
+ <item>Remove existing source modifications that previously
+ were applied.</item>
+ <item>Optionally, document what steps are necessary to
+ upgrade the Debian source package to a new upstream version,
+ if applicable.</item>
+ </enumlist>
+ This explanation should include specific commands and mention
+ any additional required Debian packages. It should not assume
+ familiarity with any specific Debian packaging system or patch
+ management tools.
+ </p>
+
+ <p>
+ <file>debian/README.source</file> may also include any other
+ information that would be helpful to someone modifying the
+ source package. Even if the package doesn't fit the above
+ description, maintainers are encouraged to document in a
+ <file>debian/README.source</file> file any source package with a
+ particularly complex or unintuitive source layout or build
+ system (for example, a package that builds the same source
+ multiple times to generate different binary packages).
+ </p>
+ </sect>
</chapt>
--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
Reply to: