[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Generated changes and patch systems

On Tue, 2008-05-27 at 18:06 -0700, Russ Allbery wrote:
> Neil Williams <codehelp@debian.org> writes:
> > On Sun, 2008-05-25 at 08:40 -0700, Russ Allbery wrote:
> >> Lots of other packages do this -- one of mine off the top of my head is
> >> xml-security-c.
> > Nope. No mention of aclocal.m4 in debian/rules for that package,
> > just /usr/share/misc/config.guess and config.sub.
> Uh, it's the same problem.  Surely the generalization is obvious?

It's not quite the same problem because aclocal.m4 is regenerated in the
clean rule itself because changing it causes ./configure --recheck to
recreate the (modified) file. So instead of copying it around, it has to
be deleted - merely because a patch system is in use. Seems crazy to me.

> > I need to patch configure and configure.ac in this package, that means
> > that aclocal.m4 is constantly being changed and reverted. It shouldn't
> > matter - it really should not.
> Right, so delete it in the clean rule.

OK. It's a bit like collateral damage - need a patch to configure,
patching causes configure --recheck which modifies aclocal.m4 so delete
aclocal.m4. Hmmm. I don't like it but I'll do it anyway.

> > I see no purpose in lintian (or dpkg come to that) testing for changes
> > in aclocal.m4 - IMHO it should simply be exempt from this check.
> Absolutely not -- you're introducing unexpected changes to the packaging
> diff file, 

Well my point is that the change is entirely expected (because the
patch modifies a file that is involved in generating the changes) - the
changes are merely a consequence of the patch. It feels wrong to have to
add a clean rule for aclocal.m4 as a direct result of patching
configure.ac when aclocal.m4 was not a problem before the patch. 

Anyway, the "problem" of the tmpl/* files is far more resistant.

> >>> I really don't want to do all that for the tmpl/* files as well - I
> >>> don't see the need, copying dozens of files into foo.safe or
> >>> foo.upstream and then moving them back?
> >> Just delete them then.
> > ??? That simply does not work. The problem is that running gtk-doc not
> > only requires tmpl/*.sgml files to exist but it *then modifies them*!
> This is extremely unusual.  Are you *sure* that it does an inplace edit of
> the files during the build process? 

$ ls libgpewidget-0.115.orig/doc/tmpl/ -1

From the build log:
 gtkdoc-mkdb --module=libgpewidget --source-dir=.. --output-format=xml
--sgml-mode --output-format=xml --tmpl-dir=tmpl

Now running lintian...
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/color-slider.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/colordialog.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/colorrenderer.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gpeclockface.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gpedialog.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gpehelp.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gtkdatecombo.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/gtksimplemenu.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/libgpewidget-unused.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/link-warning.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/pixmaps.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/smallbox.sgml
W: libgpewidget source: patch-system-but-direct-changes-in-diff doc/tmpl/spacing.sgml
Finished running lintian.

13 out of 27 files modified by gtk-doc.

Copy those files back into doc/tmpl/ from the upstream source and they
are still modified at the next build.

Add a rm -f doc/tmpl/*.sgml to clean:: and the build fails: No rule to
make doc/tmp/*.sgml

So, yes, --enable-gtkdoc is modifying files included upstream and which
are essential to the build process.

If I drop --enable-gtkdoc I get different contents in the
libgpewidget-doc package.

>  This is almost never what build
> systems do; instead, they generate files from other files using templating
> systems.  They may ship the results of that operation and then redo it
> during the build, in which case you need to delete the regenerated files
> in your clean rule.

Yes, that is what we might expect from build systems - unfortunately it
doesn't always work that way. 

The way gtk-doc works is that it adds a pre-defined gtk-doc.make file
into the upstream source and it is this file that determines everything
about how gtk-doc works. If upstream are using a different version, I
don't see what can be done. --enable-gtkdoc will update the .make file
according to the installed version on the build system. Disabling gtkdoc
means packaging outdated documentation.

It seems to me you're expecting gtk-doc-tools to implement something
like AM_MAINTAINER_MODE where upstream gtk-doc-tools is allowed to make
changes to doc/tmpl/*.sgml but Debian gtk-doc-tools is not. (I always
thought that AM_MAINTAINER_MODE was considered harmful for precisely
this reason.)

> If they really expect the files to exist and to edit them in-place during
> the build, upstream is doing something insane, and it's really something
> that should be fixed upstream; a lintian warning is drawing your attention
> to a very broken behavior.

AFAICT this is not a fault upstream - the tmpl/*.sgml files need to be
included. I suspect it is merely because upstream don't happen to use
the same version of gtk-doc-tools. The "they" in the quoted paragraph
refers to the gtk-doc-tools upstream, not libgpewidget upstream.

It is not at all insane - every package using gtk-doc-tools has the same
process. I'm not sure why only libgpewidget is currently showing this
problem but it is wrong to call it "insane".

gtk-doc-tools is, in essence, an upstream tool. It is meant to modify
upstream files during the build, that is the reason it exists. The need
to run it in the build arises, I'm presuming, from a difference in the
version of gtk-doc-tools between Debian and the libgpewidget upstream
team. Debian simply cannot stipulate that a certain version must be used
upstream. I don't see the bug.

> Right, lintian is diagnosing build system brokenness.  I'm fairly happy
> with lintian's behavior here; what it's drawing attention to is exactly
> the kind of thing that breaks repeated package builds or causes NMUs to
> introduce spurious package diffs, and is something that should really
> ideally be fixed.

Well, I'm not sure that it can be fixed - other than requiring upstream
to use the same version of gtk-doc-tools as Debian which is frankly,

> > With these gtk-doc files, it's not so much that the tmpl/*.sgml files
> > are generated but that a tool essential to the build modifies them in a
> > way that cannot be patched because the results of the patches are
> > variable according to that third party tool.
> If the tool behaves and only behaves in that way (I haven't checked), that
> tool is broken and we should fix that tool.

It probably is behaving that way but there's no accounting for changes
between versions - or for changes that may yet be made that also tweak
the output. I simply cannot risk patching these tmpl/*.sgml files and
having the patches fail at some point in the future, turning a bug in
lintian into a FTBFS in libgpewidget (which has 40 reverse

I still think that lintian is out of line here - this is not a problem
for the maintainer and it is only the maintainer who needs to worry
about updating the package from 0.115 to 0.116 and 0.117 etc.

These files are in the .diff.gz because of the behaviour of a third
party tool which updates a data source within the upstream source
according to internal rules for that tool. OK, maybe that tool could be
improved but that is not necessarily trivial - especially when Debian
cannot stipulate that any particular upstream team must use a particular
version or configuration for that tool.

I haven't been able to identify the supposed bug in gtk-doc-tools and I
don't see that it is up to me to do that - AFAICT this is something that
is not a problem in the real world and lintian is imposing unrealistic
expectations on libgpewidget.

I'm perfectly happy with the behaviour of gtk-doc-tools in libgpewidget
(I'd like gtk-doc-tools to be workable in a cross-build environment but
that's minor because cross-builds rarely care for API documentation
anyway and it is fairly easy to disable gtk-doc if the docs are not
wanted.). If others disagree, it is up to them to file a bug in
gtk-doc-tools and work out just how it should be improved. The mere fact
of a spurious lintian warning is an annoyance, not a problem but I would
still like lintian to be more intelligent about these things, accept
that the world is not 100% perfect and not blame the messenger.

If this is a bug in libgpewidget that I've missed, please let me know in
the usual manner, but for now I'm just going to ignore lintian for this


Neil Williams

Attachment: signature.asc
Description: This is a digitally signed message part

Reply to: