Re: Bug#397939: Lintian: outdated-autotools-helper-file
On Mon, Feb 11, 2008 at 10:53:48AM +0100, Bas Wijnen wrote:
> This is not true if you simply build the whole package from source.
> That is, run autotools during build, remove all generated files,
> including Makefile.in, configure, etc, in the clean target.
>
> For some reason many people seem to think that the whole package must be
> built from source, except for configure and Makefile.in. I still don't
> understand what the idea behind this exception is. Especially when it
> leads to unwanted concequences (unreadable diff.gzs, for example), I
> don't think it is a good idea to hold on to the idea that running
> autotools is not part of building the package.
It's a well-established principle of the GNU build system that there are
targets that are run by maintainers and there are targets to be run by
people building the package. This division is there to make our lives
easier, and ignoring it, IMHO, is cutting off our nose to spite our
face. (Even the section in the GNU Coding Standards regarding the
maintainer-clean target doesn't go so far as deleting configure.)
The underlying problem here is that users who change configure.ac etc.
need to do some additional manual work to update the build system, and
that this should be done automatically. I agree. However, this is not
the same as regenerating the full build system from scratch on every
build.
You should be aware, if you aren't already, that the process of
generating a package's build system from scratch is often really quite
complicated and may well involve network access, which build daemons do
not have. For example, the usual update process for a package that uses
Gnulib involves rsyncing translations for some messages from
translationproject.org. If you mandate that Debian maintainers must
support full build system regeneration on every build, then you will
force some of them to maintain a separate bootstrapping process which is
guaranteed not to access the network, which will involve duplicating
work already done by upstream and will undoubtedly introduce errors at
some point.
Mere incremental regeneration on demand is much simpler and is often
already supported by upstream build systems without requiring any change
(see below).
> I suggest to mandate "remove all generated files in the clean target"
> (formulated in a way which includes "generated by upstream", not only
> "generated by the build target), which implies "rebuild everything in
> the build target".
I object to this proposal. As an example, I am upstream for man-db and
know that its build system is properly generated. Requiring that I
regenerate it every time I build the package would introduce nothing but
hassle for me, and would not in practice provide significant benefits
for users since the build system already caters for them out of the box
(see below).
> With the current wording it is allowed to use shipped built files from
> the upstream tarball, as long as the source is present. It is even
> allowed to ship the build results (uuencoded, for example) in the
> diff.gz and use those. I suppose we all agree that this is unacceptable
> for normal build results.
configure is readable, if you have to. It's not the preferred form for
modification, certainly, but it's not an opaque object file either, and
there's no need to treat it exactly as we would treat opaque object
files.
Bear in mind also that, while upstream maintainers will probably accept
that configure.ac files that don't work with current versions of
Autoconf are bugs, they are also unlikely to have immediate sympathy for
"I rebuilt your package with a version of Autoconf three years newer
than the one you used and it produced a horrible mess; please help me
untangle it". In other words, this proposal will have the effect of
requiring that Debian maintainers become experts in the Autotools before
being able to upload policy-compliant packages again, whereas that
burden could formerly be passed to upstream to be dealt with on their
schedule. I wouldn't expect that figuring out enough about the Autotools
to be able to do this should be beyond most Debian maintainers, but
neither would I expect that they could all deal with some of the weird
stuff I've seen already.
I agree that behavioural changes due to regenerating the build system
are bugs, but what you're suggesting is that all such bugs be elevated
to the status of showstoppers, as it will be impossible to upload
packages without it. I'm sure that that would make the security team's
life a lot harder, for instance - while there are a few vulnerabilities
that require configure.ac changes to fix, in most cases you can get away
without needing to do so.
Furthermore, if as a user you're desperate to make a small tweak to
configure.ac, it *is* often quite possible - if ugly! - to just make the
obvious corresponding change to configure and then get on with your
life. You can't realistically do that with e.g. compiled C object files,
which is one reason we treat them differently.
> Now reread the previous paragraph while thinking of Makefile.in instead
> of "normal build results". It is common practice to do exactly that:
> ship and use pre-built versions, or even ship them in the diff.gz (which
> then gets huge). Makefile.am is only present as source-file, but it
> isn't used at all during the build. Any changes the user makes will not
> be noticed by the build system.
But this simply isn't true across the board! Makefile.am changes are
only ignored if you use AM_MAINTAINER_MODE. One reason people do that is
that it produces more predictable build-dependencies that aren't
affected by timestamp skew. This was more of an issue before bug #105750
was fixed. Nowadays, while I think the jury is still out in some places
regarding AM_MAINTAINER_MODE, the case for it is much less compelling
than it used to be, and the Automake manual explicitly recommends
against it. If you don't use AM_MAINTAINER_MODE, then Makefile.am
changes are automatically noticed and taken into account. A
demonstration:
$ apt-get source man-db
[...]
dpkg-source: extracting man-db in man-db-2.5.1
dpkg-source: unpacking man-db_2.5.1.orig.tar.gz
dpkg-source: applying ./man-db_2.5.1-2.diff.gz
$ cd man-db-2.5.1/
$ touch configure.ac
$ debian/rules build
./configure --prefix=/usr --libexecdir=\${libdir} \
[...]
touch configure-stamp
dh_testdir
/usr/bin/make CFLAGS="-O2 -g -Wall" LDFLAGS="-g"
make[1]: Entering directory `/home/cjwatson/man-db-2.5.1'
cd . && /bin/bash /home/cjwatson/man-db-2.5.1/tools/missing --run aclocal-1.10 -I m4 -I gnulib/m4
cd . && /bin/bash /home/cjwatson/man-db-2.5.1/tools/missing --run automake-1.10 --foreign
cd . && /bin/bash /home/cjwatson/man-db-2.5.1/tools/missing --run autoconf
/bin/bash ./config.status --recheck
[...]
(The same goes for changing Makefile.am.)
Rather than incurring the pain of gratuitous full regeneration every
time, we just regenerate it when the user has changed something. Yes,
the user now gets to resolve any problems that might have been
pre-existing, but realistically either the Debian maintainer or the
upstream maintainer is probably going to run into those at some point
anyway.
I think we should recommend (but not require) that AM_MAINTAINER_MODE
not be used, and perhaps work to specify an optional debian/rules target
that regenerates the build system in an appropriate way. That seems to
provide the necessary benefits for users who need to change these files
without imposing an unacceptable burden on developers. I don't think
there's a good cause to go much further than that at this point.
Cheers,
--
Colin Watson [cjwatson@debian.org]
Reply to: