[RFC] Hits/directions to using autoconf and friends in Debian
Hello developers,
Attached you'll find the new README.Debian file of the autotools-dev
package. Any comments/suggestions/fixes? The whole GNU autotools stuff is
a hairy topic, I know... but if we don't document how to best use them (in
Debian) somewhere, it will only get worse.
Suggestions of other places this information should be in (other than
autotool-dev's nearly-never-read README.Debian file) are welcome.
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh
autotools-dev for Debian
$Id: README.Debian,v 1.8 2001/07/16 00:21:49 hmh Exp $
--------------------------------------------------------
The config.guess and config.sub files, used by autoconf- and automake-
generated scripts, need to be constantly updated to address new architectures.
Unfortunately, Debian's automake package has failed to provide such timely
updates regularly, and even if it did, most developers are not even aware of
the need for such updates. Their upstream often isn't, either.
The result ends up as serious bugs filled in the BTS by porters. Given the
amount of packages using autoconf and automake in Debian, we are talking about
a rather large number of bugs every time we start supporting a new
architecture.
There is no way around an updated config.sub file. Even if one supplies to a
GNU autoconf (configure) script the build and target hosts, config.sub is
needed to get the canonical GNU hostname. In fact, an outdated config.sub
script in that case will break any new architecture in Debian just the same.
This package gives you tools to fix this problem, as well as some of the
accumulated knowledge I (and others) got from dealing with the GNU autotools
and their effect on CVS, upstream and Debian auto-builders.
Please read this file throughoutly before you package something that uses GNU
config/autoconf.
The outdated GNU config.{sub,guess} problem and solutions:
----------------------------------------------------------
To fix this particular problem automatically, we need both a source of fresh
copies for config.{sub,guess} in Debian, as well as a way to get them inside
the packages during their build. autotools-dev provides the fresh copies of
config.{sub,guess} for any programs that might need them (in /usr/share/misc),
including automake and libtool.
By inserting some code in the debian/rules file (the clean target is the best
place for this, as it guarantees the updated files will make it to the source
package when dpkg-buildpackage is run), the whole process can be safely
automated: The config.guess and config.sub files will be updated at every
build, then.
Just add:
-test -r /usr/share/misc/config.sub && \
cp -f /usr/share/misc/config.sub config.sub
-test -r /usr/share/misc/config.guess && \
cp -f /usr/share/misc/config.guess config.guess
to the clean target of debian/rules.
Or, if you dislike not being able to know when a auto-builder did that update
before you could, and didn't let you know that it uploaded files built against
newer config.sub/.guess than your source package (which, I should add, causes
no difference in the build result for supported archs, just allows for new
archs to be built):
clean: autotools [other dependencies]
[clean target]
# The autotools target adds forced build-time dependencies on
# autotools-dev (for /usr/share/misc/config.*) and debscripts (for dch)
# It's also a .PHONY make target.
autotools:
OLDDATESUB=`./config.sub -t | tr -d -` ;\
OLDDATEGUESS=`./config.guess -t | tr -d -` ;\
NEWDATESUB=`/usr/share/misc/config.sub -t | tr -d -` ;\
NEWDATEGUESS=`/usr/share/misc/config.guess -t | tr -d -` ;\
if [ $$OLDDATESUB -lt $$NEWDATESUB -o \
$$OLDDATEGUESS -lt $$NEWDATEGUESS ]; then \
dch -a -p "GNU config automated update: config.sub\
($$OLDDATESUB to $$NEWDATESUB), config.guess\
($$OLDDATEGUESS to $$NEWDATEGUESS)" ;\
cp -f /usr/share/misc/config.sub config.sub ;\
cp -f /usr/share/misc/config.guess config.guess ;\
echo WARNING: GNU config scripts updated from master copies 1>&2 ;\
fi
.PHONY autotools
(do not forget the build-depends!. The above script will FAIL if the
config.sub in the package is so old it doesn't support -t. Do the first upgrade
by hand -- such ancient config.sub/guess scripts are NOT allowed in Debian,
they break a lot of our ports and are considered Release Critical bugs).
Packages that can take advantage of autotools-dev at build time should
Build-Depend on it. Packages that do autotools-dev-related work in autogen.sh
only need not such a build-dependency (but please document the packages
autogen.sh needs in comments at the top of that file!).
Calling GNU configure properly:
-------------------------------
No, you should not simply call ./configure with the proper prefix and
package-dependent options and be done with it (regardless of the fact that just
about all Debian packages do so, including some of mine until a few hours ago
:P).
UNLESS you know better, and your package has very specific needs (hint: you
build several subarch variants, such as with MMX, i586-optimized,
i686-optimized... binary packages), you should call configure this way:
(GNU makefile snippet):
export DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE)
export DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)
./configure --build=$(DEB_BUILD_GNU_TYPE) --host=$(DEB_HOST_GNU_TYPE) \
....... (other configure options) ......
This will tell configure to build for the proper architecture, even if you do
not use dpkg-buildpackage, or manually override DEB_HOST_GNU_TYPE and
DEB_BUILD_GNU_TYPE for cross-compiling. It requires GNU make and dpkg-dev, but
those packages are build-essential in Debian, so that should not be a problem.
YOU STILL NEED AN UP-TO-DATE config.sub FILE TO AVOID BREAKAGE IN NEW
ARCHITECTURES, EVEN WITH THIS SETUP.
The problem with timestamp skews and Debian source packages:
------------------------------------------------------------
As of the time this document is being written, dpkg-source cannot automatically
fix the severe timestamp skew problems that applying a diff file over a
tarball-extracted tree generate. The result of these timestamp skew problems is
that most programs that use autoconf and automake will often try to regenerate
their configure and Makefiles.
Chances are very likely that the autoconf, automake and all related libraries
your package might need will NOT be exactly the same in the
auto-builder/end-user build environment and yours. This IS guaranteed to cause
severe headaches. And it wastes a LOT of CPU time in auto-builders, too. Do
recall that some architectures are very slow (m68k, for example), and that any
wastage of auto-build time is a severe problem for Debian as a whole (it delays
packages from moving to Debian testing, for one!).
One should fix these timestamp skews using a proper chain of "touch foo &&
sleep 1s && touch bar..." in the debian/rules file, to make sure files will not
be regenerated without need. You need that "sleep" in there because some build
environments are too fast for their own good, depending on which filesystem
they use.
Due to what I perceive as a design failure in current dpkg-source, we do not
have a "you must run this target to proper cleanup the tree before the first
build" target, one must choose either the clean target, or a target that gets
called by the "binary-indep" or "binary-arch" targets BEFORE the package's
makefile/configure script is run and has a chance to act on the bad timestamps.
Also, you need a
My personal choice is to run the following just before calling configure:
touch configure.in && sleep 2s && \
touch aclocal.m4 && sleep 2s && \
touch configure
(note that the package I got that example from, fetchmail, does not use
automake).
Proper ordering of the touch commands is extremely important, so you must
enforce it using the '&&'. Get it wrong, and you will break the build.
GNU config, autoconf/automake/autoheader, libtool, gettext and CVS:
-------------------------------------------------------------------
This text applies to any machine-generated build-time files, such as those one
gets when one runs gettextize, libtoolize, autoconf, automake, autoheader,
aclocal, bison...
Experience shows that such files must NOT be kept on CVS. As one prominent
Debian developer so kindly put in IRC, when I first asked about the problem:
"If you do something like that [keep gettext in your CVS tree] you are weird.
Go away." It took me no time flat to understand the wisdom of his words, and
my local fetchmail repository was properly clean of all gettext, autoconf and
other machine-generated files by the next morning.
Why? Because such files are supposed to be regenerated at every build, from
up-to-date, fresh copies of their master files. "But upstream keeps such stuff
in his CVS/tar/whatever repository!". Well, duh. Your upstream probably doesn't
know better. I know for a fact my upstream didn't, either.
And if such files are auto-generated properly at every new maintainer build,
the outdated config.{sub,guess} (as well as the outdated gettext, outdated
libtool, outdated automake...) problem is severely mitigated, as well as
helping the auto-builders.
Your .diff file WILL grow, often a lot. But this is the best compromise,
really. We do not want whatever possibly broken crap upstream has for gettext,
autoconf and friends screwing up Debian automated builds. The gettext, libtool
and autoconf Debian maintainers take great pain to make sure their packages are
up-to-date, bug-free and work sanely on all Debian-supported architectures. All
that work goes down the drain if you leave some old config.sub from an unknown
variant of some weird distribution of 3 years ago just because your upstream
happens to like it.
Debian's cvs-buildpackage properly supports generating all the autoconf/
gettext/whatever stuff on CVS export time (cvs-buildpackage will build the
package from a clean, recently exported tree; not from your working
repository).
The usual way to do this is to create an autogen.sh script, that one runs to
get it to call aclocal, autoheader, autoconf, automake, libtoolize, gettextize
in the proper order, and with the proper parameters. This is the solution
adopted by may clueful upstream projects, as well (although the name of the
autogen.sh script varies). One can either get this script to be run by
configuring CVS to do so, or by adding some Makefile rules in debian/rules.
Do note that a properly done autogen.sh invocation runs before dpkg has a
chance to build the source package, so as to make sure an user does NOT need
any of the programs called by autogen.sh to build the package. This will, in
fact, ease the load on auto-builders as well since the program will build much
faster.
DO pay attention to the timestamp skew problem discussed somewhere else in this
file, though. Also, here's a free hint: adding the automatic-generated files
to .cvsignore files, and adding the .cvsignore files to the CVS tree will ease
the usage of CVS a lot when a new upstream version arrives.
-- Henrique de Moraes Holschuh <hmh@debian.org>
Reply to: