[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Proposalto introduce compiler options passed from dpkg-buildpackage



This is a proposal to introduce a common set of compiler options which
can be set independently from the package, and passed/injected to the
package build process.  It was first discussed at the last UDS; a
corresponding wiki page can be found at [1].

== Rationale ==

The set of compiler/linker flags used for building packages is defined
in Debian policy:

 * Packages should be compiled using ''-g -O2'' unless
   DEB_BUILD_OPTIONS doesn't specify ''noopt'' in which case packages
   should be built using ''-g -O0''.

 * Packages should be stripped, unless ''nostrip'' is passed in
   DEB_BUILD_OPTIONS.

Currently two approaches are used to build the archive with modified
options:

 * Changing the compiler specs to modify the default options (either
   by patching the compiler or by adding custom spec files).

 * Using scripts which wrap the invocations of the compiler and the
   linker.

Changing the behaviour at this level has a few disadvantages:

 * The default behaviour of the development tools is changed
   unconditinally for all uses of the tools (not just package
   building. -fstack-protector), or for the most use cases (i.e. let
   build-essential depend on a package providing wrapper scripts or
   spec files).

 * Packages can hardly work around problems in the development tools
   (for example lowering the optimization level, overriding a default
   optimization level).

 * It is errorprone to combine more than one wrapper package.

 * The injected/changed options do not show up in the build log (in
   the case of spec files), or require extra output (in the case of
   wrapper scripts).

The injection of compiler/linker options should be done in the basic
package build tool (dpkg-buildpackage); the debian/rules file should
handle passing/overriding the options to the upstream build system.

Similiar implementations do exist for rpm (Fedora, OpenSuse) and the
Gentoo build system.

== Use Cases ==

 * Optimize a distribution to build for a spcific processor (as
   currently done with pentium-builder). Involves setting CFLAGS

 * Build a distribution with a common set of security/hardening
   options. Involves setting CPPFLAGS, CFLAGS, LDFLAGS

 * Build shared libraries with -Bsymbolic-functions (no effect on
   executables). Involves setting LDFLAGS.

 * Test build a set of packages using a set of non-standard set of
   compiler/linker options, overwriting any package specific
   optimization options.

== Design ==

We define a set of macros which are passed from dpkg-buildpackage to
the debian build system (debian/rules). The injected flags are
collected and passed to the upstream build system (LoicMinier: is this
only done via the environment or via command line args as well?):

 * '''DEB_HOST_CFLAGS''': Optimization options which are passed to the
   debian build system and can/should be overriden by the package
   build if needed (default value: ''-g -O2'', or ''-g -O0'' if
   ''noopt'' is specified. Overriding options can be used to explicitely
   set a higher optimization level, or work around compiler bugs, which
   only can be seen with some optimization levels (the last opt level "wins").

 * '''DEB_HOST_CPPFLAGS''': Preprocessor flags which are passed to the
   debian build system and can/should be overriden by the package
   build if needed (default: empty). This macro is seldom used (most
   build systems just use '''CFLAGS''' instead of '''CPPFLAGS''').

 * '''DEB_HOST_CXXFLAGS''': Same as '''DEB_HOST_CFLAGS''' for C++ sources.

 * '''DEB_HOST_FFLAGS''': Same as '''DEB_HOST_FFLAGS''' for Fortran sources.

 * '''DEB_HOST_LDFLAGS''': Options passed to the compiler when linking
   executables or shared objects (if the linker is called directly,
   then ''-Wl'' and '','' have to be stripped from these
   options. Default: empty.

 * Compiler flags for other languages not mentioned above (Extend the
   specification as needed).  LoicMinier: would it make sense to have
   libtool flags (LT_)?  And configure args?  What about setting the CC?

A second set of macros is used to overwrite package options, mostly
used for test rebuilds with changed cpu and optimization options.

 * '''DEB_HOST_CFLAGS_APPEND''': Optimization options appended to the
   compiler flags, which must not be overwritten by the package
   (mostly used to for test builds). Default value: empty.

 * '''DEB_HOST_CPPFLAGS_APPEND''': Preprocessor flags appended to the
   preprocessor flags, which must not be overwritten by the package
   (mostly used to for test builds). Default value: empty.

 * '''DEB_HOST_CXXFLAGS_APPEND''': Same as
   '''DEB_HOST_CFLAGS_APPEND''' for C++ sources.

 * '''DEB_HOST_FFLAGS_APPEND''': Same as '''DEB_HOST_FFLAGS_APPEND'''
   for Fortran sources.

Note: The ''DEB_HOST_'' prefix is used to define these options for
cross builds as well. For the case of a native build DEB_HOST ==
DEB_BUILD. We don't care about DEB_BUILD options for cross builds.

dpkg-buildpackage sets these flags to the default values unless the
flags are already defined in the environment.

=== Alternate naming of the macros ===

Do omit the '''DEB_HOST_''' prefix and all packages should honour the
value of CFLAGS etc. in the environment. Many packages are likely to
do this by accident anyway. For packages that set CFLAGS etc., there
are basically two common cases:

 * replace CFLAGS with -O0 etc. for debugging purposes

 * add package-specific CFLAGS

In the first case, we could recommend a standard `debian/rules`
snippet to do that; `CFLAGS += -O0 -g` should be sufficient. In the
second case, `CFLAGS += WHATEVER` should work just fine.

This has the massive advantages that it does not require implementing
an extra variable for everything that might conceivably be used by
`make`, that it is already supported by the majority of build systems
in the archive provided that it is not explicitly overridden by
`debian/rules`, and that it is supported automatically by packages
with trivial build systems that use a configure which understands
CFLAGS, LDFLAGS, etc. A few exceptions will of course exist, but they
would exist with `DEB_HOST_CFLAGS` anyway. Besides, a large number of
the exceptions could be fixed simply by changing `=` to `+=`. To me,
this scheme seems a lot simpler and easier to deploy.

It has the disadvantage that these macros might be injected into the
build for some non-standard build systems.

== Implementation ==

The implementation itself is trivial, but needs to be done in every
source package.

TODO: Examples for debian/rules

TODO: Identify other build systems not using dpkg-buildpackage and
adopt these as well (autopkgtst).

TODO: Make build systems like cdbs comply with this spec.

== Test/Demo Plan ==

The implementation status of a package can be checked by
(automatically) inspecting the debian/rules file for every package
which builds architecture dependent packages, or by inspecting the
build dependencies of a source package (in the case that a build
system supporting the spec is used).

The correctness of the implementation can be checked by inspecting the
build logs.

== Outstanding Issues ==

?

---------------------------------------------------------------------

[1] https://wiki.ubuntu.com/DistCompilerFlags



Reply to: