Extending the build profile namespace
Hi,
Since Debian Jessie, most of the tooling supports build profiles and a
few packages started using them. I think that it is a good time to
review current build profile usage and extend guidance on adding further
profiles. I will summarize current usage, evaluate current practises and
propose two changes to the build profile namespace at the end of this
mail.
The currently used profiles are:
cross: autogen flex gcc-5 gcc-6 gcc-snapshot nss openjdk-8 openjdk-9
u-boot
nobiarch: glibc ncurses
nocheck: apt botch dpkg dpkg-repack gem2deb gnutls28
kfreebsd-kernel-headers libpipeline make-dfsg mod-gnutls
network-manager nghttp2 notmuch pango1.0 perl perl
php-net-publicsuffix python-cryptography spdylay systemd
win-iconv
nodoc: botch cargo rustc
noudeb: hurd systemd
stage1: alex android-platform-build android-platform-system-core cmake
doxygen ffmpeg freetype glibc gnumach happy hurd libproxy linux
linux-grsec nautilus newt openldap pulseaudio pypy systemd
tcltk-defaults util-linux uuagc x264
stage2: cargo hurd
stage3: hurd
I draw a few observations from this listing:
* The cross profile generally adds self-dependencies. It seems
well-defined and put to use properly. What is still unclear about
this profile is whether sbuild or dpkg should enable it automatically
for cross builds (#774129). There are also two notable non-examples:
Both groff and icu avoid the need for this profile by building
themselves twice during cross.
* The nobiarch profile inhibits building multilib packages. The name
originated in the gcc-$VER packaging that allows
DEB_BUILD_OPTIONS=nolang=biarch. There also are some patches about
adding more use (#709623, #737955, #779459). When used, it seems
being used properly.
* The nocheck profile is the cousin of DEB_BUILD_OPTIONS=nocheck and
must be used in conjunction with that option. Its sole purpose is to
mark droppable dependencies and it seems to be used properly. I would
be happy to see even wider adoption (e.g. #787044), because this is
one of the easiest and safest ways to work on bootstrap problems.
Correct usage can be verified without any cross building involved by
building the package with and without and comparing the results with
diffoscope. Many of the dependency cycles relevant to early cross
bootstrap are broken using this profile.
* The nodoc profile is a bit strange. It is supposed to drop
documentation from packages or to drop documentation packages. The
former leads to packages whose content varies with profiles (which
generally is bad) and the latter mostly drops Arch:all packages, so
in many cases simply doing an arch-only build achieves the same
effect. If the only Architecture: all package from a source package
is a -doc package, then generally the nodoc profile is not necessary
(e.g. cargo and rustc). Just populate Build-Depends-Indep properly.
Maybe we should revise rules for this profile?
Also some packages implement this as a non-profile
DEB_BUILD_OPTIONS=nodoc (e.g. botch, brian, cython, dipy, isso,
libfreenect, linop, mlpy, mpi4py, neo, nipype, nitime, pandas, patsy,
pelican, pyepr, pymongo, pymvpa2, pytables, python-arrow,
python-bottle, python-skbio, scikit-learn, skimage, smlnj,
statsmodels, trilinos) or DEB_BUILD_OPTIONS=nodocs (e.g. dhcp-helper,
ipython, jsonpickle, manuel, nova, openjdk-8, openstack-doc-tools,
paramiko, pycassa, python-aodhclient, python-ddt,
python-debtcollector, python-futurist, python-glanceclient,
python-hacking, python-kajiki, python-novaclient,
python-openstackdocstheme, python-os-cloud-client,
python-oslo.privsep, python-oslotest, python-pathtools, python-pbr,
python-pylxd, python-testtools, python-tosca-parser,
python-tuskarclient, senlin, sphinxcontrib-programoutput, swauth,
swift-plugin-s3, tempest, testpath, traitlets). See #759186 for
discussion.
* The noudeb profile (despite low adoption) seems clearly defined:
Inhibit building udebs. Its main use is to speed up builds when you
don't need those udebs, but it also helps with breaking cycles.
Adding it to further source packages seems like a nice feature, but
not urgent. A few packages also use noudeb as a permitted value in
DEB_BUILD_OPTIONS (e.g. dbus, xorg-server).
* The various stage profiles serve vastly different needs. The common
theme is breaking dependency cycles, but that's about where
commonality ends.
* Some of these produce binary packages that provide reduced
functionality (e.g. glibc, systemd) while others only drop packages
but retain the functionality of the packages built (e.g. gnumach,
doxygen, linux). I'd like to call the former "unsafe" and the
latter "safe", because the former can change the API of binary
package names and thus break dependency contracts. Perhaps it would
even make sense to manifest this safety property in the profile
name?
* It is generally not well defined what functionality is dropped in
what stages. Instead, the stages are derived from practical need
(which is good). Still, we loose track of whether these stages are
still needed and whether they still work over time. I believe that
this undefinedness is a bad property of these profiles and that we
should therefore stop using them whenever feasible. Instead, I'd
like to see specific profiles (e.g. drop Perl bindings). I
acknowledge that this is the path to becoming more like Gentoo (USE
flags), but maybe that's a good direction?
* There are also quite a few non-examples. The following packages use
the DEB_STAGE environment variable in a similar way without
annotating Build-Depends or adding Build-Profiles headers:
db5.3 cracklib2 gcc-4.9 gcc-5 gcc-6 gcc-snapshot gnat-4.9
libselinux libsemanage postgresql-9.5 ppl
Further down the road, some packages implement profile-equivalent
functionality via DEB_BUILD_OPTIONS.
* The gcc-$VER packaging allows turning off functionality via
e.g. DEB_BUILD_OPTIONS=nolang=d,go,java.
* cyrus-sasl2 evaluates e.g. DEB_BUILD_OPTIONS="no-sql no-ldap
no-gssapi".
* dnsmasq evaluates e.g. DEB_BUILD_OPTIONS="nodbus nodnssec noi18n
notftp uselua".
* reprepro evaluates e.g. DEB_BUILD_OPTIONS=reprepro-nolibarchive.
I am going to provide further profile patches with the purpose of
breaking dependency cycles for audit, cdebconf, libcap-ng, and
libprelude.
Given the above analysis, I see two immediate needs for change in the
handling of build profiles:
1) We should provide a namespace (profile name prefix) where packages
can add their own custom profiles at will and where there is room
for experimentation. Such a namespace would improve the metadata for
packages like gcc-$VER, cyrus-sasl2, dnsmasq and reprepro. I propose
that packages can use "pkg.$sourcepackage.$anyting" whenever the
maintainer of $sourcepackage agrees with that use.
2) Given the mess with stage profiles, I think that we should provide
some better way for generic feature profiles (like Gentoo USE flags)
that are to be used consistently by multiple packages. Of course,
Debian is not going to replicate the diversity of Gentoo's USE
flags, but adding them driven by demand may be a sane choice. For
instance, audit, libcap-ng, libprelude and newt could use a "nopython"
profile for disabling Python language bindings instead of gaining a
meaningless "stage1" profile for breaking dependency cycles. I see
immediate practical use (for replacing stages) in profiles disabling
Go (nogolang), Java (nojava), Perl (noperl) and Python (nopython)
bindings and vague need for disabling Apparmor support (noapparmor),
Bluetooth support (nobluetooth), Lua bindings (nolua), SELinux
support (noselinux), systemd integration (nosystemd), and systemtap
support (nosystemtap).
Furthermore, having a way to distinguish "safe" (package set changing)
from "unsafe" (content changing) profiles would be very helpful.
Unless I hear objections, I will start using those profiles and ask
lintian maintainers to relax profile checks.
Helmut
Reply to: