Bug#498380: RFC: Implementing dpkg-vendor and adding vendor handling to dpkg-dev
Package: dpkg
Version: 1.14.24
Severity: wishlist
Tags: patch
Hi,
during the emDebian meeting in Extremadura we discussed the DEB_VENDOR
variable and how we could make good use of it for emDebian
packages. While discussing it we noticed that DEB_VENDOR, CFLAGS,
CXXFLGAS, LDFLAGS,... will only be set when dpkg-buildpackage is used
but not when debian/rules is invoked directly. Bug #498355 fixes this
for Lenny (to be used by backports from squeeze) and this bug extends
it to full functionality.
Below are the notes from the emDebian work session and attached a
patch for dpkg with a proof-of-concept implementation. Use cases are
described in the notes.
Note that the patch uses the existing /etc/dpkg/origins/ while the
work session uses /usr/share/vendor/ as they are not realy user/admin
configurable files but purely set by the distribution. It also does
not include the "vendor-handling" package.
MfG
Goswin
==============================================================================
DEB_VENDOR as an alternative to the patch repository
====================================================
Currently, emdebian has a massive repository of patches against Debian
packages. We would like to fold these back into Debian proper, without
changing the packages for Debian, so we need a way for a package build
to determine that we are building for emdebian (the same method could
be used for Ubuntu).
A wishlist bug for a new variable DEB_VARIANT has been filed against
dpkg-dev some time ago, and has just been accepted (with the name
DEB_VENDOR, which is fine for us) into squeeze's dpkg. While this was
not a requirement for emdebian (as our packages are cross-built, so we
never use the fact that DEB_VENDOR has a host distribution specific
default), it provides us with a standardized interface (for example,
Ubuntu could now use DEB_VENDOR == ubuntu to select the gcc SSP patch
they currently apply when /etc/debian_version says "Ubuntu". This
would allow us to build a "cross" toolchain targetting Ubuntu on
Debian, or vice versa).
There is general consensus that there should be some form of
"inheritance" relationship between vendors (emdebian derives from
Debian, individual hardware vendors derive from emdebian), and that a
package that has not been specialized for a particular vendor should
fall back to the closest "parent" vendor's behaviour.
The hierarchical "vendor" name space is not generally useful for
controlling package builds, so a middle layer is introduced that
converts the single vendor name into a set of build options in a flat
name space; the debian/rules file is responsible for doing this
resolution. On irc Raphael Hertzog suggested to provide a Makefile in
dpkg-dev and include that in debian/rules. The dpkg-dev Makefile could
then do
DEB_VENDOR ?= $(shell dpkg-vendor -qDEB_VENDOR)
DEB_BUILD_OPTIONS ?= $(shell dpkg-vendor -qDEB_BUILD_OPTIONS)
cdbs, debhelper and the like could outomatically include that.
The reason why we require debian/rules to take some action instead of
relying on dpkg-buildpackage is that we can not rely on
dpkg-buildpackage to be used to build. Way too many users invoke
debian/rules <target> directly and that should result in the same
result as via dpkg-buildpackage and higher level tools.
If DEB_VENDOR is empty, "debian" is assumed; if DEB_BUILD_OPTIONS is
empty, Debian policy is assumed (this keeps the current concept of
DEB_BUILD_OPTIONS overriding policy).
The proposed interface would be the "dpkg-vendor" program, used
directly or indirectly by including the Makefile fragment from
dpkg-dev, would be:
dpkg-vendor -q <variable name> queries a variable. If the variable is
set in the environment, the current value is returned, otherwise the
value is deduced from other variables that are set or the default for
the current vendor.
As the build options might be dependent on the actual package, the
query for them should be invoked in the top level of a source
package. If called outside (debian/control does not exist), a warning
is printed to stderr and the "generic" options for the vendor
returned; querying DEB_VENDOR is always possible.
dpkg-vendor {-p|--parent} <vendor> queries whether the current vendor
has a "parent" of <vendor> or the vendor itself. The return code is 0
(yes) or 1 (no).
dpkg-vendor {-i|--is} <vendor> queries whether the current vendor is
the same as <vendor>.
A compliant implementation could add a dependency "vendor-handling" to
dpkg-dev, which provides a single file /usr/share/vendors/current that
specifies the "current" vendor.
The "vendor-handling" package is built by each vendor (Build-Options:
*), and the implementation provided by Debian copies the DEB_VENDOR
variable provided to the build into this file and makes the package
depend on "vendor-$(DEB_VENDOR)". That way the package can be
recompiled without source changes by every vendor. The reason this
should not be in e.g. base-files is that the sources that have to be
recompiled by every vendor should be minimal. A single vendor could
actually have many vendor repositories based on a common concept with
only a minimal change between them, e.g. compile with soft or hard
floats with everything else based on emdebian.
Each "vendor-*" package would also depend on the "parent" vendor's
"vendor-*" package, so inherited options can be looked up by
dpkg-vendor.
Proposed syntax for /usr/share/vendor/<name>:
---------------------------------------------
Vendor: Name of vendor
Vendor-URL: Url of vendor
Bugs: url of bug tracking system
Parent: Name of parent if existing
BUILD_OPTIONS: +add_option -remove_option
<package>_BUILD_OPTIONS: +add_option -remove_option
The DEB_BUILD_OPTIONS variable is then generated as follows:
1) If already set return variable.
2) follow the parent till you hit the root
3) For all options in BUILD_OPTIONS add options prefixed by + to
DEB_BUILD_OPTIONS and remove options prefixed by -.
4) For all options in <package>_BUILD_OPTIONS where package is gotten from
debian/control if available add options prefixed by + to
DEB_BUILD_OPTIONS and remove options prefixed by -.
5) backtrack to the child and go back to 3
The resulting DEB_BUILD_OPTIONS variable is returned.
Example:
--------
Lets look at "TCL" for an example of how the setup would look like.
- vendor-handling is provided by Debian but recompiled by the vendor. The
package contains /usr/lib/vendors/current containing $(DEB_VENDOR),
in this case tcl. No source change needed.
- vendor-emdebian is provided by emDebian and contains
/usr/share/vendors/emdebian with a parent of debian.
- vendor-tcl is provided by tcl, depends on vendor-emdebian and
contains /usr/share/vendors/tcl with a parent of emdebian.
- dpkg-dev depends on vendor-handling -> vendor-tcl -> vendor-emdebian ->
vendor-debian. Installing dpkg-dev automatically pulls in as much of
the vendor hierachy as required. DEB_BUILD_OPTIONS would be
generated by first taking BUILD_OPTIONS from debian (empty), add the
changes for emdebian (add nodocs) and last the changes for tcl (add
notls for example).
We would like to get a freeze exception to add a minimal
"/usr/bin/dpkg-vendor" to dpkg-dev that will return the variable
queried with -q if set and "" otherwise and will return false for
-p. This reflects the current behaviour of no dpkg-vendor binary but
will alow squeeze sources to call it unconditionally and still build
without error on lenny.
[NOTE: this was later discussed on irc #dpkg and it was suggested for
dpkg-dev to provide a Makefile fragment for inclusion in debian/rules
instead.]
In squeeze the dpkg-vendor can then be properly implemeted to give the
full functionality required for a hirachy of vendors allowing for a
smooth transition. It sucks that every source has to be touched but
there is no other way to make a plain "debian/rules binary" call
behave correctly. And anything else is too error prone.
I would also be a good idea to include the DEB_VENDOR and
DEB_BUILD_OPTIONS in effect in the *.changes files so DAK (or other)
can reject packages build for other vendors.
==============================================================================
-- System Information:
Debian Release: lenny/sid
APT prefers unstable
APT policy: (500, 'unstable'), (400, 'unstable-i386')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.25-kvm-nofb (SMP w/2 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/bash
Versions of packages dpkg depends on:
ii coreutils 6.10-6 The GNU core utilities
ii libc6 2.7-13 GNU C Library: Shared libraries
ii lzma 4.43-14 Compression method of 7z format in
dpkg recommends no packages.
Versions of packages dpkg suggests:
ii apt 0.7.14+b1 Advanced front-end for dpkg
-- no debconf information
diff -Nru dpkg-1.14.23/debian/changelog dpkg-1.14.24/debian/changelog
--- dpkg-1.14.23/debian/changelog 2008-09-09 16:17:35.000000000 +0200
+++ dpkg-1.14.24/debian/changelog 2008-09-09 15:06:29.000000000 +0200
@@ -1,3 +1,11 @@
+dpkg (1.14.24) unstable; urgency=low
+
+ [ Goswin von Brederlow ]
+ * Add /usr/bin/dpkg-vendor script
+ * Update /usr/share/dpkg/Makefile.dpkg to use dpkg-vendor
+
+ -- Goswin von Brederlow <goswin-v-b@web.de> Tue, 09 Sep 2008 15:06:43 +0200
+
dpkg (1.14.23) unstable; urgency=low
[ Goswin von Brederlow ]
diff -Nru dpkg-1.14.23/debian/dpkg-dev.install dpkg-1.14.24/debian/dpkg-dev.install
--- dpkg-1.14.23/debian/dpkg-dev.install 2008-09-09 12:30:54.000000000 +0200
+++ dpkg-1.14.24/debian/dpkg-dev.install 2008-09-09 15:57:57.000000000 +0200
@@ -15,6 +15,7 @@
usr/bin/dpkg-scansources
usr/bin/dpkg-shlibdeps
usr/bin/dpkg-source
+usr/bin/dpkg-vendor
usr/lib/dpkg/parsechangelog
usr/share/dpkg/Makefile.dpkg
usr/share/locale/*/LC_MESSAGES/dpkg-dev.mo
@@ -64,4 +65,6 @@
usr/share/man/*/dpkg-shlibdeps.1
usr/share/man/*/*/dpkg-source.1
usr/share/man/*/dpkg-source.1
+usr/share/man/*/*/dpkg-vendor.1
+usr/share/man/*/dpkg-vendor.1
usr/share/perl5/Dpkg
diff -Nru dpkg-1.14.23/man/Makefile.am dpkg-1.14.24/man/Makefile.am
--- dpkg-1.14.23/man/Makefile.am 2008-07-01 19:07:54.000000000 +0200
+++ dpkg-1.14.24/man/Makefile.am 2008-09-09 15:57:19.000000000 +0200
@@ -95,6 +95,7 @@
dpkg-split.1 \
dpkg-statoverride.8 \
dpkg-trigger.1 \
+ dpkg-vendor.1 \
dpkg.1 \
dpkg.cfg.5 \
dselect.1 \
diff -Nru dpkg-1.14.23/man/Makefile.in dpkg-1.14.24/man/Makefile.in
--- dpkg-1.14.23/man/Makefile.in 2008-09-09 12:41:37.000000000 +0200
+++ dpkg-1.14.24/man/Makefile.in 2008-09-09 15:59:01.000000000 +0200
@@ -221,6 +221,7 @@
dpkg-split.1 \
dpkg-statoverride.8 \
dpkg-trigger.1 \
+ dpkg-vendor.1 \
dpkg.1 \
dpkg.cfg.5 \
dselect.1 \
diff -Nru dpkg-1.14.23/man/dpkg-vendor.1 dpkg-1.14.24/man/dpkg-vendor.1
--- dpkg-1.14.23/man/dpkg-vendor.1 1970-01-01 01:00:00.000000000 +0100
+++ dpkg-1.14.24/man/dpkg-vendor.1 2008-09-09 15:54:19.000000000 +0200
@@ -0,0 +1,77 @@
+.TH dpkg\-vendor 1 "2008-09-09" "Debian Project" "dpkg utilities"
+.SH "NAME"
+dpkg\-vendor \- determine the vendor, build-options and relationship for package building
+.
+.SH SYNOPSIS
+.B dpkg-vendor
+.RI [ commands ]
+.PP
+.
+.SH DESCRIPTION
+dpkg\-vendor does provides an interface to query the current vendor,
+vendor specific build options and relationship between vendors for
+package building.
+.PP
+You can specify a vendor by setting DEB_VENDOR in the environment or
+setting /etc/dpkg/origins/default. The later should be provided by
+your vendor already.
+.SH COMMANDS
+.TP
+.BI \-h | \-\-help
+Show the usage message and exit.
+.TP
+.B \-l
+Print the environment variables, one each line, in the format
+\fIVARIABLE=value\fP. This is the default action.
+.TP
+.BI \-q variable-name
+Print the value of a single variable.
+.TP
+.BI \-p | \-\-parent \ parent
+Check if the current vendor is parent or a child of parent. Returns 0
+on success and 1 otherwise.
+.
+.SH TERMS
+.IP "vendor" 4
+The vendor creating the distribution you are running.
+.IP "parent" 4
+Vendors can base their distribution on that of another vendor (parent)
+to inherit and modify the build options of that vendor.
+.
+.SH VARIABLES
+The following variables are known by \fBdpkg\-vendor\fP:
+.IP "\s-1DEB_VENDOR\s0" 4
+The vendor of the distribution to be building for.
+.IP "\s-1DEB_BUILD_OPTIONS\s0" 4
+The vendor specific build options to be used.
+.
+.SH "DEBIAN/RULES"
+\fBDpkg\-dev\fP provides a makefile fragment
+\fB/usr/share/dpkg/Makefile.dpkg\fP to set the standard environment
+variables for a build. This should be included in \fBdebian/rules\fP
+instead of querying the variables manually.
+.PP
+If you need to do something for a specific vendor please use:
+.IP
+.nf
+\&\s-1IS_VENDOR\s0 := $(shell dpkg\-vendor \-\-parent vendor && echo yes)
+
+ifeq ($(\s-1IS_VENDOR\s0),yes)
+ ...
+endif
+.fi
+.TP
+This way distributions based on vendor will also inherit that code.
+.
+.SH FILES
+.I /etc/dpkg/origins/debian
+Root vendor of the vendor hirachy.
+.TP
+.I /etc/dpkg/origins/default
+Default vendor for this distribution. Defaults to debian if missing.
+.
+.SH AUTHOR
+.B dpkg\-vendor
+was initially written by Goswin von Brederlow <goswin-v-b@web.de>.
+The manpage is based on dpkg-architecture.1 by
+Marcus Brinkmann <brinkmd@debian.org>.
diff -Nru dpkg-1.14.23/scripts/Makefile.am dpkg-1.14.24/scripts/Makefile.am
--- dpkg-1.14.23/scripts/Makefile.am 2008-09-09 12:40:55.000000000 +0200
+++ dpkg-1.14.24/scripts/Makefile.am 2008-09-09 15:03:23.000000000 +0200
@@ -16,7 +16,8 @@
dpkg-scanpackages \
dpkg-scansources \
dpkg-shlibdeps \
- dpkg-source
+ dpkg-source \
+ dpkg-vendor
sbin_SCRIPTS = \
cleanup-info \
diff -Nru dpkg-1.14.23/scripts/Makefile.dpkg dpkg-1.14.24/scripts/Makefile.dpkg
--- dpkg-1.14.23/scripts/Makefile.dpkg 2008-09-09 16:17:59.000000000 +0200
+++ dpkg-1.14.24/scripts/Makefile.dpkg 2008-09-09 15:05:51.000000000 +0200
@@ -21,10 +21,10 @@
# This file should be included by every source and sets up some default
# environment variables if they are unset
-vendor = $(shell if [ -r /etc/dpkg/origins/default ]; then cat /etc/dpkg/origins/default; else echo debian; fi )
-DEB_VENDOR ?= $(shell grep "^Vendor: " /etc/dpkg/origins/$(vendor) | cut -d" " -f2-)
+DEB_VENDOR ?= $(shell dpkg-vendor -qDEB_VENDOR)
+DEB_BUILD_OPTIONS ?= $(shell dpkg-vendor -qDEB_BUILD_OPTIONS)
-export DEB_VENDOR
+export DEB_VENDOR DEB_BUILD_OPTIONS
ifeq ($(findstring noopt,$(DEB_BUILD_OPTIONS)),)
diff -Nru dpkg-1.14.23/scripts/Makefile.in dpkg-1.14.24/scripts/Makefile.in
--- dpkg-1.14.23/scripts/Makefile.in 2008-09-09 12:41:38.000000000 +0200
+++ dpkg-1.14.24/scripts/Makefile.in 2008-09-09 15:09:57.000000000 +0200
@@ -225,7 +225,8 @@
dpkg-scanpackages \
dpkg-scansources \
dpkg-shlibdeps \
- dpkg-source
+ dpkg-source \
+ dpkg-vendor
sbin_SCRIPTS = \
cleanup-info \
diff -Nru dpkg-1.14.23/scripts/dpkg-vendor.sh dpkg-1.14.24/scripts/dpkg-vendor.sh
--- dpkg-1.14.23/scripts/dpkg-vendor.sh 1970-01-01 01:00:00.000000000 +0100
+++ dpkg-1.14.24/scripts/dpkg-vendor.sh 2008-09-09 16:25:31.000000000 +0200
@@ -0,0 +1,138 @@
+#! /bin/bash
+#
+# dpkg-vendor
+#
+# Copyright © 2004-2005 Scott James Remnant <scott@netsplit.com>,
+# Copyright © 1999 Marcus Brinkmann <brinkmd@debian.org>.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+# Interface to the vendor hierachy
+
+set -ex
+
+get_vendor() {
+ if ! [ -z $DEB_VENDOR ]; then
+ echo "$DEB_VENDOR"
+ return
+ fi
+ local vendor="debian"
+ if [ -r /etc/dpkg/origins/default ]; then
+ vendor="$(cat /etc/dpkg/origins/default)"
+ fi
+ grep "^Vendor: " "/etc/dpkg/origins/$vendor" | cut -d" " -f2-
+}
+
+get_vendor_file() {
+ if ! [ -z $DEB_VENDOR ]; then
+ grep -l "^Vendor: $DEB_VENDOR$" /etc/dpkg/origins/*
+ return
+ fi
+ local vendor="debian"
+ if [ -r /etc/dpkg/origins/default ]; then
+ vendor="$(cat /etc/dpkg/origins/default)"
+ fi
+ echo "/etc/dpkg/origins/$vendor"
+}
+
+get_parent() {
+ grep "^Parent: " "$1" | cut -d" " -f2-
+}
+
+get_build_options() {
+ if ! [ -z $DEB_BUILD_OPTIONS ]; then
+ echo "$DEB_BUILD_OPTIONS"
+ fi
+ local vendor_file
+ if [ -n "$1" ]; then
+ vendor_file="$1"
+ else
+ vendor_file="$(get_vendor_file)"
+ fi
+ local parent="$(get_parent "$vendor_file")"
+ local deb_build_options=""
+ if ! [ -z $parent ]; then
+ deb_build_options="$(get_build_options "/etc/dpkg/origins/$parent")"
+ if ! [ -z "$deb_build_options" ]; then
+ deb_build_options=" $deb_build_options"
+ fi
+ fi
+ local build_options="$(grep "^Build-Options: " "$vendor_file" | cut -d" " -f2-)"
+ for option in $build_options; do
+ case "$option" in
+ "+"*) deb_build_options="$deb_build_options ${option:1}";;
+ "-"*) deb_build_options="$(echo "$deb_build_options" | sed s/" ${option:1}\b"//)";;
+ *) echo "Error in Build-Options in $vendor_file"
+ exit 1
+ esac
+ done
+ echo "${deb_build_options:1}"
+}
+
+usage() {
+ echo "$(basename $0) - list all variables"
+ echo "$(basename $0) -qVAR - query VAR"
+ echo "$(basename $0) -p|--parent <parent> - is vendor child of <parent>?"
+}
+
+show_all() {
+ echo "DEB_VENDOR=\"$(get_vendor)\""
+ echo "DEB_BUILD_OPTIONS=\"$(get_build_options)\""
+ exit 0
+}
+
+get_var() {
+ case "$1" in
+ DEB_VENDOR) get_vendor;;
+ DEB_BUILD_OPTIONS) get_build_options;;
+ "") echo "Argument to -q missing"
+ exit 1;;
+ *) echo "Unknown argument to -q"
+ exit 1;;
+ esac
+}
+
+is_parent() {
+ if [ -z "$1" ]; then
+ echo "Argument to -p|--parent missing"
+ exit 1
+ fi
+ parent="$1"
+ local vendor_file
+ if [ -n "$2" ]; then
+ vendor_file="$2"
+ else
+ vendor_file="$(get_vendor_file)"
+ fi
+ while ! [ "$vendor_file" = "/etc/dpkg/origins/" ]; do
+ local vendor="$(grep "^Vendor: " "$vendor_file" | cut -d" " -f2-)"
+ if [ "$parent" = "$vendor" ]; then
+ exit 0
+ fi
+ vendor_file="/etc/dpkg/origins/$(get_parent "$vendor_file")"
+ done
+ exit 1
+}
+
+case "$1" in
+ -h|--help) usage;;
+ -q) get_var "$2";;
+ -q*) get_var "${1:2}";;
+ -p|--parent) is_parent "$2";;
+ -l|"") show_all;;
+ *) echo "Bad arguments"
+ usage
+ exit 1;;
+esac
Reply to: