Re: Re: Updating dpkg-buildflags to enable reproducible=+fixfilepath by default
Hi!
On Mon, 2021-01-11 at 00:48:42 +1100, Stuart Prescott wrote:
> Sune Stolborg Vuorela wrote:
> > Can you provide some kind of hook in the environment so we at least can work
> > around it in the users, so that the internal functions (where the output of
> > __FILE__ is forwarded to) can glue e.g. the content of an environment
> > variable in front of the usage of the __FILE__ content to try to
> > reconstruct it at runtime, so that all packages doesn't need to do export
> > SOURCE_BASE_DIR=`pwd`; make test, but it will just work if we patch Qt and
> > similar libraries ?
Sure, that sounds doable, but as always given that we have not yet
decided to obsolete supporting debian/rules as a build entry point,
any such support would either need to be conditional from the upstream
or packaging side, or something like debhelper would also need to
guarantee it is set up. :/
> SOURCE_BASE_DIR has a delightful symmetry to SOURCE_DATE_EPOCH and the
> algorithm for use is similar: if it is in the environment then use it, if not,
> fall back to the previous behaviour.
Yes! And as this was triggered by reproducible builds, and as you
mention due to the similarities, it would nice to publish a “spec”
once we reach some agreement, given that would end up being part of
a neutral project, with more chances for upstream adoption.
> At its crudest level, a package's debian/rules could include SOURCE_BASE_DIR=
> $(shell pwd); however, putting this into the build environment saves source
> changes in multiple packages, sets us up for future use across the entire
> ecosystem, signals to upstreams that this is a broad standard and not a mere
> hack in d/rules, and also saves maintainers from thinking about nasty corner
> cases to do with current directories with makefile invocation.
>
> Looking to the future, tests using SOURCE_BASE_DIR to locate test data would
> allow build-time tests to be repurposed as autopkgtest tests too, as the
> autopkgtest could tell the tests where the test input data is to be found.
> This would be a substantial improvement on the current situation where the
> tests can only be run at build time and not on ci.debian.net.
>
> By dpkg making SOURCE_BASE_DIR 'real', there is a distinct prospect of Debian
> carrying Qt5 patches to use it (thanks Sune!) and for Qt6 to include them
> upstream.
I agree with all of this, barring my initial comment above. I guess I
need to go back to the thread proposing to obsolete debian/rules as a
supported build entry point. :)
I think Simon's remark might be worth having a thought about too. But
as a starting point I prepared the attached patch, which sets the
variable from dpkg-buildpackage and pkg-info.mk. Once there's
agreement on the variable(s), I'm fine merging this, but whether to
include this for bullseye would need release-team approval at this
point I guess.
Thanks,
Guillem
From 33f7213e34dad402d5159b3a82bf2efc9454222f Mon Sep 17 00:00:00 2001
From: Guillem Jover <guillem@debian.org>
Date: Sat, 16 Jan 2021 14:23:55 +0100
Subject: [PATCH] dpkg-buildpackage: Add SOURCE_BASE_DIR support
This is a generic variable that can be used by upstream projects wanting
to get at the source root directory, independently of out tree builds,
and in a portable way in contrast to making assumptions about for
example the content of the C __FILE__ preprocessor macro.
Proposed-by: Sune Stolborg Vuorela <sune@vuorela.dk>
---
man/dpkg-buildpackage.pod | 5 +++++
scripts/dpkg-buildpackage.pl | 3 +++
scripts/mk/pkg-info.mk | 4 ++++
scripts/t/dpkg_buildpackage.t | 8 ++++++++
t-func/atlocal.in | 3 +++
5 files changed, 23 insertions(+)
diff --git a/man/dpkg-buildpackage.pod b/man/dpkg-buildpackage.pod
index e648a8063..d6bf58749 100644
--- a/man/dpkg-buildpackage.pod
+++ b/man/dpkg-buildpackage.pod
@@ -668,6 +668,11 @@ B<binary-targets>.
This variable is set to the Unix timestamp since the epoch of the
latest entry in I<debian/changelog>, if it is not already defined.
+=item B<SOURCE_BASE_DIR>
+
+This variable is set to the absolute pathname of the unpacked source
+directory, if it is not already defined (since dpkg 1.20.8).
+
=back
=head1 FILES
diff --git a/scripts/dpkg-buildpackage.pl b/scripts/dpkg-buildpackage.pl
index aacb83118..dbff851ce 100755
--- a/scripts/dpkg-buildpackage.pl
+++ b/scripts/dpkg-buildpackage.pl
@@ -23,6 +23,7 @@
use strict;
use warnings;
+use Cwd;
use File::Temp qw(tempdir);
use File::Basename;
use File::Copy;
@@ -460,6 +461,8 @@ if ($changedby) {
# <https://reproducible-builds.org/specs/source-date-epoch/>
$ENV{SOURCE_DATE_EPOCH} ||= $changelog->{timestamp} || time;
+$ENV{SOURCE_BASE_DIR} ||= getcwd();
+
my @arch_opts;
push @arch_opts, ('--host-arch', $host_arch) if $host_arch;
push @arch_opts, ('--host-type', $host_type) if $host_type;
diff --git a/scripts/mk/pkg-info.mk b/scripts/mk/pkg-info.mk
index bccde2317..f0568373c 100644
--- a/scripts/mk/pkg-info.mk
+++ b/scripts/mk/pkg-info.mk
@@ -12,6 +12,8 @@
# SOURCE_DATE_EPOCH: source release date as seconds since the epoch, as
# specified by <https://reproducible-builds.org/specs/source-date-epoch/>
# (since dpkg 1.18.8).
+# SOURCE_BASE_DIR: source base directory, as an absolute pathname, so that
+# upstream code can use in a generic way (since dpkg 1.20.8).
dpkg_late_eval ?= $(or $(value DPKG_CACHE_$(1)),$(eval DPKG_CACHE_$(1) := $(shell $(2)))$(value DPKG_CACHE_$(1)))
@@ -23,5 +25,7 @@ DEB_VERSION_UPSTREAM = $(call dpkg_late_eval,DEB_VERSION_UPSTREAM,echo '$(DEB_VE
DEB_DISTRIBUTION = $(call dpkg_late_eval,DEB_DISTRIBUTION,dpkg-parsechangelog -SDistribution)
SOURCE_DATE_EPOCH ?= $(call dpkg_late_eval,SOURCE_DATE_EPOCH,dpkg-parsechangelog -STimestamp)
+SOURCE_BASE_DIR ?= $(CURDIR)
export SOURCE_DATE_EPOCH
+export SOURCE_BASE_DIR
diff --git a/scripts/t/dpkg_buildpackage.t b/scripts/t/dpkg_buildpackage.t
index dd4f67a6f..3aff3117a 100644
--- a/scripts/t/dpkg_buildpackage.t
+++ b/scripts/t/dpkg_buildpackage.t
@@ -50,6 +50,10 @@ delete $ENV{$_} foreach grep { m/^DEB_/ } keys %ENV;
chdir $tmpdir;
+my $tmpl_source_canary = <<'TMPL_SOURCE_CANARY';
+Canary for the SOURCE_BASE_DIR support.
+TMPL_SOURCE_CANARY
+
my $tmpl_format = <<'TMPL_FORMAT';
3.0 (native)
TMPL_FORMAT
@@ -84,11 +88,14 @@ DI := debian/${binary-name-all}
DA := debian/${binary-name-any}
clean:
+ test -e "$(SOURCE_BASE_DIR)/source-canary"
rm -f debian/files
rm -rf $(DI) $(DA)
build-indep:
+ test -e "$(SOURCE_BASE_DIR)/source-canary"
build-arch:
+ test -e "$(SOURCE_BASE_DIR)/source-canary"
build: build-indep build-arch
binary-indep: build-indep
@@ -147,6 +154,7 @@ sub gen_source
make_path("$dirname/debian/source");
+ gen_from_tmpl("$dirname/source-canary", $tmpl_source_canary, $substvars);
gen_from_tmpl("$dirname/debian/source/format", $tmpl_format, $substvars);
gen_from_tmpl("$dirname/debian/changelog", $tmpl_changelog, $substvars);
gen_from_tmpl("$dirname/debian/control", $tmpl_control, $substvars);
diff --git a/t-func/atlocal.in b/t-func/atlocal.in
index 1e77669fa..bcea3c678 100644
--- a/t-func/atlocal.in
+++ b/t-func/atlocal.in
@@ -19,6 +19,9 @@ export TZ
SOURCE_DATE_EPOCH=0
export SOURCE_DATE_EPOCH
+SOURCE_BASE_DIR="@abs_top_srcdir@"
+export SOURCE_BASE_DIR
+
# Cleanup variables that might affect the tests.
unset GZIP
unset BZIP
--
2.30.0
Reply to: