Re: New dh_python proposal
On Fri, 09 Jun 2006, Raphael Hertzog wrote:
> This dh_python depends neither on python-central nor on python-support but it
> can work with both.
>
> It will use python-central if it appears in a build-dependency.
> It will use python-support if it detects /usr/share/python-support/$package.
>
> If the XS-Python-Version field is present, it will work following the new
> policy otherwise it should follow the old policy.
>
> At least that's the theory. I did some tests on two recent packages with
> python-central and python-support. I'd like people to try to recompile
> some actual python-related packages modules with that dh_python and check
> if it's really enough backwards compatible.
OK, I did test it myself finally this morning (with regular feedback from
Matthias as well). It leads me to some corrections so please find an
updated (and final!) dh_python in attachment.
Matthias and Josselin, please say clearly here in the bug report that
you are OK with this dh_python implementation so that Joey can confidently
integrate it in debhelper RSN.
[ Joey Hess ]
> So I'm confused, why do we have competing implementations here?
Because Matthias and Josselin do not (or can't or don't want to) work
together. I tried my best to fill the gap. :-/
FWIW, my opinion is that the BoF at Debconf clearly decided of a new
python policy but didn't explicitely choose an implementation. At that
time, only python-support was working and integrated in unstable.
We all knew doko's ambitious plan with python-central and since we didn't
discuss the implementation, there was a kind of implicit agreement that we
would use whatever would be ready in time for the python2.4 switch
(with many people convincend that python-central wouldn't be ready in
time).
Now, we have two working "python helper tool" but python-central has
not yet been widely tested.
If you check the code you'll see that the python-(support|central)
specific code is very limited and mainly consists on adding postinst
script snippet and dependencies when needed. I tried my best to have a
dh_python which is agnostic but still effective. :-)
I hope you understand better the situation and I really hope that you'll
accept this dh_python so that we can effectively start moving to python2.4
by default RSN.
Cheers,
--
Raphaël Hertzog
Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/
--- dh_python.orig 2006-06-08 10:57:59.000000000 +0200
+++ dh_python 2006-06-09 12:45:52.000000000 +0200
@@ -21,7 +21,42 @@
will also add a postinst and a prerm script if required.
The program will look at python scripts and modules in your package, and
-will use this information to generate a dependency on python, with the
+will use this information to generate adequate dependencies. There is two
+scenarios: if the package uses the XS-Python-Version field then
+the new policy will be applied, otherwise the old policy will be used.
+
+If dh_python detects "python-central" in one of the build-dependencies,
+then it will automatically use it: it will be added in the dependencies
+and corresponding postinst/postrm scripts are generated.
+
+If dh_python detects that you have a directory
+/usr/share/python-support/$PACKAGE, then it will automatically generate the
+required postinst/postrm scripts.
+
+=head2 New policy
+
+The XS-Python-Version field (on the source package) defines which Python
+versions are supported by the package. It can be "all", "current",
+"current, >= X.Y", a single version ("2.3") or a list of versions with
+optional comparison operators (ex: "2.3, 2.4" or ">= 2.2, << 2.5" or
+">= 2.4").
+
+The binary packages should have a "XB-Python-Version: ${python:Versions}"
+field and dh_python will generate the right substvar for that. The
+resulting value can be "all" for public modules which work with all python
+versions, "current" for private modules which are always byte-compiled
+with the current python version or a list of of all versions for which the
+extensions have been compiled (ex: "2.3, 2.4"). The dependencies are
+adjusted accordingly as well.
+
+Packages with public extensions should also have a "Provides:
+${python:Provides}" field. The corresponding substvar will indicate
+"pythonX.Y-foo, pythonX.Z-foo" according to all the extensions
+effectively available in the package.
+
+=head2 Old policy
+
+It looks at scripts and modules in your package and adds a dependency on python, with the
current major version, or on pythonX.Y if your scripts or modules need a
specific python version. The dependency will be substituted into your
package's control file wherever you place the token "${python:Depends}".
@@ -32,6 +67,8 @@
If you use this program, your package should build-depend on python.
+Note: in the old policy, /usr/lib/site-python is also scanned for modules.
+
=head1 OPTIONS
=over 4
@@ -40,11 +77,10 @@
If your package installs python modules in non-standard directories, you
can make dh_python check those directories by passing their names on the
-command line. By default, it will check /usr/lib/site-python,
-/usr/lib/$PACKAGE, /usr/share/$PACKAGE, /usr/lib/games/$PACKAGE,
+command line. By default, it will check /usr/lib/$PACKAGE, /usr/share/$PACKAGE, /usr/lib/games/$PACKAGE,
/usr/share/games/$PACKAGE and /usr/lib/python?.?/site-packages.
-Note: only /usr/lib/site-python, /usr/lib/python?.?/site-packages and the
+Note: only /usr/lib/python?.?/site-packages and the
extra names on the command line are searched for binary (.so) modules.
=item B<-V> I<version>
@@ -53,6 +89,9 @@
pythonX.Y version, you can use this option to specify the desired version,
such as 2.3. Do not use if you ship modules in /usr/lib/site-python.
+With the new policy, this option is mostly deprecated. Use the
+XS-Python-Field to indicate that you're using a specific python version.
+
=item B<-n>, B<--noscripts>
Do not modify postinst/postrm scripts.
@@ -85,10 +124,10 @@
}
# The next python version
-my $python_nextversion = $python_version + 0.1;
+my $python_nextversion = next_minor_version($python_version);
my $python_nextmajor = $python_major + 1;
-my @python_allversions = ('1.5','2.1','2.2','2.3','2.4');
+my @python_allversions = ('1.5','2.1','2.2','2.3','2.4','2.5');
foreach (@python_allversions) {
s/^/python/;
}
@@ -109,6 +148,19 @@
s#^/##;
}
+# Check if we have an XS-Python-Version and extract it from the control
+# file, also check if we use python-central
+my $python_header = "";
+my $use_python_central = 0;
+{
+ local $/ = ""; # Grab until empty line
+ open(CONTROL, "debian/control"); # Can't fail, dh_testdir has already been called
+ my $source = <CONTROL>;
+ $python_header = $1 if ($source =~ m/^XS-Python-Version: \s*(.*)$/m);
+ chomp($python_header);
+ $use_python_central = 1 if ($source =~ m/^Build-Depends(-Indep)?:.*\bpython-central\b/m);
+}
+
# dependency types
use constant PROGRAM => 1;
use constant PY_MODULE => 2;
@@ -120,29 +172,46 @@
my $tmp = tmpdir($package);
delsubstvar($package, "python:Depends");
+ delsubstvar($package, "python:Provides");
+ delsubstvar($package, "python:Versions");
- my @dirs = ("usr/lib/site-python", "usr/lib/$package", "usr/share/$package", "usr/lib/games/$package", "usr/share/games/$package", @ARGV );
- my @dirs_so = ("usr/lib/site-python", @ARGV );
+ my @dirs = ("usr/lib/$package", "usr/share/$package", "usr/lib/games/$package", "usr/share/games/$package", @ARGV );
+ my @dirs_so = (@ARGV);
my $dep_on_python = 0;
my $strong_dep = 0;
- my $look_for_pythonXY = 1;
+
+ # Fail early if the package use usr/lib/site-python
+ if (-d "$tmp/usr/lib/site-python") {
+ if ($python_header) {
+ # Error only on new policy
+ error("The package $package puts files in /usr/lib/site-python: forbidden by policy");
+ } else {
+ # Old policy allowed that directory, so scan it
+ push @dirs, "usr/lib/site-python";
+ push @dirs_so, "usr/lib/site-python";
+ }
+ }
+
+ if ($use_python_central) {
+ # Python-central will move files, that's why it's called
+ # so early in the loop
+ doit("pycentral debhelper $package $tmp >/dev/null");
+ }
# First, the case of python-foo and pythonX.Y-foo
if ($package =~ /^python-/) {
$dep_on_python = 1;
$strong_dep = 1;
+ # This adds a dependency to python<current>-foo in python-foo
+ # This is old policy but is done only if pythonX.Y-foo exists
+ # it does no harm and assure backwards compatibility
my $pack = $package;
$pack =~ s/^python/python$python_version/;
if (grep { "$_" eq "$pack" } getpackages()) {
addsubstvar($package, "python:Depends", $pack);
}
}
- if ($package !~ /^python[0-9].[0-9]-/) {
- push @dirs, "usr/lib/$usepython/site-packages";
- push @dirs_so, "usr/lib/$usepython/site-packages";
- $look_for_pythonXY = 0;
- }
@dirs = grep -d, map "$tmp/$_", @dirs;
@dirs_so = grep -d, map "$tmp/$_", @dirs_so;
@@ -202,8 +271,7 @@
if ($has_module) {
if ($dh{V_FLAG_SET}) {
$verdeps{$usepython} |= SO_MODULE_NONSTANDARD;
- }
- else {
+ } else {
$deps |= SO_MODULE;
}
}
@@ -212,9 +280,10 @@
# Dependencies on current python
$dep_on_python = 1 if $deps;
- $strong_dep = 1 if($deps & (PY_MODULE|SO_MODULE));
+ $strong_dep = 1 if ($deps & (PY_MODULE|SO_MODULE));
- if ($dep_on_python) {
+ if ($dep_on_python and not $python_header) {
+ # Generate dependencies following old policy
addsubstvar($package, "python:Depends", $python, ">= $python_version");
if ($strong_dep) {
addsubstvar($package, "python:Depends", $python, "<< $python_nextversion");
@@ -223,35 +292,54 @@
}
}
+ # At this point we have scanned everything for executables scripts
+ # and directories for private modules.
+
my $need_prerm = 0;
# Look for specific pythonX.Y modules
+ my %pyversions_found;
foreach my $pyver (@python_allversions) {
my $pydir="/usr/lib/$pyver/site-packages";
- if ($look_for_pythonXY) {
- if (grep -d,"$tmp$pydir") {
- find sub {
- return unless -f;
- if (/\.py$/) {
- $verdeps{$pyver} |= PY_MODULE;
- doit(("rm","-f",$_."c",$_."o"));
- }
- $verdeps{$pyver} |= SO_MODULE if /\.so$/;
- }, "$tmp$pydir";
- }
+ if (grep -d,"$tmp$pydir") {
+ find sub {
+ return unless -f;
+ if (/\.py$/) {
+ $verdeps{$pyver} |= PY_MODULE;
+ doit(("rm","-f",$_."c",$_."o"));
+ }
+ $verdeps{$pyver} |= SO_MODULE if /\.so$/;
+ }, "$tmp$pydir";
}
+ if ($verdeps{$pyver} & (PY_MODULE|SO_MODULE)) {
+ # Remember that files specific to this version have
+ # been found
+ $pyver =~ /^python([\d\.]+)/ and $pyversions_found{$1} = 1;
+ }
+
+ # Stop here when using new policy
+ next if $python_header;
+
# Go for the dependencies
addsubstvar($package, "python:Depends", $pyver) if $verdeps{$pyver};
# And now, the postinst and prerm stuff
+ # It takes care of byte-compiling modules
if ($pyver eq "$usepython") {
+ # Current version of python, also byte-compile the
+ # private directories (stored in $dirlist)
if ($verdeps{$pyver} & PY_MODULE) {
$pydir = $pydir.$dirlist;
} else {
+ # Only private directories
$pydir = $dirlist;
}
$verdeps{$pyver} |= PY_MODULE if($deps & PY_MODULE);
+ } else {
+ # Non-current python versions: we only have to
+ # byte-compile stuff in /usr/lib/python2.X
+ # Leave $pydir as is
}
if ($verdeps{$pyver} & (PY_MODULE|PY_MODULE_NONSTANDARD) && ! $dh{NOSCRIPTS}) {
autoscript($package,"postinst","postinst-python","s%#PYVER#%$pyver%;s%#DIRLIST#%$pydir%");
@@ -261,6 +349,208 @@
if ($need_prerm && ! $dh{NOSCRIPTS}) {
autoscript($package,"prerm","prerm-python","s%#PACKAGE#%$package%");
}
+
+ if ($python_header) {
+ # New policy: generate the depends to accept all python
+ # versions that this package effectively provides
+ my $min_version = "";
+ my $stop_version = "";
+ my $versions_field = "";
+
+ # Reset again, analysis using new policy follows
+ $dep_on_python = 0;
+
+ # Extracting min version from XS-Python-version
+ if ($python_header =~ /^current(?:,\s*>=\s*([\d\.]+))?/) {
+ if (defined $1) {
+ $min_version = $1;
+ }
+ $versions_field = "current";
+ } elsif ($python_header =~ />=\s*([\d\.]+)/) {
+ $min_version = $1;
+ }
+ if ($python_header =~ /<<\s*([\d\.]+)/) {
+ $stop_version = $1;
+ }
+
+ # Private extensions, must be rebuilt for each python version
+ if ($deps & SO_MODULE) {
+ $dep_on_python++;
+ $stop_version = next_minor_version($python_version);
+ # Packages using a private extension can only
+ # support one version and they indicate which one
+ # in XS-Python-Version
+ if ($versions_field eq "current") {
+ $versions_field = $python_version;
+ } else {
+ $versions_field = $python_header;
+ }
+ }
+
+ # Private modules
+ if ($deps & PY_MODULE) {
+ if (! $use_python_central) {
+ # Use python-support to ensure that it's always
+ # byte-compiled for the current version
+ mkdir("$tmp/usr/share/python-support");
+ $dirlist =~ s/^\s*//;
+ $dirlist =~ s/\s*$//;
+ open(DIRLIST, "> $tmp/usr/share/python-support/$package.dirs") ||
+ error("Can't create $tmp/usr/share/python-support/$package.dirs: $!");
+ print DIRLIST join("\n", split(/\s+/, $dirlist));
+ close(DIRLIST);
+ $versions_field = "current" unless $versions_field;
+ } else {
+ # Already handled by python-central
+ }
+ }
+
+ # Python scripts
+ if ($deps & PROGRAM) {
+ $dep_on_python++;
+ }
+
+ # Public extensions
+ # XXX: if we have *.py files left in /usr/lib/python2.X/site-packages
+ # they are no more byte-compiled... but that shouldn't happen if the
+ # package really conforms to the policy
+ if (scalar keys %pyversions_found) {
+ # Extensions will always trigger this (as well as public
+ # modules not handled by python-support/python-central)
+ $dep_on_python++;
+ $min_version = min(keys %pyversions_found);
+ unless ($stop_version) {
+ my $max_version = max(keys %pyversions_found);
+ $stop_version = next_minor_version($max_version);
+ }
+ # Generate the Python-Version field
+ foreach (keys %pyversions_found) {
+ addsubstvar($package, "python:Versions", $_);
+ }
+ # Generate provides for the python2.X-foo packages that we emulate
+ if ($package =~ /^python-/) {
+ foreach (keys %pyversions_found) {
+ my $virtual = $package;
+ $virtual =~ s/^python-/$python$_-/;
+ addsubstvar($package, "python:Provides", $virtual);
+ }
+ }
+ } else {
+ # Still try to describe the versions that the package support
+ $versions_field = "all" unless $versions_field;
+ addsubstvar($package, "python:Versions", $versions_field);
+ }
+
+ # Private extensions compiled with non-current python
+ # or scripts using non-current python
+ foreach my $pyver (keys %verdeps) {
+ if ($verdeps{$pyver} & (SO_MODULE_NONSTANDARD|PROGRAM)) {
+ addsubstvar($package, "python:Depends", $pyver);
+ # Heuristic: if we have pythonX.Y program and if we have precisely only that
+ # $pyversions_found, we don't need to depend on a specific interval of python
+ my @found = keys %pyversions_found;
+ if (($verdeps{$pyver} & PROGRAM) &&
+ (scalar @found == 1) &&
+ ("python$found[0]" eq $pyver)
+ ) {
+ $dep_on_python--; # Cancel the dep_on_python due to shared extensions
+ $min_version = "";
+ $stop_version = "";
+ }
+ }
+ }
+
+ if ($dep_on_python) {
+ # At least a script has been detected
+ addsubstvar($package, "python:Depends", $python, ">= $min_version") if $min_version;
+ # If a stronger dependency is needed
+ addsubstvar($package, "python:Depends", $python, "<< $stop_version") if $stop_version;
+ # Let's depend on python anyway
+ addsubstvar($package, "python:Depends", $python) unless ($min_version or $stop_version);
+ }
+
+ }
+
+ # Let's check for python-support
+ if (-d "$tmp/usr/share/python-support/$package") {
+ # Handle public modules
+ my $ps_dir = "/usr/share/python-support/$package";
+ addsubstvar($package, "python:Depends", "python-support");
+ if (! $dh{NOSCRIPTS}) {
+ autoscript($package, "postinst", "postinst-python-support", "s,#OPTIONS#,-i,;s,#DIRS#,$ps_dir,");
+ autoscript($package, "prerm", "prerm-python-support", "s,#OPTIONS#,-i,;s,#DIRS#,$ps_dir,");
+ }
+ # TODO: we should generate
+ # /usr/share/python-support/$package/version from
+ # $python_header
+ }
+ if (-f "$tmp/usr/share/python-support/$package.dirs") {
+ # Handle private modules
+ my $ps_dir = "/usr/share/python-support/$package.dirs";
+ addsubstvar($package, "python:Depends", "python-support");
+ if (! $dh{NOSCRIPTS}) {
+ autoscript($package, "postinst", "postinst-python-support", "s,#OPTIONS#,-b,;s,#DIRS#,$ps_dir,");
+ autoscript($package, "prerm", "prerm-python-support", "s,#OPTIONS#,-b,;s,#DIRS#,$ps_dir,");
+ }
+ }
+
+ # Let's check for python-central
+ if ($use_python_central and
+ # If we found private modules to byte-compile or if shared
+ # modules are available in /usr/share/pycentral or if we
+ # have public modules in /usr/lib/python2.X/site-packages, then we
+ # should depend on python-central and add the postinst/prerm
+ # scripts
+ (($deps & PY_MODULE) or
+ (-d "$tmp/usr/share/pycentral") or
+ (scalar keys %pyversions_found)
+ )
+ ) {
+ addsubstvar($package, "python:Depends", "python-central", ">= 0.4.7");
+ if (! $dh{NOSCRIPTS}) {
+ autoscript($package,"postinst","postinst-pycentral","s%#PACKAGE#%$package%");
+ autoscript($package,"prerm","prerm-pycentral","s%#PACKAGE#%$package%");
+ }
+ }
+
+}
+
+sub next_minor_version {
+ my $version = shift;
+ # Handles 2.10 -> 2.11 gracefully
+ my @items = split(/\./, $version);
+ $items[1] += 1;
+ $version = join(".", @items);
+ return $version;
+}
+
+sub compare_version {
+ my ($a, $b) = @_;
+ my @A = split(/\./, $a);
+ my @B = split(/\./, $b);
+ my $diff = 0;
+ for (my $i = 0; $i <= $#A; $i++) {
+ $diff = $A[$i] - $B[$i];
+ return $diff if $diff;
+ }
+ # They are the same
+ return 0;
+}
+
+sub max {
+ my $max = shift;
+ foreach (@_) {
+ $max = $_ if (compare_version($_, $max) > 0);
+ }
+ return $max;
+}
+
+sub min {
+ my $min = shift;
+ foreach (@_) {
+ $min = $_ if (compare_version($_, $min) < 0);
+ }
+ return $min;
}
=head1 SEE ALSO
@@ -269,9 +559,10 @@
This program is a part of debhelper.
-=head1 AUTHOR
+=head1 AUTHORS
Josselin Mouette <joss@debian.org>
+Raphael Hertzog <hertzog@debian.org>
most ideas stolen from Brendan O'Dea <bod@debian.org>
#!/usr/bin/perl -w
=head1 NAME
dh_python - calculates python dependencies and adds postinst and prerm python scripts
=cut
use strict;
use File::Find;
use Debian::Debhelper::Dh_Lib;
=head1 SYNOPSIS
B<dh_python> [S<I<debhelper options>>] [B<-n>] [B<-V> I<version>] [S<I<module dirs ...>>]
=head1 DESCRIPTION
dh_python is a debhelper program that is responsible for generating the
${python:Depends} substitutions and adding them to substvars files. It
will also add a postinst and a prerm script if required.
The program will look at python scripts and modules in your package, and
will use this information to generate adequate dependencies. There is two
scenarios: if the package uses the XS-Python-Version field then
the new policy will be applied, otherwise the old policy will be used.
If dh_python detects "python-central" in one of the build-dependencies,
then it will automatically use it: it will be added in the dependencies
and corresponding postinst/postrm scripts are generated.
If dh_python detects that you have a directory
/usr/share/python-support/$PACKAGE, then it will automatically generate the
required postinst/postrm scripts.
=head2 New policy
The XS-Python-Version field (on the source package) defines which Python
versions are supported by the package. It can be "all", "current",
"current, >= X.Y", a single version ("2.3") or a list of versions with
optional comparison operators (ex: "2.3, 2.4" or ">= 2.2, << 2.5" or
">= 2.4").
The binary packages should have a "XB-Python-Version: ${python:Versions}"
field and dh_python will generate the right substvar for that. The
resulting value can be "all" for public modules which work with all python
versions, "current" for private modules which are always byte-compiled
with the current python version or a list of of all versions for which the
extensions have been compiled (ex: "2.3, 2.4"). The dependencies are
adjusted accordingly as well.
Packages with public extensions should also have a "Provides:
${python:Provides}" field. The corresponding substvar will indicate
"pythonX.Y-foo, pythonX.Z-foo" according to all the extensions
effectively available in the package.
=head2 Old policy
It looks at scripts and modules in your package and adds a dependency on python, with the
current major version, or on pythonX.Y if your scripts or modules need a
specific python version. The dependency will be substituted into your
package's control file wherever you place the token "${python:Depends}".
If some modules need to be byte-compiled at install time, appropriate
postinst and prerm scripts will be generated. If already byte-compiled
modules are found, they are removed.
If you use this program, your package should build-depend on python.
Note: in the old policy, /usr/lib/site-python is also scanned for modules.
=head1 OPTIONS
=over 4
=item I<module dirs>
If your package installs python modules in non-standard directories, you
can make dh_python check those directories by passing their names on the
command line. By default, it will check /usr/lib/$PACKAGE, /usr/share/$PACKAGE, /usr/lib/games/$PACKAGE,
/usr/share/games/$PACKAGE and /usr/lib/python?.?/site-packages.
Note: only /usr/lib/python?.?/site-packages and the
extra names on the command line are searched for binary (.so) modules.
=item B<-V> I<version>
If the .py files your package ships are meant to be used by a specific
pythonX.Y version, you can use this option to specify the desired version,
such as 2.3. Do not use if you ship modules in /usr/lib/site-python.
With the new policy, this option is mostly deprecated. Use the
XS-Python-Field to indicate that you're using a specific python version.
=item B<-n>, B<--noscripts>
Do not modify postinst/postrm scripts.
=back
=head1 CONFORMS TO
Debian policy, version 3.5.7
Python policy, version 0.3.7
=cut
init();
my $python = 'python';
# The current python major version
my $python_major;
my $python_version = `$python -V 2>&1`;
if (! defined $python_version || $python_version eq "") {
error("Python is not installed, aborting. (Probably forgot to Build-Depend on python.)");
}
elsif ($python_version =~ m/^Python\s+(\d+)\.(\d+)(\.\d+)*/) {
$python_version = "$1.$2" ;
$python_major = $1 ;
} else {
error("Unable to parse python version out of \"$python_version\".");
}
# The next python version
my $python_nextversion = next_minor_version($python_version);
my $python_nextmajor = $python_major + 1;
my @python_allversions = ('1.5','2.1','2.2','2.3','2.4','2.5');
foreach (@python_allversions) {
s/^/python/;
}
# Check for -V
my $usepython = "python$python_version";
if($dh{V_FLAG_SET}) {
$usepython = $dh{V_FLAG};
$usepython =~ s/^/python/;
if (! grep { $_ eq $usepython } @python_allversions) {
error("Unknown python version $dh{V_FLAG}");
}
}
# Cleaning the paths given on the command line
foreach (@ARGV) {
s#/$##;
s#^/##;
}
# Check if we have an XS-Python-Version and extract it from the control
# file, also check if we use python-central
my $python_header = "";
my $use_python_central = 0;
{
local $/ = ""; # Grab until empty line
open(CONTROL, "debian/control"); # Can't fail, dh_testdir has already been called
my $source = <CONTROL>;
$python_header = $1 if ($source =~ m/^XS-Python-Version: \s*(.*)$/m);
chomp($python_header);
$use_python_central = 1 if ($source =~ m/^Build-Depends(-Indep)?:.*\bpython-central\b/m);
}
# dependency types
use constant PROGRAM => 1;
use constant PY_MODULE => 2;
use constant PY_MODULE_NONSTANDARD => 4;
use constant SO_MODULE => 8;
use constant SO_MODULE_NONSTANDARD => 16;
foreach my $package (@{$dh{DOPACKAGES}}) {
my $tmp = tmpdir($package);
delsubstvar($package, "python:Depends");
delsubstvar($package, "python:Provides");
delsubstvar($package, "python:Versions");
my @dirs = ("usr/lib/$package", "usr/share/$package", "usr/lib/games/$package", "usr/share/games/$package", @ARGV );
my @dirs_so = (@ARGV);
my $dep_on_python = 0;
my $strong_dep = 0;
# Fail early if the package use usr/lib/site-python
if (-d "$tmp/usr/lib/site-python") {
if ($python_header) {
# Error only on new policy
error("The package $package puts files in /usr/lib/site-python: forbidden by policy");
} else {
# Old policy allowed that directory, so scan it
push @dirs, "usr/lib/site-python";
push @dirs_so, "usr/lib/site-python";
}
}
if ($use_python_central) {
# Python-central will move files, that's why it's called
# so early in the loop
doit("pycentral debhelper $package $tmp >/dev/null");
}
# First, the case of python-foo and pythonX.Y-foo
if ($package =~ /^python-/) {
$dep_on_python = 1;
$strong_dep = 1;
# This adds a dependency to python<current>-foo in python-foo
# This is old policy but is done only if pythonX.Y-foo exists
# it does no harm and assure backwards compatibility
my $pack = $package;
$pack =~ s/^python/python$python_version/;
if (grep { "$_" eq "$pack" } getpackages()) {
addsubstvar($package, "python:Depends", $pack);
}
}
@dirs = grep -d, map "$tmp/$_", @dirs;
@dirs_so = grep -d, map "$tmp/$_", @dirs_so;
my $deps = 0;
my %verdeps = ();
foreach (@python_allversions) {
$verdeps{$_} = 0;
}
# Find scripts
find sub {
return unless -f and (-x or /\.py$/);
local *F;
return unless open F, $_;
if (read F, local $_, 32 and m%^#!\s*/usr/bin/(env\s+)?(python(\d+\.\d+)?)\s%) {
if ( "python" eq $2 ) {
$deps |= PROGRAM;
} elsif(defined $verdeps{$2}) {
$verdeps{$2} |= PROGRAM;
}
}
close F;
}, $tmp;
# Look for python modules
my $dirlist="";
if (@dirs) {
foreach my $curdir (@dirs) {
my $has_module = 0;
$curdir =~ s%^$tmp/%%;
find sub {
return unless -f;
if (/\.py$/) {
$has_module = 1;
doit(("rm","-f",$_."c",$_."o"));
}
}, "$tmp/$curdir" ;
if ($has_module) {
if ($dh{V_FLAG_SET}) {
$verdeps{$usepython} |= PY_MODULE_NONSTANDARD;
} else {
$deps |= PY_MODULE;
}
$dirlist="$dirlist /$curdir";
}
}
}
if (@dirs_so) {
foreach my $curdir (@dirs_so) {
my $has_module = 0;
$curdir =~ s%^$tmp/%%;
find sub {
return unless -f;
$has_module = 1 if /\.so$/;
}, "$tmp/$curdir" ;
if ($has_module) {
if ($dh{V_FLAG_SET}) {
$verdeps{$usepython} |= SO_MODULE_NONSTANDARD;
} else {
$deps |= SO_MODULE;
}
}
}
}
# Dependencies on current python
$dep_on_python = 1 if $deps;
$strong_dep = 1 if ($deps & (PY_MODULE|SO_MODULE));
if ($dep_on_python and not $python_header) {
# Generate dependencies following old policy
addsubstvar($package, "python:Depends", $python, ">= $python_version");
if ($strong_dep) {
addsubstvar($package, "python:Depends", $python, "<< $python_nextversion");
} else {
addsubstvar($package, "python:Depends", $python, "<< $python_nextmajor");
}
}
# At this point we have scanned everything for executables scripts
# and directories for private modules.
my $need_prerm = 0;
# Look for specific pythonX.Y modules
my %pyversions_found;
foreach my $pyver (@python_allversions) {
my $pydir="/usr/lib/$pyver/site-packages";
if (grep -d,"$tmp$pydir") {
find sub {
return unless -f;
if (/\.py$/) {
$verdeps{$pyver} |= PY_MODULE;
doit(("rm","-f",$_."c",$_."o"));
}
$verdeps{$pyver} |= SO_MODULE if /\.so$/;
}, "$tmp$pydir";
}
if ($verdeps{$pyver} & (PY_MODULE|SO_MODULE)) {
# Remember that files specific to this version have
# been found
$pyver =~ /^python([\d\.]+)/ and $pyversions_found{$1} = 1;
}
# Stop here when using new policy
next if $python_header;
# Go for the dependencies
addsubstvar($package, "python:Depends", $pyver) if $verdeps{$pyver};
# And now, the postinst and prerm stuff
# It takes care of byte-compiling modules
if ($pyver eq "$usepython") {
# Current version of python, also byte-compile the
# private directories (stored in $dirlist)
if ($verdeps{$pyver} & PY_MODULE) {
$pydir = $pydir.$dirlist;
} else {
# Only private directories
$pydir = $dirlist;
}
$verdeps{$pyver} |= PY_MODULE if($deps & PY_MODULE);
} else {
# Non-current python versions: we only have to
# byte-compile stuff in /usr/lib/python2.X
# Leave $pydir as is
}
if ($verdeps{$pyver} & (PY_MODULE|PY_MODULE_NONSTANDARD) && ! $dh{NOSCRIPTS}) {
autoscript($package,"postinst","postinst-python","s%#PYVER#%$pyver%;s%#DIRLIST#%$pydir%");
$need_prerm = 1;
}
}
if ($need_prerm && ! $dh{NOSCRIPTS}) {
autoscript($package,"prerm","prerm-python","s%#PACKAGE#%$package%");
}
if ($python_header) {
# New policy: generate the depends to accept all python
# versions that this package effectively provides
my $min_version = "";
my $stop_version = "";
my $versions_field = "";
# Reset again, analysis using new policy follows
$dep_on_python = 0;
# Extracting min version from XS-Python-version
if ($python_header =~ /^current(?:,\s*>=\s*([\d\.]+))?/) {
if (defined $1) {
$min_version = $1;
}
$versions_field = "current";
} elsif ($python_header =~ />=\s*([\d\.]+)/) {
$min_version = $1;
}
if ($python_header =~ /<<\s*([\d\.]+)/) {
$stop_version = $1;
}
# Private extensions, must be rebuilt for each python version
if ($deps & SO_MODULE) {
$dep_on_python++;
$stop_version = next_minor_version($python_version);
# Packages using a private extension can only
# support one version and they indicate which one
# in XS-Python-Version
if ($versions_field eq "current") {
$versions_field = $python_version;
} else {
$versions_field = $python_header;
}
}
# Private modules
if ($deps & PY_MODULE) {
if (! $use_python_central) {
# Use python-support to ensure that it's always
# byte-compiled for the current version
mkdir("$tmp/usr/share/python-support");
$dirlist =~ s/^\s*//;
$dirlist =~ s/\s*$//;
open(DIRLIST, "> $tmp/usr/share/python-support/$package.dirs") ||
error("Can't create $tmp/usr/share/python-support/$package.dirs: $!");
print DIRLIST join("\n", split(/\s+/, $dirlist));
close(DIRLIST);
$versions_field = "current" unless $versions_field;
} else {
# Already handled by python-central
}
}
# Python scripts
if ($deps & PROGRAM) {
$dep_on_python++;
}
# Public extensions
# XXX: if we have *.py files left in /usr/lib/python2.X/site-packages
# they are no more byte-compiled... but that shouldn't happen if the
# package really conforms to the policy
if (scalar keys %pyversions_found) {
# Extensions will always trigger this (as well as public
# modules not handled by python-support/python-central)
$dep_on_python++;
$min_version = min(keys %pyversions_found);
unless ($stop_version) {
my $max_version = max(keys %pyversions_found);
$stop_version = next_minor_version($max_version);
}
# Generate the Python-Version field
foreach (keys %pyversions_found) {
addsubstvar($package, "python:Versions", $_);
}
# Generate provides for the python2.X-foo packages that we emulate
if ($package =~ /^python-/) {
foreach (keys %pyversions_found) {
my $virtual = $package;
$virtual =~ s/^python-/$python$_-/;
addsubstvar($package, "python:Provides", $virtual);
}
}
} else {
# Still try to describe the versions that the package support
$versions_field = "all" unless $versions_field;
addsubstvar($package, "python:Versions", $versions_field);
}
# Private extensions compiled with non-current python
# or scripts using non-current python
foreach my $pyver (keys %verdeps) {
if ($verdeps{$pyver} & (SO_MODULE_NONSTANDARD|PROGRAM)) {
addsubstvar($package, "python:Depends", $pyver);
# Heuristic: if we have pythonX.Y program and if we have precisely only that
# $pyversions_found, we don't need to depend on a specific interval of python
my @found = keys %pyversions_found;
if (($verdeps{$pyver} & PROGRAM) &&
(scalar @found == 1) &&
("python$found[0]" eq $pyver)
) {
$dep_on_python--; # Cancel the dep_on_python due to shared extensions
$min_version = "";
$stop_version = "";
}
}
}
if ($dep_on_python) {
# At least a script has been detected
addsubstvar($package, "python:Depends", $python, ">= $min_version") if $min_version;
# If a stronger dependency is needed
addsubstvar($package, "python:Depends", $python, "<< $stop_version") if $stop_version;
# Let's depend on python anyway
addsubstvar($package, "python:Depends", $python) unless ($min_version or $stop_version);
}
}
# Let's check for python-support
if (-d "$tmp/usr/share/python-support/$package") {
# Handle public modules
my $ps_dir = "/usr/share/python-support/$package";
addsubstvar($package, "python:Depends", "python-support");
if (! $dh{NOSCRIPTS}) {
autoscript($package, "postinst", "postinst-python-support", "s,#OPTIONS#,-i,;s,#DIRS#,$ps_dir,");
autoscript($package, "prerm", "prerm-python-support", "s,#OPTIONS#,-i,;s,#DIRS#,$ps_dir,");
}
# TODO: we should generate
# /usr/share/python-support/$package/version from
# $python_header
}
if (-f "$tmp/usr/share/python-support/$package.dirs") {
# Handle private modules
my $ps_dir = "/usr/share/python-support/$package.dirs";
addsubstvar($package, "python:Depends", "python-support");
if (! $dh{NOSCRIPTS}) {
autoscript($package, "postinst", "postinst-python-support", "s,#OPTIONS#,-b,;s,#DIRS#,$ps_dir,");
autoscript($package, "prerm", "prerm-python-support", "s,#OPTIONS#,-b,;s,#DIRS#,$ps_dir,");
}
}
# Let's check for python-central
if ($use_python_central and
# If we found private modules to byte-compile or if shared
# modules are available in /usr/share/pycentral or if we
# have public modules in /usr/lib/python2.X/site-packages, then we
# should depend on python-central and add the postinst/prerm
# scripts
(($deps & PY_MODULE) or
(-d "$tmp/usr/share/pycentral") or
(scalar keys %pyversions_found)
)
) {
addsubstvar($package, "python:Depends", "python-central", ">= 0.4.7");
if (! $dh{NOSCRIPTS}) {
autoscript($package,"postinst","postinst-pycentral","s%#PACKAGE#%$package%");
autoscript($package,"prerm","prerm-pycentral","s%#PACKAGE#%$package%");
}
}
}
sub next_minor_version {
my $version = shift;
# Handles 2.10 -> 2.11 gracefully
my @items = split(/\./, $version);
$items[1] += 1;
$version = join(".", @items);
return $version;
}
sub compare_version {
my ($a, $b) = @_;
my @A = split(/\./, $a);
my @B = split(/\./, $b);
my $diff = 0;
for (my $i = 0; $i <= $#A; $i++) {
$diff = $A[$i] - $B[$i];
return $diff if $diff;
}
# They are the same
return 0;
}
sub max {
my $max = shift;
foreach (@_) {
$max = $_ if (compare_version($_, $max) > 0);
}
return $max;
}
sub min {
my $min = shift;
foreach (@_) {
$min = $_ if (compare_version($_, $min) < 0);
}
return $min;
}
=head1 SEE ALSO
L<debhelper(7)>
This program is a part of debhelper.
=head1 AUTHORS
Josselin Mouette <joss@debian.org>
Raphael Hertzog <hertzog@debian.org>
most ideas stolen from Brendan O'Dea <bod@debian.org>
=cut
Reply to: