[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

RFC: dh_splitinstall in kde packaging



Hello!

I have mostly complete dh_splitinstall. It works as filter on list of installed files. This list is extracted from make install, so it can gather also originating source directory and filter on it. Then it gets list of all files in installation directory and adds ones, that it didn't find in make output. It also reads override files (debian/*.inst_over) and removes these from list of files to be processed. Then it reads control file (debian/splitinstall) containing filtering information (regexps on source and destination filename and on file mode, but last one is still mostly unusable). It runs all files through filters, ones that get a match are immediatly removed from further processing (we don't want same file in more packages, do we?). It creates *.install file for every package and appends respective .inst_over file to it.

Now why all this? I think it can save considerable amount of work, because filters are much more flexible than hard-coded file-lists. Almost every time i rebuild packages from cvs, some of .install files lists non-existent files or misses some needed one. IMHO dh_splitinstall, even if it is not 100%, can do much better job with updated sources.

One thing it cannot do atm is separating *.so symlinks into -dev packages, as there are many *.so binaries in kde. If desired, i can make it work, but IMHO it is not worth the effort.

So far i converted these packages:
kdebase
koffice
kdemultimedia
kdenetwork

I'll submit patches against kde-cvs from last weekend in separate e-mails. They contain whole script, thus they are not usable for commiting, just for testing the thing. If respective maintainers show interest in them, i can get them in better shape (both patches and script).

Comments?

yenar

PS: Attached is newest version of the script... if you want to play with it, use this version, some patches contain older revisions.

--
-----------------------------------------------------------------------
inetname: Yenar Calentaure
realname: Peter Rockai
    mail: yenar(at)host.sk
homepage: http://yenar.host.sk
-----------------------------------------------------------------------
The universe is entering maintenance mode in 2 minutes. Please logout.
                                     -- Your administrator
-----------------------------------------------------------------------
#!/usr/bin/perl
###{{{###############################################################
# Copyright (c) 2002                                                #
#     Peter Rockai (yenar) <yenar@host.sk>                          #
#                                                                   #
# This program is free software; you can redistribute it and/or     #
# modify it under the terms of the GNU General Public               #
# License as published by the Free Software Foundation; either      #
# version 2 of the License, or (at your option) any later version.  #
#                                                                   #
# This program is distributed in the hope that it will be useful,   #
# but WITHOUT ANY WARRANTY; without even the implied warranty of    #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU #
# General Public License for more details.                          #
#                                                                   #
# You should have received a copy of the GNU General Public License #
# along with this program; see the file COPYING.  If not, write to  #
# the Free Software Foundation, Inc., 59 Temple Place - Suite 330,  #
# Boston, MA 02111-1307, USA.                                       #
###############################################################}}}###

use Debian::Debhelper::Dh_Lib;

&init;


$makeoutput = "debian/make_install.debhelper"; #FIXME
$override = `cat debian/*.inst_over`; # FIXME
@override = split /\n/, $override;

# {{{ make sense of commandline args
$dir = $dh{U_PARAMS}[0];
$dest = $dh{TMPDIR};
$dest = "debian/tmp/" unless $dest;
$ctrl = pkgfile ($dh{MAINPACKAGE}, "splitinstall");
die "oops" unless $ctrl;
$wd = &trim(`pwd`);
$dest = &trim("$wd/$dest");
print STDERR "$dir -> $dest\n";
# }}}

##### helper subroutines
# {{{ trim: trim whitespace from beginning/end of string
sub trim {
	my $a = shift;
	$a =~ s/^[ \t\n]+//;
	$a =~ s/[ \t\n]+$//;
	return $a;
}
# }}}
# {{{ norm: normalize file name
sub norm {
	my $d = shift;
	my @d = split /\//, $d;
	my @nd;
	my $nd;
	for (@d) {
		if ($_ eq ".." and scalar @nd > 0) {
			pop @nd;
		} else {
			push @nd, $_;
		}
	}
	my $i = 0;
	for (@nd) {
		$nd .= "$_";
		$nd .= "/" unless $i + 1 == scalar @nd;
		$i ++;
	}
	$nd =~ s!//*!/!g;
	return $nd;
}
# }}}
# {{{ nowd: no working directory name of file
sub nowd {
	my $f = shift;
	$f =~ s!^$wd/?!!sm;
	return &trim ($f);
}
# }}}
# {{{ getmod: extract textual mode description from octal form
sub getmod {
	my $_m = shift;
	my @m = split //, $_m;
	my @mg = ("owner", "group", "world");
	my @mp = ("execute", "write", "read");
	my $ms;
	my $i = 0;
	#print STDERR "mode: $_m\n";
	while ($i < 3) {
		$_ = $m [$i];
		my $j = 0;
		while ($j < 3) {
			#print STDERR "testing: $i, $j (${mg[$i]}-${mp[$j]})\n";
			$ms .= "${mg[$i]}-${mp[$j]} " if ($_ & (1 << $j));
			$j ++;
		}
		$i ++;
	}
	return &trim ($ms);
}
# }}}
# {{{ getf: get file record reference
sub getf {
	my %f;
	$f{mode} = shift;
	$sd = shift;
	$src = shift;
	$dst = shift;
	$f{src} = &norm ("$sd/$src");
	$f{dest} = &nowd (&norm ($dst));
	if ( -d $f{dest} && $f{mode} !~ /symlink/) {
		my $fn = $src;
		$fn =~ s#^.*/(.*?)$#$1#;
		$f{dest} .= "/$fn";
		$f{dest} = &norm ($f{dest});
	}
	return \%f;
}
# }}} 

##### main program
# {{{ global variables
# eat up parameters from command
$_p = "(?:-[a-zA-Z0-9]| )*";
my @files;
my $i;
# }}}
# {{{ read control file
open CONTROL, "<$ctrl";
while (<CONTROL>)
{
	$in .= $_;
}
close CONTROL;
$in =~ s/^#.*?\n//gsm;
@pkgs = split /\n[ \t]*\n/ms, $in;
$opts = shift @pkgs;
$opts =~ /^Install-To: (.*?)$/smi;
$dest = $1 if ($1);

$i = 0;
while ($i < scalar @pkgs) {
	my $pkg = $pkgs[$i];
	if ($pkg =~ /^package:(.*?)$/smi) {
		my $pk = &trim ($1);
		print STDERR "package $i ($pk) is ok\n";
		push @pkgnames, $pk;
		$i ++;
	} else {
		print STDERR "package $i is bad\n";
		splice (@pkgs, $i, 1);
	}
}
#sort @pkgnames;
$i = 0;
while ($i < scalar @pkgnames) {
	my $p = $pkgnames [$i - 1];
	my $c = $pkgnames [$i];
	if ($p eq $c) {
		splice (@pkgnames, $i, 1);
	} else {
		$i ++;
	}
}
for (@pkgnames) {
	print STDERR "truncating debian/$_.install\n";
	truncate "debian/$_.install", 0;
}
# }}}
# {{{ read 'make install' output
# is this acceptable speed-wise?
open MAKE, $makeoutput;
while (<MAKE>) {
	$make .= $_;
}
close MAKE;
$make =~ s/\\\n/ /g;
$make =~ s/;\n/\n/g; # ERR...
@lines = split /\n/, $make;
# }}}
# {{{ iterate over lines of make output, build file list
$i = 0;
while ($i < scalar @lines) {
	$_ = $lines [$i];

	if (/^[ \t]*[\/a-zA-Z]*install$_p-m ([0-7]{3,4})$_p(.+?) (.+?)$/) {
		push @files, &getf (&getmod ($1), $sd, $2, $3);
	}
	elsif (/^[ \t]*[\/a-zA-Z]*install$_p(.+?) (.+?)$/) {
		push @files, &getf ("unknown", $sd, $1, $2);
	}
	elsif (/ln[ \t]+-s[ \t]+(.+?)[ \t]+(.+?)$/) {
		push @files, &getf ("symlink", $sd, $1, $2);
	}
	elsif (/libtool.*?--mode=install [a-zA-Z\/.]*install $_p(?:-m [0-7]{3,4})$_p(.+?)[ \t]+(.+?)$/ ||
		/libtool.*?--mode=install [a-zA-Z\/.]*install $_p(.+?)[ \t]+(.+?)$/) {
		my $f = &getf ("libtool execute", $sd, $1, $2);
		my $destd = $$f{dest};
		$destd =~ s#^(.*)/.*?$#$1#x;
		push @files, $f;
		if ($$f{src} =~ /\.la$/) {
			my ($names, @names);
			$names = `grep library_names $$f{src}`;
			$names =~ /library_names[ \t]*=[ \t]*'(.*?)'/;
			@names = split /[ \t]/, $1;
			for (@names) {
				push @files, &getf ("libtool execute slave",
					$sd, $_, "$destd/$_");
			}
		}
	}

	if (/make\[[0-9]+\]: Entering directory `(.+?)'/) {
		# make changed source directory
		$sd = $1;
	}
	$i ++;
}
# }}}
# {{{ find uncaptured files, append them to file list
$allf = `find $dest -type f`;
@allf = split /\n/, $allf;
for (@allf) {
	push @files, &getf ("z-noncaptured", "", "", &trim($_));
}
# }}}
# {{{ postprocess file list (sort, remove dups)
@files = sort {($$a{dest}) cmp ($$b{dest}) ||
	$$a{mode} cmp $$b{mode}} @files;
$i = 1;
while ($i < scalar @files) {
	my $p = $files [$i - 1];
	my $f = $files [$i];
	if ($$p{dest} eq $$f{dest}) {
		splice (@files, $i, 1);
	} elsif (scalar grep {$_ =~ $$f{dest}} @override) {
		splice (@files, $i, 1);
	} else {
		$i ++;
	}
}
# }}}
# {{{ iterate over control file sections
for (@pkgs) {
	my (@m_mod, @x_mod, @m_src, @x_src, @m_dst, @x_dst, @pkg);
	my $ref;
	my $sd = "";
	my %f;
	# split up section into lines
	@pl = split /\n/;
	for (@pl) {
		if (/^(.+?):(.*?)$/) {
			$tag = $1;
			$val = &trim ($2);
			$_ = $tag;
			if (/^package/i) {
				$ref = \@pkg;
			} elsif (/^match/i) {
				$ref = \@m_mod if /mode$/i;
				$ref = \@m_src if /source$/i;
				$ref = \@m_dst if /dest$/i;
			} elsif (/^exclude/i) {
				$ref = \@x_mod if /mode$/i;
				$ref = \@x_src if /source$/i;
				$ref = \@x_dst if /dest$/i;
			} else {
				print STDERR "warning: unknown tag!\n";
			}
			print STDERR "$tag: $val (using $ref)\n";
			push @$ref, $val;
		} else {
			die "Parse error\n";
		}
	}
	print STDERR "Package: ${pkg[0]}\n";
	open OUT, ">>debian/${pkg[0]}.install";
	my $i = 0;
	while ($i < scalar @files) {
		%f = %{$files[$i]};
		my $ok = 1;
		foreach (@m_src) { $ok = 0 if ($f{src} !~ /$_/) }
		foreach (@m_dst) { $ok = 0 if ($f{dest} !~ /$_/) }
		foreach (@m_mod) { $ok = 0 if ($f{mode} !~ /$_/) }
		foreach (@x_src) { $ok = 0 if ($f{src} =~ /$_/) }
		foreach (@x_dst) { $ok = 0 if ($f{dest} =~ /$_/) }
		foreach (@x_mod) { $ok = 0 if ($f{mode} =~ /$_/) }
		if ($ok) {
			# cool, all matches ok
			print STDERR "($f{src}\n$f{dest}\n$f{mode})\n\n";
			# send it into current package file-list
			print OUT "$f{dest}\n";
			# stop processing of this file (no file should belong to 2
			# packages AFAIK)
			splice (@files, $i, 1);
		} else {
			# no match, don't worry for now; we have probably more passes
			# to do
			$i ++;
		}
	}
	# prepare output
	print STDERR "\n-------------------------\n\n";
	close OUT;
}
# }}}
# {{{ append overrides
for (@pkgnames) {
	open OUT, ">>debian/$_.install";
	open OVER, "debian/$_.inst_over";
	while (<OVER>) {
		print OUT $_;
		print STDERR "override: $_";
	}
	close OVER;
	close OUT;
}
# }}}
# {{{ warn about remaining lines (that were not matched by any rule)
print STDERR "-------------------------------\n";
print STDERR "unmatched files follow:\n";
for (@files) {
	my %f = %$_;
	print STDERR &nowd ($f{src}) . " ($f{dest})\n";
	#	unless (/make\[[0-9]+\]: Entering directory `.+?'/);
}
# }}}

Reply to: