[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: additions to dpkg-architecture and dpkg-cross

Volker,

  Great that you've put this up to public. Still, for this to be
accepted, we need to come up with patches for apt that contain an
algorithm that will pick versions of packages matching user's
requirements, known architecture hierarchies (i386->i486->...)
and available sources. Currently, apt is quite dumb in that respect
AFAIK, i.e. it only looks at binary-$DEB_HOST_ARCH while it could
also look binary-all, binary-$DEB_HOST_ARCH, and
binary-$DEB_HOST_SUPERARCH (where $DEB_HOST_SUPERARCH=i386 when
DEB_HOST_ARCH=i686) etc.

  I've used this sort of logic for rootfs generation script I've been
playing around with (I'll be bold enough to post it here just for
example). It takes a Debian architecture name as $1 and then for
each package name that it reads from /dev/stdin it finds a version
from list of source repositories in this order:

1. binary-$DEB_HOST_ARCH (e.g. armv5te)
2. binary-$DEB_HOST_SUPERARCH (e.g. arm)
3. local compiles (e.g. /usr/src/Debian)
4. binary-all.

  It only has one-level hierarchy, but that's just a home-grown sample...

  Also, dpkg-cross's logic of barfing at conversion between non-equal
architecture (names) MUST be changed... It really should also look
at the architecture hierarchy and allow for "down-casting" of
packages to their cross- counterparts since you really don't want
to compile the whole distribution with optimizing/cross compilers
(we are not Gentoo are we?).

  Furthermore, isn't there a dpkg requirement that all packages
installed must have either DEB_HOST_ARCH or "all" as Architecture?

Should we somehow coordinate our efforts to make this into official
apt/dpkg/dpkg-cross?

Pjotr

Volker Grabsch wrote:
> Dear Debian Developers,
> 
> I've been pleased to summarize some ideas for the general public.
> More details and sources can be found at the end of this mail.
> I'm not subscribed to all lists, so please CC any replies to me.
> 
> --------------------------------------------------------------------------
> 
> I propose to add more CPU types to dpkg-architecture. In particular,
> I'd like to see the different i386 architectures there, i.e.
> i586, i686, k6, ...
> 
> The dpkg-cross tool is currently only used for cross compiling
> applications. However, dpkg-cross is a more generic tool. The
> "dpkg-buildpackage" wrapper of dpkg-cross allows you to compile
> a package with different compilers, without the need to change
> your debian/rules. (usually, some compatibility changes are needed
> to make the debian/rules file "cleaner", but that's all)
> 
> For instance, some programs with lots of calculations (e.g. mplayer)
> are compiled with different processor optimizations (e.g. mplayer-i586).
> Such packages are created by very redundant entries in debian/rules.
> Exactly such redundancy is removed by dpkg-cross.
> 
> Also, these optimized versions get names like "mplayer-i586",
> but their architecture is set to "i386". That means, the *real*
> architecture is encoded in the package name instead of the package's
> architecture.
> 
> This is a work around the inability of apt to automatically use
> the best suited binary packages. (e.g. uses on a i686 system the
> mplayer-i586 instead if mplayer-i386, and the kernel-image-i686)
> 
> If one day apt evolves, packages like mplayer-i586 will step in their
> way. The "Package:" field should not contain architectural information.
> The "Architecture:" field serves that purpose.
> 
> Until APT becomes aware of multiple architectures, this won't change
> much. However, there are still lots of interesting cases where
> more architectures are desirable.
> 
> For example, suppose you support a new OS, such as the w32 platform.
> Currently, your only choice is "w32-i386", which means that you must
> use an i486-mingw32msvc Compiler. However, for a w32 system a i486
> compile doesn't make a lot of sense. Since those systems (except very
> old Windows versions) need at least a Pentium, it is reasonable to
> compile such a distribution at least for i586, not i486.
> 
> That means, the "linux" OS dictates that i386 should mean "at least
> i486", while other OSes (e.g. win32) have different needs (e.g. "at
> least i586"). Such dependencies between the OS and CPU type are simply
> unnecessary and disturbing.
> 
> In that example, there should be a "linux-i386" platform (which may
> be abbreviated to i386), while dpkg-architecture should also allow
> a Debian platform like "w32-i586". This proposal is similar to
> the addition of multiple arm CPU variants to dpkg-architecture.
> 
> Of course, if you want to allow, e.g. the -i386 packaes to be installed
> on a -i586 distribution, some bigger parts of dpkg and apt would have
> to be changed. However, that is not my proposal at all. I don't intend
> to create a w32-i586 Debian distribution which must also accept w32-i386
> packages. I want to create an entire w32-i586 distribution.
> (well, currently, I just want to provide cross compiling packages for
> such a platform)
> 
> I also heard about people who compiled a whole (Debian based) system
> for i586 to increase the overall performance of their system. They,
> too, had the same problem: How to name this architecture/cpu? As far
> as I remember, they "solved" that problem by adding a "pentium" line
> (or similar, AFAIR) to their cputable.
> 
> Thus, I don't think it is the question whether one wants to compile
> for i586, i686, etc.  There are always good reasons for some groups to
> do so. IMHO, the more important question is how to name that thing. So
> it would be very nice if dpkg-architecture would be flexible enough to
> support that, instead of stepping in the way.
> 
> To solve the problem at least for cross compiling, there are two
> possible ways to go:
> 
> 1) insert more CPU types into dpkg-architecture
>     (e.g. i586)
> 
> 2) handle an extra CPU list internally by dpkg-cross
>     (e.g. when cross compiling for w32-i586, create a package
>      like gettext-i586_w32-i386.deb)
> 
> While option 2) is a huge amount of work for dpkg-cross,
> option 1) is relatively easy to do in dpkg.
> In addition, 1) opens the way for apt to support optimized
> binaries natively in the future, and also to use dpkg-cross
> for optimized packages, instead of additional debian/rules
> entries.
> 
> Optimized and cross compiles packages are not that different, IMHO.
> 
> --------------------------------------------------------------------------
> 
> This mail goes to:
> 
> * debian-devel mailing list,
>     (to see if there's a general consensus)
> 
> * debian-embedded mailing list,
>     (because here the thread started)
> 
> * debian-dpkg mailing list,
>     (because it affects the future of dpkg/apt)
> 
> * Dpkg Developers
>     (because this discussion affects dpkg-architecture)
> 
> 
> This mail is a summary of the discussions on the debian-embedded mailing
> list. Thread overview:
> 
>     http://lists.debian.org/debian-embedded/2006/06/threads.html#00016
> 
> In particular, please have a look at these postings:
> 
>     http://lists.debian.org/debian-embedded/2006/06/msg00072.html
>     http://lists.debian.org/debian-embedded/2006/06/msg00045.html
>     http://lists.debian.org/debian-embedded/2006/06/msg00068.html
>     http://lists.debian.org/debian-embedded/2006/06/msg00044.html
>     http://lists.debian.org/debian-embedded/2006/06/msg00059.html
>     http://lists.debian.org/debian-embedded/2006/06/msg00019.html
> 
> 
> Greets,
> 
>     Volker
> 


#!/bin/bash
da=$1
b=$2
d=$3
s=$4

[[ -z "$da$b$d$s" ]] && { echo "no args" >/dev/stderr; exit 1; }

all_cpus=(arm mipsel)
mipsel_vars=(mipsel-viper)
arm_vars=(armv4 armv5te strongarm xscale)

bins=$b

a=${da/-linux-diet*}
[[ $a != $da ]] && bins={$b,$d}

a=${a/-linux-gnueabi*/-eabi}
a=${a/-linux-uclibc*/-uclibc}
a=${a/-linux-gnu*/}

archs={all,$a}

cpu=$a
cpu=${cpu%%-eabi}
cpu=${cpu%%-uclibc}

#echo cpu:\'$cpu\' >/dev/stderr

for gen_cpu in ${all_cpus[@]}; do
	l=`eval echo \\${${gen_cpu}_vars[@]}`
	for x in $l; do	[[ $x == $cpu ]] && {
		archs={all,$gen_cpu${a##$cpu},$a}
		break
	} done
done

#echo archs:$archs >/dev/stderr

while read p; do
	f=
	pa=(${p/=/ })
	package=${pa[0]}
	version=${pa[1]}
	[[ -z $version ]] && version='*'
#	echo $p p:$package v:$version >/dev/stderr
	list=`eval echo {$bins/binary-$archs,$s}/${package}_${version}_$archs.*deb`
#	echo $list | tr ' ' '\n' >/dev/stderr
	
	for x in $list; do [[ -r $x ]] && {
		f=$x
	} done
	
	[[ -z "$f" ]] && {
		echo "ERROR: package $package (version $version) not supported for $a" >/dev/stderr
		exit 1
	}
	
	echo $package $f
	dpkg -x $f .
	
	rm -rf usr/share/{doc,doc-base,info,man,linda,lintian,menu}
	rm -rf usr/share/{zoneinfo,locale}
	rm -rf usr/lib/gconv
done

Attachment: signature.asc
Description: OpenPGP digital signature

Reply to: