Re: subarchitectures
Greetings, and thank you so much for your very helpful reply!
Adam C Powell IV <hazelsct@mit.edu> writes:
> Camm Maguire wrote:
>
> > Greetings! The good news is that in days, we should have some
> > optimized atlas blas libraries for a variety of recent processors.
> > The challenge is how to adapt the package -- when built on an Athlon,
> > PIII, and PII, the resulting binaries will run only on those
> > subarchitectures, (and much faster!)
>
> Excellent!
>
> > I suppose all binaries could be
> > included in the package, and only the appropriate one installed, but
> > how does this jive with the automatic build daemons? Such a package
> > would have to be partially built on several different machines. On
> > the other hand, I could have several different packages, each of which
> > would build only on a given subarch, and would produce a distinct
> > package name. That might not work so well with automated build
> > daemons either.
>
> I remember reading in a recent email (debian-mentors?) that ldconfig and ld.so
> under glibc 2.2 look into subarch-specific directories, e.g. /usr/lib/i586,
> /usr/lib/i686, etc. before looking in /usr/lib. So if you could build with say
> -mcpu=i686 and the PIII asm flags and install that in /usr/lib/i686, then
> rebuild the package with -mcpu=i586 and say MMX but not newer instructions,
> you'd have a much larger package which would be optimized in a lot of places!
>
This is very helpful -- Thank you! My immediate question is if the
Athlon and PIII are both 686, in which case we have a problem, as
their simd stuff is different. But I can look into this. This is
definitely a good possibility for the *installation* of the package.
I still have a problem with *building* the package, however.
Currently, atlas both compiles and *executes/times* the resulting code
on the compilation machine, to determine the fastest code. To my
knowledge, there is currently no cross-compilation capability to
atlas. I'm checking into this, but assuming its the case, how can we
auto*build* either one package, or many specific architecture
dependent packages, where the build process depends on the building
cpu?
I suppose in principle, this is no different from our current
conception of 'ports', and the _i386,_alpha,_arm, etc. extension to
the package name. Its just that in this case, this needs to be
extended slightly to (at least) _i386_p3, _i386_p2,_i386_k6,_i386_k7.
There should be no problem in principle autobuilding these, except for
1) The autobuilding system/package naming system currently has no
infrastructure for this, and 2) I'm not even sure if the Debian
organization has all of these architectures at its disposal. Of
course, yet another possibility is that I could simply compile on
these architectures myself, upload the binaries, and skip the
autobuilders for _i386 at least. I'm not familiar enough with the
archive maintenance though to know if this would cause problems.
If I did it here, and if the ld.so name space would support, I'd
probably do the following:
1) unpack/patch the source
2) nfs mount source tree on k6,k7,p2,p3
3) build on each machine, without cleaning (atlas has the ability to
build the libs under the same source tree simultaneously on
different archs, without the build interfering with each other.)
4) on the original machine, debian/rules install
5) upload one or several binary packages.
> The problem with having, say, postinst determine the local subarch and only
> installing the appropriate lib is that it fails on CPU change. The above
> approach (building for all of the targets and installing all of the libs) would
> be disk-intensive, but easy on the end-users- they'd just need to install
> "atlas2" and ld.so would make it always work optimally.
>
A very good observation. Again, thanks!
> The other approach, like what you've outlined, would have a single source
> package build all of the separate binary packages by repeatedly cleaning and
> building. This would save disk space vs. what I have above, but would be a pain
> for those who wanted to build from source (building would take forever), but the
> autobuilders should like it, and it wouldn't be too hard on the end-users:
> installing a package requiring "atlas2" would present them with the choice of
> atlas2-386, atlas2-pentium, atlas2-athlon, etc. And you'd have to tailor it by
> processor family, e.g. Alpha builds ev4, ev5, ev56, ev6, ev67; PPC builds 601,
> 603, 603e, 604, 604e, 750, Altivec... okay, this gets really ugly really fast!
>
I'm inclined to agree. The name space can get rather large, whether
handled in a debian package name, or via ld.so. But fortunately, for
all cases you mention above except the k6,k7,p2,p3, code compiled on
one family (ev67, say) will run on ev4, though not optimally. In the
case we're discussing here, code compiled on p3 will not run on
p2,k6,k7. Argh.
> Unfortunately, I'm not sure how fine the subarches are, e.g. PII and PIII are
> both i686 but may have separate /usr/lib/xxx subdirs, and I don't know the docs
> well enough to tell you where to find this info. :-(
>
> So that's what I know.
>
Thanks again!
> -Adam P.
>
> Welcome to the best software in the world today cafe!
>
>
> --
> To UNSUBSCRIBE, email to debian-beowulf-request@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
>
>
>
--
Camm Maguire camm@enhanced.com
==========================================================================
"The earth is but one country, and mankind its citizens." -- Baha'u'llah
Reply to: