Bug#663762: debian-policy: default for DEB_BUILD_OPTIONS=parallel=N
On Sun, Mar 25, 2012 at 04:44:23PM +0200, Bill Allombert wrote:
> On Sun, Mar 25, 2012 at 09:18:18AM +1030, Ron wrote:
> >
> > Hi Jakub, Russ,
> >
> > On Sat, Mar 17, 2012 at 04:55:19PM -0700, Russ Allbery wrote:
> > > Jakub Wilk <jwilk@debian.org> writes:
> > >
> > > > How should packages behave if there is no explicit "parallel=N" in
> > > > DEB_BUILD_OPTIONS? I saw two different approaches:
> > >
> > > > 1) Behave (roughly) like if parallel=1 was set.
> > >
> > > > 2) Be clever and try to guess the "right" level of parallelism, e.g. by
> > > > using "getconf _NPROCESSORS_ONLN" or parsing /proc/cpuinfo (ugh!).
>
> On an hyperthreaded system, "getconf _NPROCESSORS_ONLN" return the number of threads
> and not the number of cores, so it is not always what the user want.
It's actually somewhat tricky to reliably separate the number of 'real' cores
from the number of 'virtual' cores on such systems (though it can be done, and
the ease of performing such a query probably could be improved if there was a
proven need for it).
In practice though, on a 4 core HT system reporting it has 8 cores, make -j8
is actually almost always what you want. I have several such systems and it
it doesn't burden them any harder than -j4 on a strictly 4-core system does,
and it does build many or most things ~2x as fast as -j4 would.
In fact given the way that recursive make works, one quite surprising point
is that some things will actually benefit with a speedup of notably greater
than N for a given -jN.
Witness a factor of 8 walltime speedup on a non-HT 4 core machine:
(with hot cache and both tests run several times)
$ time make clean
real 0m4.194s
user 0m3.980s
sys 0m1.944s
$ time make -j4 clean
real 0m0.498s
user 0m1.780s
sys 0m0.460s
I've seen larger multipliers than that too, but obviously this depends
more on the work that make is doing at each of its levels, than on the
degree of CPU loading. The time saving is very real just the same though.
Best,
Ron
Reply to: