Re: Parallel build results

[Adam Borowski <kilobyte@angband.pl>]

On Mon, Dec 31, 2007 at 10:07:15PM +0100, Robert Millan wrote:
> On Sat, Dec 01, 2007 at 09:21:33PM -0500, Daniel Schepler wrote:
> > I finally got through the test builds of all the source packages in sid for 
> > i386 using dpkg-buildpackage -j3 on a dual core machine.  The results as 
> > before are at http://people.debian.org/~schepler/build-logs/bymaint.html .  
> 
> Why -j3 ?  It's already quite an effort to aim for -j2.  Shouldn't we try to
> fix those first?

Once you go over -j1, nearly all usual race conditions can be potentially
triggered.  You would need a quite contrived build system to get a three-way
race which cannot fail with -j2.

And races caused by doing more than one task at once are pretty rare, in
most in the cases the failure is due to wrong dependences.  The root of the
problem is, people are used to a single-tasked make which always builds
everything from left to right.  Things won't break until -j or -k is used.

What about changing make so it builds prerequisites in a random order, or at
least right-to-left which is typically the worst case?  This would help spot
dependency problems other than true races.


(an example which will not be caught by hypothethical make --order=random
but will likely break for -j higher than 1:

all: foo bar

foo: foo.c
	gcc src.c
	mv a.out $(target)/foo

bar: bar.c
	gcc bar.c
	mv a.out $(target)/bar

Another example could be building two flavours of the same package.)


True race conditions are non-deterministic, so even repeated test builds
are likely to find most of them, but wrong dependencies are a lot easier
to catch.

-- 
1KB		// Microsoft corollary to Hanlon's razor:
		//	Never attribute to stupidity what can be
		//	adequately explained by malice.