[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: distcc recommendation

On 17/04/2010, at 2:06 AM, Thorsten Glaser wrote:


how well does distcc interact with dpkg-buildpackage? ISTR that
there are some specific requirements on the build system wrt.
parallel build capacity. Does that possibly introduce breakage?
(Not a distcc problem but a problem with debian/rules and/or
upstream build systems.)

Transparently. There are issues with after compiling library X, use library X to compile package Y. You could use a shared NFS to keep clients in sync, or a meta script to install results after compiling each step.

Distcc is a server/client configuration that just splits out the object compilation parts for compiling C/C++. Instead of calling C/C++ in a block, distccd does the preprocessing* and sends the compilation to your compilation servers, then gets the object files back and links them. Since compilation typically takes up to 90% of the time, you can get great speedups using a few additional computers. The important things are the redirections to get distcc and ccache (also recommended) to work instead of gcc.

I noticed when I was testing times for m68k Mac sid compilations, that 10baseT networking is the major bottleneck. At 5 cpus adding more cpus does not seem to make a difference as limits on network and host cpu seem to stop improvements. (the flashy lights just stay on :( ) With older systems I would also have a host setup that either placed the host last in the distcc host file or not at all.

My numbers were cpus versus speedup (quadra650 host and quadra605 68040 for clients)
1 ~100%
2 ~180%
3 ~250%
4 ~290%
5 ~300%
6 ~300%

These used a "time make" get the rough numbers. I did get a 4 ~330% by adding an additional network card in the q650. But then I ran out of network cards and computers to test further. This was back in 2007 with 2.6.18/2.6.21 kernels and 3.4 GCC. The longest part was getting all client (Q605 68040) systems the same.

After about a month of bashing my head against 4.2 GCC builds I moved house and shelved the project (because I no longer had powered cupboards :) or even space inside.)

* distcc 3.1 can also send out the header files to allow preprocessing on client systems, feature called pump.

And if nothing else use ccache to speed up compilation of clean code. make clean strips all object files, ccache puts back those that have identical (according to a hash lookup) headers and source files.
Michael Tomkins
+61 433 154 347

Reply to: