Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 10:31:54AM +0200, Adam Borowski wrote:
> On the other hand, making builds significantly faster is not
> something that you can shake a stick at. Typically make -jX is faster
> even on uniprocessor, and I don't need to tell you why it's much
> faster on SMP.
Well, make -jX is not everywhere faster on UPs. It depends on other factors
as well. If you specify -j2 and the second make is causing lots of swapping,
you won't gain much if anything at all.
> Too bad, a C++ build where every file takes 1GB memory obviously
> should not be parallelized. Also, no one but the maintainer knows
> whether a package is SMP-clean or not. You cannot guess this in an
> automated way.
Correct. But you should extend the decision whether a package should be
built in a parallel make to the machine admin as well. When both say it's ok
to make use of -jX, then go for it. If one of these two have any
objections, then leave out the -j option.
> Thus, my counter-proposal:
> Let's allow maintainers to use make -jX according to their common
> sense, requiring obeying an env variable to opt out.
I would prefer an opt-in variable.
> Rationale:
> Nearly every buildd and nearly every user building the packages on
> his own will benefit from -j2 [2], even on non-SMP. Unless it's a
> piece of heavily-templated code, any modern box will have enough
> memory to handle it. The maintainer know whether the code is heavily
> templated or not.
>
> Giving the choice to someone who doesn't know the package (upstream
> and/or the maintainer) will lead to _random_ FTBFSes.
Therefore a double opt-in would be nice (machine admin and package
maintainer saying it's OK).
> Having DEB_MAKE_NON_PARALLEL [3] disable this behavior would make
> m68k happy, while giving a reasonable default for everyone else.
I don't think it's good to define an opt-out variable (*_NON_PARALLEL).
Think positive! So, it would be better to define DEB_MAKE_PARALLEL, but even
better it would be to use something existing: CONCURRENCY_LEVEL as Don
Amstrong suggests.
If this variable exists (e.g. if [ -z CONCURRENCY_LEVEL ]) the maintainer
can make use of -jX.
Keep in mind that there are archs that don't have that much resources to
build in parallel, and I'm not thinking of m68k. M68k has buildds with 512M
RAM, but for example armeb which has only 32M.
I would like to see a method to allow usage of other compile accelerators as
well (distcc (for using crosscc, but testing locally), ccache, etc).
--
Ciao... // Fon: 0381-2744150
Ingo \X/ SIP: 2744150@sipgate.de
gpg pubkey: http://www.juergensmann.de/ij/public_key.asc
Reply to: