Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 12:38:51PM +0200, Ingo Juergensmann wrote:
> On Wed, Jun 28, 2006 at 10:31:54AM +0200, Adam Borowski wrote:
> > On the other hand, making builds significantly faster is not
> > something that you can shake a stick at. Typically make -jX is faster
> > even on uniprocessor, and I don't need to tell you why it's much
> > faster on SMP.
>
> Well, make -jX is not everywhere faster on UPs. It depends on other factors
> as well. If you specify -j2 and the second make is causing lots of swapping,
> you won't gain much if anything at all.
Exactly, just like I said: it depends on the memory.
The benefits on UP are small (~10%), but except for huge working
sets, non-negative. And the maintainer knows if the package handles
huge chunks at once or not.
> > Nearly every buildd and nearly every user building the packages
> > on his own will benefit from -j2 [2], even on non-SMP. Unless
> > it's a piece of heavily-templated code, any modern box will have
> > enough memory to handle it. The maintainers know whether the
> > code is heavily templated or not.
> >
> > Giving the choice to someone who doesn't know the package
> > (upstream and/or the maintainer) will lead to _random_ FTBFSes.
>
> Therefore a double opt-in would be nice (machine admin and package
> maintainer saying it's OK).
>
> > Having DEB_MAKE_NON_PARALLEL disable this behavior would make
> > m68k happy, while giving a reasonable default for everyone else.
>
> I don't think it's good to define an opt-out variable (*_NON_PARALLEL).
> Think positive! So, it would be better to define DEB_MAKE_PARALLEL, but even
> better it would be to use something existing: CONCURRENCY_LEVEL as Don
> Amstrong suggests.
> If this variable exists (e.g. if [ -z CONCURRENCY_LEVEL ]) the maintainer
> can make use of -jX.
Indeed, using the existing name is better.
But, did you really want to say -z? It's what I want, and you seem
to argue against me :p
However, it's very unlikely someone would set a random env var unless
the person knows about it. Defaults are what a vast majority of
users use, so I would go for sensible ones.
> Keep in mind that there are archs that don't have that much resources to
> build in parallel, and I'm not thinking of m68k. M68k has buildds with 512M
> RAM, but for example armeb which has only 32M.
Good point, but such machines are RARE. I would rather go for either
have the admin opt out or them, or, have the packages detect low-mem
conditions and optimize their builds accordingly.
> I would like to see a method to allow usage of other compile accelerators as
> well (distcc (for using crosscc, but testing locally), ccache, etc).
In theory, you would set CC, but not everything obeys it. I
personally tend to mess with what "gcc" points to. Usually with more
than one levels of indirection, too -- colorgcc is a nice thing for
when the output goes to a terminal.
--
1KB // Microsoft corollary to Hanlon's razor:
// Never attribute to stupidity what can be
// adequately explained by malice.
Reply to: