Re: make -j in Debian packages
On Wed, Jun 28, 2006 at 01:37:37PM +0200, Adam Borowski wrote:
> > Well, make -jX is not everywhere faster on UPs. It depends on other factors
> > as well. If you specify -j2 and the second make is causing lots of swapping,
> > you won't gain much if anything at all.
> Exactly, just like I said: it depends on the memory.
Not only on memory. Take into calculation such values like context switch
overhead, task scheduling overhead, etc.
On SMP systems there's is even some gain in parallel processing, when you
can nail certain process onto certain CPUs to prevent their migration across
CPU bounds. But that's another story.
> The benefits on UP are small (~10%), but except for huge working
> sets, non-negative. And the maintainer knows if the package handles
> huge chunks at once or not.
Right. The maintainer should know it. But when the benefits are so small,
one can argue if it's worth the work to change the build system? ;)
Therefore I think it's best to first go with opt-in before make opt-out the
default, let's say when etch+2 is out.
> > I don't think it's good to define an opt-out variable (*_NON_PARALLEL).
> > Think positive! So, it would be better to define DEB_MAKE_PARALLEL, but even
> > better it would be to use something existing: CONCURRENCY_LEVEL as Don
> > Amstrong suggests.
> > If this variable exists (e.g. if [ -z CONCURRENCY_LEVEL ]) the maintainer
> > can make use of -jX.
> Indeed, using the existing name is better.
> But, did you really want to say -z? It's what I want, and you seem
> to argue against me :p
*cough* missing ! or -n instead of -z of course ;)
> However, it's very unlikely someone would set a random env var unless
> the person knows about it. Defaults are what a vast majority of
> users use, so I would go for sensible ones.
The most sensible value, that should work for almost every user would be:
make without -j option set. That's the default now, isn't it? ;)
> > Keep in mind that there are archs that don't have that much resources to
> > build in parallel, and I'm not thinking of m68k. M68k has buildds with 512M
> > RAM, but for example armeb which has only 32M.
> Good point, but such machines are RARE. I would rather go for either
> have the admin opt out or them, or, have the packages detect low-mem
> conditions and optimize their builds accordingly.
I don't think that those machines are rare. OTOH, almost everything is rare
when compared to x86 boxes.
But let some porters/policy people decide whether to use opt-out or opt-in.
> > I would like to see a method to allow usage of other compile accelerators as
> > well (distcc (for using crosscc, but testing locally), ccache, etc).
> In theory, you would set CC, but not everything obeys it. I
> personally tend to mess with what "gcc" points to. Usually with more
> than one levels of indirection, too -- colorgcc is a nice thing for
> when the output goes to a terminal.
Of course, but my point is: when we make a decision now about -j flag, we
can have some thougts about such accelerators as well and implement both in
one go instead of having two separate changes to the build system.
--
Ciao... // Fon: 0381-2744150
Ingo \X/ SIP: 2744150@sipgate.de
gpg pubkey: http://www.juergensmann.de/ij/public_key.asc
Reply to: