[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: make -j in Debian packages

On Wed, Jul 05, 2006 at 08:23:00PM +0200, Wouter Verhelst wrote:
> On Mon, Jul 03, 2006 at 03:04:14PM +0200, Adam Borowski wrote:
> > On Sun, Jul 02, 2006 at 11:57:50AM +0200, Wouter Verhelst wrote:
> > > Additionally, it puzzles me how you think a maintainer will be able to
> > > accurately predict how much RAM a certain build is going to use. There
> > > are so many variables, that I think anything but 'this is the fastest
> > > way to build it on my machine' is going to be unfeasible.
> > 
> > Let's say:
> > program X consist of a number of C files; it seems like compiling
> > every file takes around 24MB,
> Like I said, there's just too many variables. Also, I wouldn't be
> interested in figuring out how much RAM the build takes if I were to
> maintain a package like, say, X or so.

No one forces you to do so.  No one even said a word about making
concurrency mandatory.  It's just a way to make builds faster on
machines that are beefy enough (ie, virtually all).

> You're not convincing me that you can indeed accurately predict for
> every compiled language out there how much RAM you're going to need
> during the build.

So what?  If you're not sure, you simply don't provide your
prediction.  And with a huge safety margin, you would have to be
MALICIOUS to make a mistake.

> Before you've proven that this is indeed possible, I don't think
> there's much point in this whole exercise; otherwise there
> *is* going to be a problem with you overloading build machines, and
> *you will* get bugs filed about that (from me, at the very least).

Here, you got a piece of working code, is that not a good enough
proof?  Tell me how exactly a build using 4 processes using ~24MB
each overloading a build machine which has 512+MB ram.  And builds
which do require 1+GB of memory, well, simply are not going to use
any concurrency.

And note that the system I propose actually _limits_ concurrency in
some cases.  The whole thread started with gem packages choking the
m68k build.  It's a big package, and the maintainer rightfully
thinked that it is completely useless on small arches.  The
optimization he employed was to use -j4 -- something that can't hurt
a machine on arches where the package is usable.  Too bad, the
maintainer's fault was that he didn't protect the -j4 from arches
which can't handle it.  And handling this, exceptionally rare in
normal use, case is what we can fix.

Even you, in the initial post of this thread, proposed a way to
enable concurrency in some cases, so this can't be such a bad idea :p

1KB		// Microsoft corollary to Hanlon's razor:
		//	Never attribute to stupidity what can be
		//	adequately explained by malice.

Reply to: