Re: need help making shell script use two CPUs/cores
Bob Proulx put forth on 1/23/2011 8:16 PM:
Apparently I've missed some of the thread since my earlier participation.
> Carl Johnson wrote:
>> #CPUs time theoretical time-theoretical gain/CPU(theoretical)
>> 1 66
>> 2 36 66/2 = 33 36-33 = 3 (+9%) 1 -1/2 = 1/2
>> 3 25 66/3 = 22 25-22 = 3 (+14%) 1/2-1/3 = 1/6
>> 4 20 66/4 = 16.5 20-16.5 = 3.5 (+21%) 1/3-1/4 = 1/12
>
> I liked that analysis.
>
> Here is some raw data from another test using GraphicsMagic from Debian
> Sid on an Intel Core2 Quad CPU Q9400 @ 2.66GHz.
>
> #CPUs real user sys
> 1 ... 32.17 100.15 2.29
> 2 ... 28.02 102.09 2.25
> 3 ... 26.96 101.41 2.02
> 4 ... 26.18 99.85 2.10
> 5 ... 26.03 98.58 2.27
> 6 ... 27.07 97.32 2.17
> 7 ... 27.74 100.09 2.03
> 8 ... 26.76 97.83 1.99
> 9 ... 27.24 97.31 2.88
> 10 ... 26.27 99.05 2.76
> 11 ... 26.35 99.30 1.84
> 12 ... 25.91 97.63 2.08
So, I'm not understanding how we have a quad core CPU with 12 CPUs. Is "#CPUs"
here your xargs "-P" argument in the script you posted in response to my
question that started this thread? Why bother going up to 12 processes with a
quad core chip? Anything over 4 processes/threads won't gain you anything, as
your results above demonstrate.
> And the same thing using ImageMagick on the same system.
>
> #CPUs real user sys
> 1 ... 24.69 62.60 2.87
> 2 ... 19.28 63.17 2.50
> 3 ... 17.82 60.34 2.65
> 4 ... 17.48 58.86 2.55
> 5 ... 16.60 58.11 2.34
> 6 ... 15.85 58.03 2.38
> 7 ... 15.61 58.09 2.44
> 8 ... 15.36 57.68 2.48
> 9 ... 15.48 57.76 2.38
> 10 ... 15.38 57.76 2.28
> 11 ... 15.36 57.97 2.27
> 12 ... 15.73 58.76 2.17
>
> Watching the individual cpu load I observe that while the 1 cpu case
> did consume one cpu fully that the other three were also showing quite
> a bit of activity too.
Imagemagick will use threads on larger images. To keep it from threading, in
order for your testing to make more sense, use smaller images.
> There was already quite a bit of parallelism
> happening before adding the second cpu, and third, and so forth. With
See above. BTW, you weren't "adding the 2nd CPU". You were merely spawning
more _processes_ no?
> three running all four cpus were looking pretty much 100% consumed. I
> was timing all of the shell's for loop, the xargs and the convert
> processes all together.
If you are converting images large enough that the threading kicks in, there's
little reason to use multiple processes at that point. We'd already discussed
this. Were you simply trying to confirm that with these tests?
> I also tried running this same test on some slower hardware. I have
> gotten spoiled by the faster machine. The benchmark is still running
> on my slower machines. :-) I am not going to wait for it to finish.
What are the CPU specs of this older machine?
--
Stan
Reply to: