#include <hallo.h>
* Guus Sliepen [Sun, Jan 04 2009, 10:45:23AM]:
> On Sat, Jan 03, 2009 at 09:57:33PM +0100, Eduard Bloch wrote:
>
> > PS: I plan to hack it a little bit and use syssconf function on Debian
> > systems to determine the real number of CPU cores (#x) since pigz's
> > default value is 8 which is much more than home systems have nowadays,
> > and the performance isn't getting (much) better with a constant number
> > of idle threads, they just consume more memory.
>
> Although sysconf() can tell you the total number of cores and the number of
> cores that are online in your system, it does not tell you how many cores are
> available for your program. It's better to use sched_getaffinity() to get the
> set of CPU (cores) available to your program. And if you know better than the
> OS, use sched_setaffinity() to bind each thread to it's own core.
Sounds like a plan, but I don't feel very comfortable to do that in the
Debian package. Let me explain why:
- sched_setaffinity method seems to be Linux specific
- though it might be beneficial to pinpoint each thread to the same
CPU, I don't think (*assumption*) that the data resides long enough
in the CPU cache anyway so we wouldn't win much
- I have not used it before but I used the sysconf method in portable
apps (i.e. in cloop-utils), sysconf method is also used in pbzip2
- it's hard to imagine environments with big difference between
count(cores) and count(available cores)
- the code change is minimal, see below
@Marc: please check the -s option, it's called -S in gzip (upper case).
What's the reason for the different case?
Regards,
Eduard.
--- pigz-2.1.4.orig/pigz.c
+++ pigz-2.1.4/pigz.c
@@ -2795,8 +2795,13 @@
#ifdef NOTHREAD
procs = 1;
#else
+
+#ifdef _SC_NPROCESSORS_ONLN
+ procs=sysconf(_SC_NPROCESSORS_ONLN);
+#else
procs = 8;
-#endif
+#endif /* _SC_NPROCESSORS_ONLN */
+#endif /* NOTHREAD */
size = 131072UL;
rsync = 0; /* don't do rsync blocking */
dict = 1; /* initialize dictionary each thread */
--
Naja, Garbage Collector eben. Holt den Müll sogar vom Himmel.
(Heise Trollforum über Java in der Flugzeugsteuerung)
Attachment:
signature.asc
Description: Digital signature