Re: Bug#510624: ITP: pigz -- Parallel Implementation of GZip
Josselin Mouette, le Mon 05 Jan 2009 00:20:42 +0100, a écrit :
> Samuel Thibault, le Sun 04 Jan 2009 23:45:22 +0100, a écrit :
> > > It’s already the case in HPC environments, and CPU pinning is certainly
> > > going to be used more widely as the number of cores increases.
> >
> > And that's a shame. Linux shouldn't be so happy to move tasks between
> > CPUs...
>
> Actually it doesn’t. Since CPU affinity was included (IIRC in 2.6.16) it
> is much less prone to move tasks, and the performance impact of not
> using CPU pinning is small.
Things have improved, yes.
> Still, it is better to use CPU pinning since you often want finer
> control than that, and that’s especially true in multi-user environments
> where resources can be sub-host.
Sure, but that should be only a user-explicitely-wanting thing. I would
really not like to see pigz systematically bind threads. What if I e.g.
want to run several pigz processes at the same time because I have a lot
of cores and a lot of files (I guess pigz doesn't scale so much)?
Ideally, we should just let Linux manage everything, i.e. put related
threads together on the same dies, balancing the load according to the
observed behavior, which can vary a lot depending on the latency of
reading files, the time to compress, etc.
Samuel
Reply to: