[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: sys cpu usage

--On August 26, 2008 8:14:50 PM +0200 Marek Podmaka <marki@marki-online.net> wrote:


Tuesday, August 26, 2008, 17:18:01, Michael Loftis wrote:
their primary filesystem is on NFS, the NICs do UDP/TCP checksum
offloading, and they mostly push large packets I see up to about 15%
or so on dual core 2Ghz processors (Xeon 5130s not hyperthreaded, so
four CPUs). But that's peak, average is more like 7-8% at around 70
hits/second, and we've a pretty high overhead too. So 25%+ sounds

How is TCP checksum offload affecting the SYS usage? It is the chip on
network card which is doing the stuff and not kernel, right? Or does
it show in the sys cpu usage? (On this server, only checksumming
offload is supported.)

Depends on what is supported on your card, use ethtool -k <dev> to find out and to change settings. On our systems we're doing checksum and TSO. The offload doesn't show at all in the CPU usage stats. The CPU basically skips computing checksums, and for TSO, segmentation, and just sends big packets to the DMA ring buffers, the card then picks it up, does any segmentation (replicating/affixing IP headers), computes checksums, and pushes it out over the wire. This frees up a decent amount of CPU at high packet rates or high line rates. The result is lower %sys CPU, thus more CPU available for userspace (and system) operations. UDP checksum offloading is actually pretty important for us since we're using NFS as primary storage.

What kernel are you running BTW? And debian version? Kernels prior to 2.6 didn't do well on high interrupt loads P4, Xeon, and Core2's because of large stack frames. Somewhere in 2.6 they made a change that allows the kernel to export an optimal path for entering/leaving the kernel routines without necessarily saving the entire stack frame (sysenter/sysexit calling convention), I can't recall if that shows up as %sys, or just high overall CPU usage. Try using vmstat too, it'll tell you your number of context switches.

Someone else mentioned oprofile to profile kernel CPU usage too.

Tuesday, August 26, 2008, 17:48:35, blorb wrote:
well, if it's within the mysql, try mytop but if it's something
within the apahce or php i'm not sure myself...
but try logging while working

Tuesday, August 26, 2008, 17:52:00, Joel Merrick wrote:
Take a look at apache-top
You need the server-status page enabled and it has to be apache 2.
Also the mysql slow query log can be useful

Thanks for the tips, but I'm already using all this (mytop, slow log
and apache's status page) for performance tuning, but that wouldn't
help in case I need to track the system cpu usage.

Tuesday, August 26, 2008, 18:51:43, Marcin Sochacki wrote:
oprofile is a Linux tool for kernel profiling.
Regarding the original issue with high load, it's worth noting that USER
CPU usage is also very high and corresponding to the SYS usage. I think
that simply the web application you're using is not optimised enough, or
you just have a lot of traffic.

Oh yes, I was playing with oprofile some time ago, so I will try it.

The total CPU load is high now and mysql will be migrated to new
server next month. I just published the graph so that someone will be
able to tell if the SYS usage is too high compared to USER or not. And
yes, there's lot of traffic - about 40 hits/s for apache/php and another
100-120 hits/s for images using lighttpd (but 95% of them are served
from mem-cache.

  bYE, Marki

To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact

Michael Loftis
Modwest Operations Manager
Powerful, Affordable Web Hosting

Reply to: