On Thu, Sep 09, 2004 at 06:03:20AM +1000, Russell Coker wrote:
You have to either be doing something very intensive or very wrong to need
more than one server for 20K users. Last time I did this I got 250K users
per server, and I believe that I could have easily doubled that if I was
allowed to choose the hardware.
We have a little over 10K users, and the disk subsystem seems to be the
bottleneck. When we reach about 600 read transactions + 150 write
transactions per second (as reported by sar -b), the load average starts
to grow expotentially instead of proportionally. There are about 20K
sectors read, and 3K written per second. (That was before I turned noatime
on. After that we had about 2K sector writes and 70 write transactions
less, and load average dropped to a more sane value - about 3, instead
of 20.)
More than 90% of the disk transactions are on the (logical) disk where
mail is stored. The only processes which touch that disk, are qmail
delivery processes (qmail handed mail by another SMTP-IN box: 0.8 local
deliveries per second) and courierpop3d processes (7.2 logins per
second).
We are using an "Intel SRCU42X" SCSI RAID controller, and the logical
disk which caries mail is made of 3 Fujitsu 36GB 15K RPM disks.
Please tell me, what problem we are facing? Is the hardware so weak? Is
it underperforming? Or maybe our load is exceptionally high? I can
provide more statistics if they are needed.
Also, did you implement virus/spam scanning on that box?
kind regards,
Marcin