Re: mail server w/ 65000++ users
On Fri, 12 May 2000, Craig Sanders wrote:
>On Fri, May 12, 2000 at 04:10:40PM +0800, Chad A. Adlawan wrote:
>> does anybody have any URL's or docs w/ talks on how to build
>> a mail server (both Exim and Sendmail are OK w/ me) with more
>> than 65,000 users ? i.e., what are the available methods (and
>> what are the best ones) of having mail users w/o having them on
>> /etc/passwd.
>
>i'd suggest postfix + cyrus. from comments in the postfix-users list, it
>seems to be a nearly ideal combination for doing what you want.
I'd suggest Postfix + the Qmail POP server. Postfix (and Postfix-tls) are in
Debian. There's a package of the Qmail source which allows you to compile
your own Qmail POP server.
Cyrus uses a different mail storage format to anything else and sequesters
all your mail.
For users who aren't in the /etc/passwd file use LDAP, give all users the
same UID and with /dev/null as the shell (so they can't login).
Use the NSS modules for LDAP.
>hint, for performance you probably want to look at a machine with
>multiple fast scsci drives for the mail spool (raid striping), formatted
>with reiserfs. and lots of memory, of course. CPU speed isn't a big
>issue - mail systems are I/O bound.
Last year I was working on an AIX machine (AIX is slow) that had old 2G and
4G SSA drives (drive performance was less than my Thinkpad in every test).
The AIX machine ran 27000 mail accounts, an Oracle server, and some shell
accounts. After I had finished with it performance was quite OK.
It really depends on the type of access the machine will get. 27000
university students don't produce much load (especially when most of them are
arts students who only check mail once a week). 1000 people on a corporate
network sending emails with Word and Excel documents attached will produce
1000 times the load.
When mail is being delivered and immidiately downloaded via POP (no mail left
on server) my Thinkpad 600E (10G IDE hard drive, Celeron 400) can do 20G of
email traffic a day. An ISP with >500000 users I know of has about 15G of
email a day.
For best performance have no direct TCP connections between your mail server
and the outside world. Have the MX records point to an inbound-relay which
sends the mail to the real server. Have the clients SMTP relay address point
to a machine that's configured to just be an outbound relay. Have your
server setup with ipchains or TCP wrappers to deny SMTP connections from
machines other than the inbound relay.
When mail comes from the Internet it generally comes in slowly, and in
spurts. This hurts the caching on the queue partition. Have the mail come
in from the inbound relay (or relays, you'll need several for a big system)
in only a small number of TCP connections. That way data will generally
never be read from the queue partition (it'll be in the cache).
Have seperate physical media for the queue file system. All writes of email
data are synchronous. Writing the queue data generally involves creating 2
or 3 queue files synchronously and one mail-store file (for maildir). This
can make the queue a performance bottleneck for the mail system. Have a
seperate pair of hard drives in RAID-1 setup for the queue and you'll save
disk bandwidth for where you want it.
Run bonnie++ (it's a Debian package) with the "-b" option to see how
synchronous writes slow things down (you'll probably be surprised).
Run zcav (part of the bonnie++ suite) on your hard drives to make sure that
you only use the best performing parts in your RAID arrays.
Run Postal (it's a Debian package) to test the overall performance of your
mail delivery system, but note that it doesn't test the affects of slow
transfers - it tests what you'll get when you have inbound and outbound
relays. It's a Debian package, but the latest version is on Sourceforge and
on http://www.coker.com.au/postal/ (the version in Debian is old).
I think that a machine with 512M of RAM, Postfix, Qmail-POP, and two of the
IBM 34G IDE hard drives in RAID-1 configuration running ReiserFS will provide
all the performance you need (unless you've got the office email system with
Word documents being mailed around).
Test it out with Postal, if that hardware isn't enough then try two RAID-1
sets, one for queue and logs, the other for the mail data.
Russell Coker
PS I am the author of Postal and the primary author of the Bonnie++ suite.
--
My current location - X marks the spot.
X
X
X
Reply to: