Re: Sendmail or Qmail ? ..
On Sat, 6 Sep 2003 00:08, Eric Sproul wrote:
> On Fri, 2003-09-05 at 01:14, Russell Coker wrote:
> > I was under the impression that Sendmail also queues everything to disk.
> > How does it's queue operate then?
>
> While the message is coming in, Sendmail buffers the message to memory,
> optionally piping the DATA portion to a socket (for milter scanning).
> Only after the <CR>.<CR> does Sendmail accept responsibility for the
> message (providing it was not rejected by a milter) and queue it. Some
> might say this risky (power outages and such) but I would counter that
It's not risky, if anything goes wrong before the <CR>.<CR> the message must
be discarded.
However in a modern system there is no performance benefit in buffering to
memory over writing to a file without sync(). For a large message you
probably want to write it to disk instead of keeping it in memory to avoid
thrashing.
> until the entire message has been received and processed, the receiving
> MTA is not responsible for the message. In fact, I think this is
> RFC-specified. Why then, if the receiver isn't responsible, would it
> want to spend disk I/O queuing a message that may end up being rejected
> or may fail to come completely in?
The incidence of messages that fail part way through is quite low. Expecially
in communications between big servers (which corresponds to a large portion
of the non-spam traffic). Optimising for the common case makes sense to me.
Also I believe that in Ext3 if you write data to a file and then unlink the
file before the data is committed to disk then the data will never be
written. So there seems no loss as long as the file isn't opened with O_SYNC
and you don't call fsync() (and no-one calls sync()).
> > I'm not sure what the situation was like in 1999, now Qmail and LDAP
> > support is adequate.
>
> But only with patches to the source code. And since it sounds like you
> can't distribute modified binaries, you'd have to patch/build qmail on
> every MTA. I choose not to install a development environment on my
> production servers. I distribute only binary packages with apt from a
> central repository.
True, this is a significant issue, which is why I recommend Postfix.
> > You need two mail storage servers for 60,000 accounts?
>
> Yes. Actually we now have 4 mail stores. We have discovered, at least
> for our situation, that it is not wise to put more than 20K accounts on
> a single mailstore. This is not so much for the mail delivery, but for
> POP3. As many other ISP admins know, a large percentage of customers
> are the psychotic kind, prone to POPing their multi-MB mailboxes every
> $%^&#@! minute, and leaving all the messages on the server. This puts a
> non-trivial strain on even a fairly hefty dual-x86 box with H/W RAID5
> and 2GB of RAM.
I have not noticed that. I have only noticed a very small portion of users
doing that. With 1,000,000 users the number of psychotic POP users is small
enough that you can deal with them individually.
Maybe customers of Dutch ISPs are smarter than those of whichever country you
are in.
> Yes, I know we could set a larger minimum interval for POP, but the
> political implications of generating tech support calls about "why can't
> I POP my mail?" prevent it. Don't get me started on THAT. 8^o
Why not change your POP server to instead of rejecting the connections to put
gratuitous delays. So if the time since the last connection is < 5 minutes
then make every operation take an extra 18 seconds (some pop servers have 20
second time outs). That would delay a minimal POP session by 72 seconds
which better than halve the load.
Also if you use Maildir format the impact of checking for mail should not be
particularly high. The dentry cache is all that's consulted, give the server
plenty of RAM and disk reads should be quite rare.
> I did some more figuring on our mail volume and found that even though
> each of our 4 mail routers processes 11-12 messages/second (each message
> requires up to 20 LDAP lookups and a milter for spam filtering), I see
A caching LDAP proxy would be good for this situation. Converting 20 LDAP
lookups over the network to a single LDAP lookup and 19 accesses to a local
cache daemon should provide some significant benefits.
--
http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/ Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/ My home page
Reply to: