Re: Sendmail or Qmail ? ..
On Sat, 2003-09-06 at 22:34, Russell Coker wrote:
> On Sat, 6 Sep 2003 00:08, Eric Sproul wrote:
> > until the entire message has been received and processed, the receiving
> > MTA is not responsible for the message. In fact, I think this is
> > RFC-specified. Why then, if the receiver isn't responsible, would it
> > want to spend disk I/O queuing a message that may end up being rejected
> > or may fail to come completely in?
>
> The incidence of messages that fail part way through is quite low. Expecially
> in communications between big servers (which corresponds to a large portion
> of the non-spam traffic). Optimising for the common case makes sense to me.
>
I should think, though, that using a milter that will reject a message
based on the DATA content as it is streaming in would increase the
likelihood of such occurrences. For instance, a virus-scanning milter
will reject the message as soon as it sees a signature, causing the
conversation to be aborted. During a large outbreak, a non-trivial
percentage of SMTP traffic may be viruses, so there is still a benefit
to buffering in RAM vs. queueing to disk. However, I do agree that
asynchronous writes and ext3 unlink-before-commit would mitigate the I/O
hit.
> > As many other ISP admins know, a large percentage of customers
> > are the psychotic kind, prone to POPing their multi-MB mailboxes every
> > $%^&#@! minute, and leaving all the messages on the server. This puts a
> > non-trivial strain on even a fairly hefty dual-x86 box with H/W RAID5
> > and 2GB of RAM.
>
> I have not noticed that. I have only noticed a very small portion of users
> doing that. With 1,000,000 users the number of psychotic POP users is small
> enough that you can deal with them individually.
>
> Maybe customers of Dutch ISPs are smarter than those of whichever country you
> are in.
I'm in the US, but let's not start a flame thread over the collective
intelligence of our respective populations... ;^P
> Why not change your POP server to instead of rejecting the connections to put
> gratuitous delays. So if the time since the last connection is < 5 minutes
> then make every operation take an extra 18 seconds (some pop servers have 20
> second time outs). That would delay a minimal POP session by 72 seconds
> which better than halve the load.
I agree, but the political problem remains. Whether I outright reject
or impose delays (which will probably generate timeout errors on the
client), it generates calls from confused people. But, rather than
spend the time explaining the situation, the company would rather the
calls not come in at all. Ah, the joy of engineering.
> > I did some more figuring on our mail volume and found that even though
> > each of our 4 mail routers processes 11-12 messages/second (each message
> > requires up to 20 LDAP lookups and a milter for spam filtering), I see
>
> A caching LDAP proxy would be good for this situation. Converting 20 LDAP
> lookups over the network to a single LDAP lookup and 19 accesses to a local
> cache daemon should provide some significant benefits.
Yes, we run local slapd daemons on all the mailrouters, updated from a
central master. We see very good performance from this setup, as noted
above.
Eric
Reply to: