Re: Can we build a proper email cluster? (was: Re: Why is debian.org email so unreliable?)
On Thu, 14 Oct 2004, Russell Coker wrote:
> On Thu, 14 Oct 2004 01:47, Henrique de Moraes Holschuh <hmh@debian.org> wrote:
> > On Wed, 13 Oct 2004, Russell Coker wrote:
> > > On Wed, 13 Oct 2004 07:29, Henrique de Moraes Holschuh <hmh@debian.org>
> wrote:
> > > > We have a lot of resources, why can't we invest some of them into a
> > > > small three or four machine cluster to handle all debian email (MLs
> > > > included),
> > >
> > > A four machine cluster can be used for the entire email needs of a
> > > 500,000 user ISP. I really doubt that we need so much hardware.
> >
> > Including the needed redundancy (two MX at least), and a mailing list
> > processing facility that absolutely has to have AV and AntiSPAM measures at
> > least on the level gluck has right now?
>
> The Debian email isn't that big. We can do it all on a single machine
> (including spamassasin etc) with capacity to spare.
Or we can do it in two, with capacity to spare AND no downtime.
> One machine should be able to do it with AV and antispam. Four AV/antispam
> machines can handle the load for an ISP with almost 1,500,000 users, one
> should do for Debian.
That depends on how much delay you want to have when processing mail. It'd
be nice to know how many messages/minute @d.o and gluck receive, to stop
guessing, though.
> > But we really should have two of them (in
> > different backbones), with the same priority as MX.
>
> Why?
No downtime. Easy maintenance. Redundancy when we have network problems
(these are rare, thank god).
> > It would be nice to
> > have a third MTA with less priority and heavier anti-spam machinery
> > installed.
>
> Bad idea.
Ok.
> > > OK, having a single dedicated mail server instead of a general machine
> > > like master makes sense.
> >
> > Two so that we have some redundancy, please. IMHO email is important enough
> > in Debian to deserve two full MX boxes (that never forward to one another).
>
> As long as the machine is fixed within four days of a problem we don't need
> more than one. Email can be delayed, it's something you have to get used to.
And while that email is being delayed, our work suffers, and there could
even be security concerns as well. Developer time IS an important resource,
I don't think we should be wasting it because we don't want to have a second
MX. Would you set up a mail system for any ISP (including small, 1000-user
ones) with only one MX?
> We don't need high-end hardware. Debian's email requirements are nothing
> compared to any serious ISP.
True. But we don't need cheap-ass, will-break hardware either. Debian's
admin requirements are different. The less on-site intervention needed, the
better.
> > > http://www.umem.com/16GB_Battery_Backed_PCI_NVRAM.html
> >
> > How much? It certainly looks very good.
>
> If you want to buy one then you have to apply for a quote.
I.e.: quite expensive. argh. It is very nice to know about it, though.
> > > I've run an ISP with more than 1,000,000 users with LDAP used for the
> > > back-end. The way it worked was that mail came to front-end servers
> > > which did LDAP lookups to determine which back-end server to deliver to.
> > > The
> >
> > I meant LDAP being used for the MTA routing and and rewriting. That's far
> > more than one lookup per mail message :(
>
> Yes, I've done all that too. It's really no big deal. Lots of Debian
> developers have run servers that make all Debian's servers look like toys by
> comparison.
So do I. And I can tell you that I experienced a lot of improvement when
big mass-delivery mail hits, on the order of _minutes_ (thousands of
recipients, every one of them causes postfix to generate a minimum of 4 LDAP
searches, due to the way the LDAP maps were required to be deployed), and
the way postfix map lookup happens. Moving that to a hash DB sped things up
considerably.
And our requirements for the LDAP cluster went down a lot too, so it was all
benefits without a single drawback.
> > Well, we are talking MTA and not mail stores. The LDAP workload on a MTA
> > is usually quite different for the one in a mail store.
>
> Yes, it should be less load because you don't have POP or IMAP checks.
Try the other way around... Not all MTA setups do a single LDAP lookup per
recipient... of course, if @d.o requires only one lookup, then we don't need
to worry, but...
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh
Reply to: