[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: high performance, highly available web clusters

On Thu, 20 May 2004 15:48, David Wilk <myca@cia-g.com> wrote:
> The cluster is comprised of a load-balancer, several web servers
> connected to a redundant pair of NFS servers and a redundant pair of
> MySQL servers.  The current bottle-neck is, of course, the NFS servers.
> However, the entire thing needs an increase in capacity by several
> times.

The first thing I would do in such a situation is remove the redundant NFS 
servers.  I have found the NFS client code in Linux to be quite fragile and 
wouldn't be surprised if a cluster fail-over killed all the NFS clients (a 
problem I often had in Solaris 2.6).

> However, for alot less money, one could simply do away with the file
> server entirely.   Since this is static content, one could keep these
> files locally on the webservers and push the content out from a central
> server via rsync.  I figure a pair of redundant internal web server
> 'staging servers' could be used for content update.  Once tested, the
> update could be pushed to the production servers with a script using
> rsync and ssh.  Each server, would of course, require fast and redundant
> disk subsystems.

Yes, that's a good option.  I designed something similar for an ISP I used to 
work for, never got around to implementing it though.  My idea was to have a 
cron job watch the FTP logs to launch rsync.  That way rsync would only try 
to copy the files that were most recently updated.  There would be a daily 
rsync cron job to cover for any problems in launching rsync from ftpd.

With local disks you get much more bandwidth (even a Gig-E link can't compare 
with a local disk), better reliability, and you can use the kernel-httpd if 
you need even better performance for static content.  Finally such a design 
allows you to have a virtually unlimited number of web servers.

http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/    Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page

Reply to: