[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: hardware/optimizations for a download-webserver

On Sat, 17 Jul 2004 14:09, Nate Duehr <nate@natetech.com> wrote:
> Other good ways to do this include a shared RAID'ed network filesystem
> on a central box and two front-end boxes that are load-balanced with a
> hardware load-balancer.  That gets into the "must be up 24/7" realm, or
> close to it.  I worked on an environment that did this with a hardware
> NFS server (NetApp) and the front-ends could be up or down, it just
> didn't matter... as long as enough of them were up to handle the
> current load.

There are two ways of doing the "storage is available to two machines.  One is 
to have a shared SCSI bus and clustering software - but this is a major cause 
of clusters being less reliable than stand-along machines in my experience.  
The other way is using an NFS server.

For an NFS server there are two main options, one is using a Linux NFS server 
and the other is a dedicated hardware box such as NetApp.  The problem with 
using a Linux machine is that Linux as an NFS server is probably no more 
reliable than Linux as an Apache server (and may be less reliable).  In 
addition you have network issues etc, so you may as well just have a single 
machine.  Using a NetApp is expensive but gives some nice features in terms 
of backup etc (most of which can be done on Linux if you have the time and 
knowledge).  A NetApp Filer should be more reliable than a Linux NFS server, 
but you still have issues with the Linux NFS client code.

My best idea for a clustered web server was to have a master machine that 
content is uploaded to via a modified FTP server.  The FTP server would 
launch rsync after the file transfer to update the affected tree.  Cron jobs 
would periodically rsync the lot in case the FTP server didn't correctly 
launch the rsync job.  That way there are machines that have no dependencies 
on each other.  The idea was to use IPVS to direct traffic to all the 

http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/    Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page

Reply to: