Re: hardware/optimizations for a download-webserver
On Sat, 17 Jul 2004 14:09, Nate Duehr <nate@natetech.com> wrote:
> Other good ways to do this include a shared RAID'ed network filesystem
> on a central box and two front-end boxes that are load-balanced with a
> hardware load-balancer. That gets into the "must be up 24/7" realm, or
> close to it. I worked on an environment that did this with a hardware
> NFS server (NetApp) and the front-ends could be up or down, it just
> didn't matter... as long as enough of them were up to handle the
> current load.
There are two ways of doing the "storage is available to two machines. One is
to have a shared SCSI bus and clustering software - but this is a major cause
of clusters being less reliable than stand-along machines in my experience.
The other way is using an NFS server.
For an NFS server there are two main options, one is using a Linux NFS server
and the other is a dedicated hardware box such as NetApp. The problem with
using a Linux machine is that Linux as an NFS server is probably no more
reliable than Linux as an Apache server (and may be less reliable). In
addition you have network issues etc, so you may as well just have a single
machine. Using a NetApp is expensive but gives some nice features in terms
of backup etc (most of which can be done on Linux if you have the time and
knowledge). A NetApp Filer should be more reliable than a Linux NFS server,
but you still have issues with the Linux NFS client code.
My best idea for a clustered web server was to have a master machine that
content is uploaded to via a modified FTP server. The FTP server would
launch rsync after the file transfer to update the affected tree. Cron jobs
would periodically rsync the lot in case the FTP server didn't correctly
launch the rsync job. That way there are machines that have no dependencies
on each other. The idea was to use IPVS to direct traffic to all the
servers.
--
http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/ Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/ My home page
Reply to: