Re: reverse proxy with directory replication capability

[Evuraan <evuraan@gmail.com>]

nevermind. i figured this out, proxy_store in nginx does exactly this.
go nginx!

location /images/ {
  root                 /data/www;
  error_page           404 = /fetch$uri;
}

location /fetch {
  internal;
  proxy_pass           http://backend;
  proxy_store          on;
  proxy_store_access   user:rw  group:rw  all:r;
  proxy_temp_path      /data/temp;
  alias                /data/www;
}

http://wiki.nginx.org/NginxHttpProxyModule#proxy_store

2010/9/21 Evuraan <evuraan@gmail.com>:
> I've a web server (server1) with 30G data footprint - i used to rsync
> the directory to a rather frail (but expensive) VM, but the sheer no.
> of files is pushing rsync out of memory on the VM.
>
> ever since rsync failed on me, i  run squid as reverse proxy on the VM
> against server1 - it works "ok", but let me ask this anyway:
>
> i am looking for a reverse proxy or a similar method, which would
> replicate the directory structure *exactly as* the backend as it
> serves clients, populate the structure as it fetches objects from the
> backend. if the object requested is in its cache, serves it locally
> (no IMS queries to server1), if not, it fetches from the backend and
> also populates the directory structure. if the backend is dead, it
> serves what it can from its own directory structure, and returns 404s
> for stuff that it does not have and failed to ask backend for.
>
> i am hoping this ask makes sense.
>
> many thanks in advance for your advice..!
>