Re: Reliably transferring a large amount of data from one machine to another over DSL

[Alvin Oga <aoga@ns.Linux-Consulting.com>]

On 9 Aug 2003, Shri Shrikumar wrote:

> Hi,
> 
> I have two machines connected to the internet using DSL. What I would
> like to do is backup one machine to the other. The files are compressed
> and encrypted and then transferred usinc SCP. Atleast, thats what its
> supposed to do. It does the first two steps fine but the transfer is
> regularly unsuccessful - it aborts with "connection lost" or similart
> (cant remember exact error message). The procedure is run from cron.
> 
> Any ideas on a reliable way to transfer 1 or 2 gb of data over dsl ?

for 1GB  of data transfer... your best bet is to do some thinking ...

a) did 1GB of data really change ??
	- if so .. you need a better backup/transfer mechanism

	- a perfect 1.54Mbps T1 line will transfer about 16GByte per day
	and add hiccups and bad nic and collisions and you're down to
	4-6GByte/day of useful data

b) send only "changes" to the other side .. NOT the 1GB of data
	- make -2, -8, -32, -96 day incremental changes
	( daily, weekly, monthly, quarterly )
	( alway make sure incrementals overlapp each other
	( so that if the last incremental fails, you're still covered

c) make a dvd and that is your "full" backup and send the dvd instead

d) get a full or burstable T1 from a reliable isp so that you dont get
   any timeouts or lost connections

c ya
alvin