On Wed, Aug 08, 2012 at 03:14:50PM +0800, lina wrote: > Hi, > > It's a bit big data to transfer, around 1.1 T, > > from one server to another server. > > I checked that rsync is faster than scp, > but in my situations rsync has elapsed for 1 hour, I guess the network > is also a problem, > > Here I wish to know are there some tools (better default) can use for > fast transferring, regardless the security reason, my data is just > some data, no need special security care. > In addition to the suggestions mentioned by other people, consider the compressability of your data. I don't believe it's possible to definitely predict the trade-offs here (it depends on how well the data compresses, basically, but also on how your CPUs compare to the network bandwidth, but you may find that spending some time compressing the data reduces the overall time. In that case try, the following. On the receiver: $ nc -l -p 12345 | $COMP -d | pv > outfile and on the sender: $ pv infile | $COMP | nc receiver 12345 where $COMP is your preferred streaming compressor (gzip, bzip2 and xz should all work nicely here). By the way, "pv" (package: pv) is a useful pipeline-viewer and will show you progress (on the sending side) as well as throughput levels.
Attachment:
signature.asc
Description: Digital signature