Re: Transfer data between two comps without network
On Thu, 18 May 2000, Dariush Pietrzak wrote:
>> Assuming you are worried by people with promiscuous ethernet cards,
>> packet-sniffing. Put in a second NIC, run a crossover UTP? I assume the
>.. encrypting would solve that problem. or private network between two
>ccomps.
>And - if I could connect those two comps by some network daily data
>transfer would rapidly go down - 100, 1000times less.
>problem is - machine with source data contains security-sensitive
>information, which my employee wants to be physically separated from
>network.
Using CD-ROMs would take 60+ CDs.
Using DVD would take 6 or 12 disks (do they support writable 10G DVD's yet?).
If using external media then you must encrypt the data first, so the time
taken to transfer the data is compression time + write time + transport time
+ read time + decompression time. The time taken should be considerably less
than 24 hours for obvious reasons.
I have included a message I wrote to some colleagues comparing different
options for transferring files over the network. Tests were done on
moderately high-end Sun machines >300MHz UltraSPARC processors talking over a
switched full-duplex fast Ethernet. The machines were also in use for other
tasks, so if there was no load then the results would be slightly higher, but
the overall trend would remain.
This may interest some of you who use ssh/scp a lot, and is also relevant to
this discussion.
One thing to note that Gigabit Ethernet is useless if you use ssh, as ssh
can't get near to saturating fast Ethernet.
There is an option in ssh to choose the cypher to use. The default cypher is
3DES which has withstood numerous attack attempts, but is quite slow.
Another option is blowfish which is quite strong (an AES candidate) but
hasn't withstood the decades of attack that 3DES has.
Blowfish is a much faster cypher:
bash-2.02$ time scp aaa001:/netscape/server4/https-portal/logs/errors.02May-1131AM .
errors.02May-1131AM | 235084 KB | 736.9 kB/s | ETA: 00:00:00 | 99%
real 5m31.620s user 3m20.520s sys 0m15.250s
bash-2.02$ time scp -c blowfish aaa001:/netscape/server4/https-portal/logs/errors.02May-1131AM .
errors.02May-1131AM | 234004 KB | 2571.5 kB/s | ETA: 00:00:00 |
99% real 1m30.932s user 1m1.160s sys 0m14.010s
I recommend using blowfish for the large scp operations (such as copying
gigabytes of log files) to reduce the time taken to copy the data, and also
to reduce the amount of CPU load used (on both the source and destination
machines).
Now here's a run using 3des and compression:
bash-2.02$ time scp -C aaa001:/netscape/server4/https-portal/logs/errors.02May-1131AM .
errors.02May-1131AM | 235751 KB | 3683.6 kB/s | ETA: 00:00:00 |
100% real 1m3.898s user 0m14.320s
sys 0m7.170s
It makes things a bit faster than even blowfish because after compressing the
data (web logs compress well) it's less to encrypt (and encryption seems to be
significantly slower than compression).
Now here's the results of blowfish and compression. As you can see for web
logs this is the best option, 6 times faster than the default.
bash-2.02$ time scp -c blowfish -C aaa001:/netscape/server4/https-portal/logs/errors.02May-1131AM .
errors.02May-1131AM | 235751 KB | 4064.7 kB/s | ETA: 00:00:00 |
100% real 0m57.237s user 0m14.760s
sys 0m6.770s
--
My current location - X marks the spot.
X
X
X
Reply to: