Re: How long will this take?
On 2020-06-26 18:25, David Wright wrote:
On Fri 26 Jun 2020 at 15:06:31 (-0700), David Christensen wrote:
On 2020-06-26 06:07, David Wright wrote:
On this slow machine with an oldish PATA disk,
I can get about 75% speed from urandom, 15MB/s vs 20MB/s on a 29GiB
partition (no encryption). There's a noticeable slowdown because,
I presume, the machine runs a bit short of entropy after a while.
I think you are noticing a slowdown when the Linux write buffer fills.
I'm not sure where these write buffers might be hiding: the
2000-vintage PC has 512MB memory, and the same size swap partition,
though the latter is on a disk constructed one month earlier than the
target disk (Feb/Mar 2008). The target disk has 8MB of cache.
With a leisurely determination of dd's PID, my first USR1 poke
occurred no earlier than after 4GB of copying, over three minutes in.
I seem to recall that most of my EIDE interfaces and drives were 100
MB/s. (A few were 133 MB/s.) So, bulk reads or writes can completely
use an 8 MB cache in a fraction of a second.
top(1) reports memory statistics on line 4. I believe "buff/cache" is
the amount of memory being used for I/O write buffering and read
caching. Line 5 has statics for swap. I do not know if memory write
buffer / read cache usage interacts with swap usage, but it would not
surprise me. top(1) should be able to show you.
Perhaps I misinterpreted your "slowdown" statement. I assumed you ran a
command similar to:
# dd if=/dev/urandom of=/dev/sdxn bs=1M status=progress
dd(1) is copying PRN data from the CPU to the kernel write buffer (in
memory) and the kernel input/ output stack is copying from the write
buffer to the HDD (likely via direct memory access, DMA). The
'status=progress' option will cause dd(1) to display the rate at which
the write buffer is being filled. I am not sure how to monitor the rate
at which the write buffer is being drained. Assuming the write buffer
is initially empty, the filling process is "fast", and the draining
process is "slow" when the above command is started, dd(1) should show
fast throughput until the write buffer fills and then show slow
throughput for the remainder of the transfer. And, without a 'sync'
option to dd(1), dd(1) will exit and the shell will display the next
prompt as the final write buffer contents are being written to the HDD
(e.g. the HDD will be busy for a short while after dd(1) is finished).
Another possibility -- magnetic disk drives have more sectors in outer
tracks (lower sector number) than they have in inner tracks (higher
sector number). When filling an entire drive, I have seen the transfer
rate drop by 40~50% over the duration of the transfer. This is normal.
Is this what you are referring to?