Re: How long will this take?
On Mon, Jun 08, 2020 at 08:22:39PM +0000, Matthew Campbell wrote:
I bought a new 4 terrabyte hard drive that is connected with a USB cable using
USB2. It took about 32 hours to read every sector on the drive to look for bad
sectors. I started blanking the sectors using /dev/zero last Friday night. It
still isn't done. Is there a way I can find out how much data a particular
process has written to the disk? I'm using Debian 10.4.
dd if=/dev/zero of=/dev/sdb ibs=4096 count=976754646
This command line gets data in 4k chunks from /dev/zero and then writes
them to the disk in 512 byte chunks. That's pretty much the worst
possible case for writing to a disk. You want "bs", not "ibs". I'd
suggest
dd if=/dev/zero of=/dev/sdb bs=64k
and I wouldn't bother trying to calculate a count if you're trying to
overwrite the entire disk (any human is likely to screw up the math and
there's no actual benefit).
IME performance peaks at 16-64k. Beyond that things don't improve, and
can potentially get worse or cause other issues.
On Mon, Jun 08, 2020 at 04:33:29PM -0400, Dan Ritter wrote:
Matthew Campbell wrote:
I bought a new 4 terrabyte hard drive that is connected with a USB cable using USB2. It took about 32 hours to read every sector on the drive to look for bad sectors. I started blanking the sectors using /dev/zero last Friday night. It still isn't done. Is there a way I can find out how much data a particular process has written to the disk? I'm using Debian 10.4.
dd if=/dev/zero of=/dev/sdb ibs=4096 count=976754646
USB2 disks are good for about 25MB/s.
I've been getting sustained USB2 disk writes in the low 40MB/s range for
more than 15 years. I'd suggest either checking that you're using a
reasonable block size or getting a better USB2 adapter. 25MB/s is
definitely low. That said, these days you'd be much better off with
USB3 because any modern disk is going to bottleneck on USB2.
On Mon, Jun 08, 2020 at 06:02:46PM -0400, Dan Ritter wrote:
deloptes wrote:
Dan Ritter wrote:
> USB2 disks are good for about 25MB/s.
>
Where do you have those numbers?
USB 2.0 standard can theoretically transfer data at a very high 480 megabits
per second (mbps), or 60 megabytes per second (MBps) [for example in
wikipedia).
Yes, that's the theory. In years of running several USB 2.0
attached disks, I found that they were actually good for about
25MB/s long-term. Bursts to 37MB/s were not uncommon.
# dd if=/dev/zero of=/dev/sdh bs=64k count=10000 conv=fdatasync
10000+0 records in
10000+0 records out
655360000 bytes (655 MB, 625 MiB) copied, 15.1168 s, 43.4 MB/s
The OP's 4K writes will be particularly badly performing.
OP was doing 512 byte writes.
(same adapter/disk)
# dd if=/dev/zero of=/dev/sdh ibs=4096 count=10000 conv=fdatasync
10000+0 records in
80000+0 records out
40960000 bytes (41 MB, 39 MiB) copied, 3.15622 s, 13.0 MB/s
I didn't have the patience to write the same amount of data. :D
Reply to: