Re: Speed Problem Copying Files

["Thomas Schmitt" <scdbackup@gmx.net>]

Hi,

Lothar Schilling wrote:
> Fast enough...
> dd if=/daten/testfile bs=1G oflag=direct of=/daten/testfile2
> 10737418240 Bytes (11 GB, 10 GiB) kopiert, 72,7297 s, 148 MB/s

So this is sufficiently fast, but

  cp /daten/testfile /daten/testfile2

lasts 2000 seconds ?

> dd if=/daten/testfile of=/dev/null
> 10737418240 Bytes (11 GB, 10 GiB) kopiert, 36,6887 s, 293 MB/s

It is not about the number of read/write operations.
(How fast is bs=1G on the second try ?)

------------------------------------------------------------------------

It is also not about the filesystem and its health status.
So where could be any difference ?

dd is an i/o dinosaur. It performs read/write without much mind.

cp uses a centralized copy facility by a function named "copy ()".
This facility makes much ado about sparse files:
  https://sources.debian.org/src/coreutils/8.30-3/src/copy.c/#L387
  https://sources.debian.org/src/coreutils/8.30-3/src/copy.c/#L224

man cp says:

  By default, sparse SOURCE files are detected by a crude  heuristic  and
  the corresponding DEST file is made sparse as well.  That is the behav‐
  ior selected by --sparse=auto.  Specify  --sparse=always  to  create  a
  sparse  DEST  file  whenever  the  SOURCE  file  contains a long enough
  sequence of zero bytes.  Use  --sparse=never  to  inhibit  creation  of
  sparse files.

So i'd try cp with --sparse=never in order to see whether it then resembles
the behavior of dd.


Have a nice day :)

Thomas