Re: Could RAM possibly be just 3-4 times faster than bare hdd writes and reads? or, is the Linux kernel doing its 'magic' in the bg? or, ...

[<tomas@tuxteam.de>]

On Wed, Jun 17, 2020 at 01:23:41PM -0700, David Christensen wrote:

[...]

> I was referring to the 'fdatasync', 'fsync', 'dsync', 'sync', and
> 'nocache' options to dd(1).  Given the terse manual page, and a
> unwillingness to crawl the dd(1) and/or kernel code, I can only
> guess at my understanding of what is happening.  But, all of those
> options seem to imply some opposite of asynchronous or buffered I/O
> as I know it.

FWIW, the info file is more --uh-- informative on those. I copy
the bits which seem related to this thread. Elisions marked with
[...]:

‘oflag=FLAG[,FLAG]...’
     Access the output file using the flags specified by the FLAG
     argument(s).  (No spaces around any comma(s).)

     Here are the flags.  Not every flag is supported on every operating
     system.

     [...]

     ‘dsync’
          Use synchronized I/O for data.  For the output file, this
          forces a physical write of output data on each write.  For the
          input file, this flag can matter when reading from a remote
          file that has been written to synchronously by some other
          process.  Metadata (e.g., last-access and last-modified time)
          is not necessarily synchronized.

     ‘sync’
          Use synchronized I/O for both data and metadata.

     ‘nocache’
          Request to discard the system data cache for a file.  When
          count=0 all cached data for the file is specified, otherwise
          the cache is dropped for the processed portion of the file.
          Also when count=0, failure to discard the cache is diagnosed
          and reflected in the exit status.

          Note data that is not already persisted to storage will not be
          discarded from cache, so note the use of the “sync” options in
          the examples below, which are used to maximize the
          effectiveness of the ‘nocache’ flag.

          Here are some usage examples:

               # Advise to drop cache for whole file
               dd if=ifile iflag=nocache count=0

               # Ensure drop cache for the whole file
               dd of=ofile oflag=nocache conv=notrunc,fdatasync count=0

               # Advise to drop cache for part of file
               # Note the kernel will only consider complete and
               # already persisted pages.
               dd if=ifile iflag=nocache skip=10 count=10 of=/dev/null

               # Stream data using just the read-ahead cache.
               # See also the ‘direct’ flag.
               dd if=ifile of=ofile iflag=nocache oflag=nocache,sync


So to test disk write speed, 'dsync' seems the way to go. When dumping
to a device, there are no metadata (am I right there?), so probably
again you want 'dsync'.

I don't know what 'nocache' would do for writing. For reading, it seems
to make sense to avoid huge files (think a video) uselessly clobbering
your cache (you're going to look at the blocks once, right?).

Cheers
-- tomás

Attachment: signature.asc
Description: Digital signature