[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Growisofs input cache (get away from dd?)

Thomas Schmitt wrote:

Bill Davidsen wrote:
not passing
multiple GB of once-used data through the system buffers and blowing
everything else out.

I see some effects which could make the O_DIRECT
approach less superior to normal reading than your
scenario would let expect:

- If the (ISO) image is freshly generated then the
  formatter program just blew out everything else
  anyway. And for the ISO formatter it makes few
  sense to read O_DIRECT as it has a random access
  pattern on eventually small files.

You are still thinking about benefit to the image generation application rather than the overall system. The reads are small and scattered, but they are single use i/o, meaning that it is highly unlikely that the data will be used before it passes out of the cache. Therefore there is no gain to the ISO creation application from using a buffered read, and the certainty that at least 4GB of data used by other applications or the system itself will be evicted from the cache. And the same thing holds true for writing the ISO image, unless you have a machine with 4GB or more to use for write cache, the data will be out of cache before the media is burned.

And even if it isn't, there's no performance gain to mention, since all of the burning applications worth consideration use readahead buffering and therefore will have the data to burn before it's needed. So the possible gain ranges from "very little" to "none" for the burning application, while the possible slowdown to other applications on the machine can be up to 20% on a machine doing a lot of transactions. That's measured in the real world, I used to be project leader for the NNTP server group at a national ISP, and compressing the log files would drop the throughput that much under load. Putting dd in front of the file read, using O_DIRECT, eliminated the problem, and only added 3-4% to the real time for the process.

- The cache area in RAM is nowadays quite large
  and i understand that the least recently used
  cache chunks get thrown out. While a growisofs
  read chunk is ageing in the cache for several
  seconds, any busy process can refresh its own
  chunks' read time.

I don't know how large your machines are, but I certainly don't want 4.7GB, or 8.5GB, or 25GB to be used for caching something used by one application which benefits marginally if at all from the caching. And if you have large memory, run mkisofs to generate a "most of memory" size image, type "sync" in one window and try to do something requiring reading disk in another. All that data gets queued to the disks, commands dropped into NCQ if you have good disks, and reads virtually stop (Linux) or slow to a crawl (Solaris).

A good reason why caching data is bad unless it will be read again soon.

  So i expect that the cache of growisofs' input
  stream can only blow out chunks which are as
  fewly used as its own chunks.
This would explain why i never saw the need to do
O_DIRECT reading: my ISO images are fresh, if
they are big. Typically they get piped directly
into the burner process.

Which doesn't mean that using O_DIRECT for reading the input files would not be nicer for your cache.

I have one use case, though, where i burn two identical
copies of the same DVD image for redundancy reasons.
One-by-one via a single burner drive from an
image file on disk onto 16x DVD+R.
Even then i experience no undue impact on other
I also never noticed a difference when i switched
from growisofs to my own programs for that.

Burning is a slow process relative to creating the image, so that is a lesser impact. But by pulling an image through the cache you will have an impact on other things using the cache, although even 16x is pretty slow i/o compared to current disk speed (and I bet you have good ones).
Is there a realistic scenario where O_DIRECT is
reproduciblrye superior to normally buffered reading ?
I ponder whether i shall offer a O_DIRECT option
with the source objects of libburn.

That depends on how large your machine is. ;-) If you have 64GB a Blu-Ray will fit. But on a typical machine with 2-8GB, I would expect that flushing cache when the dirty times runs out would make the system notably slower doing things needing read (this depends on tuning somewhat).

You could try:
1 - drop cache (echo 1 >/proc/sys/vm/drop_caches)
2 - start a kernel compile
3 - start building a 4GB or larger ISO image.

Then repeat with O_DIRECT. Use of iostat may give you additional data.

Bill Davidsen <davidsen@tmr.com>
 "Woe unto the statesman who makes war without a reason that will still
be valid when the war is over..." Otto von Bismark

Reply to: