Andy Polyakov wrote:
as you still have to pull a lot of data from disk, you still put quite a pressure on VM subsystem, so direct I/O can still help,But how to talk afio, star or mkisofs into that ?I was thinking that a simple wrapper to open() which adds O_DIRECT might be sufficient, but it turned out that this alone is not sufficient: the buffers used by the programs must have a certain alignment. This is not guaranteed without modifying the way how those programs allocate memory.Do note that I assert that it *can* help. In particular if it eliminates need for substantial amount of redundant page faults, most notably other applications' working sets from being offloaded to swap partition because of pressure on the block buffer and repeatedly requested back during recording [with modern bloated GUI it's more than common scenario], or some frequently used portions of block buffer from being purged. "Frequently used" means "multiple times during recording, but not frequent enough for raging data stream not to repeatedly kick it out." As was pointed out by Bill if you want *guaranteed* success with direct I/O, you have to complement your code with some asynchronism, e.g. you at least want to partially undertake the block buffer role and perform pre-fetch [and pre-open per Bill's suggestion]. Once again, growisofs attempts to take advantage of direct I/O for "image" recordings. "Attempts" means that it passes the O_DIRECT flag to the kernel and aligns buffers, but I can't give any guarantees that some particular kernel actually respects the flag. Test it, provide feedback...
My belief is that if O_DIRECT is in the kernel headers it works. I would love to have time to test timing of O_DIRECT on (a) disk, (b) partition, (c) file on disk, and alignment on minimal vs. page size boundaries. Doing O_DIRECT writes of anything avoids buffer pool collisions. To test either hack mkisofs to write with O_DIRECT or pipe into Dwriter or similar and see the difference in create time of a DVD image. I may actually do the hack and enable it when -o is used (and on Linux). If I do I'll put up the patch and post a link here. Then Joerg can reject it because it makes mkisofs run faster on Linux.
Hum, have to put Dwriter and a doc file on my web site with the other tools :-(
On a side note it should also be noted that in particular Linux requirements for alignment are not really reasonable. Minimal alignment required by DMA controller, cache line size to be specific, should suffice, but they've chosen to insist on block or sector alignment. A.
-- bill davidsen <davidsen@tmr.com> CTO TMR Associates, Inc Doing interesting things with small computers since 1979