Joerg Schilling wrote:
Bill Davidsen <davidsen@tmr.com> wrote:I played with this a bit, if you are using higher level calls you can replace fopen() with an open() using O_DIRECT, an fdopen() after that, and a setbuffer() to get a big alligned buffer in place. I did that just to see if I could get a gain without doing a bunch of code which might be non-portable in practice. The main benefit was to have less impact on the rest of the system, the i/o in the program didn't run that much faster.I was thinking that a simple wrapper to open() which adds O_DIRECT might be sufficient, but it turned out that this alone is not sufficient: the buffers used by the programs must have a certain alignment. This is not guaranteed without modifying the way how those programs allocate memory.O_DIRECT is no standard and it seems that it is just a reimplementation of the very old DG/UX idea of O_DG_UNBUFFERED.
The idea of doing unbuffered i/o is hardly that young, I believe you will trace it back to MULTICS and GECOS in the late 60's. IIRC that was done when the buffer was alligned and size to allow it, not by explicit user request. Having the user set a flag to request that behaviour came later, and I'm unsure of the portability, since there is a POSIX call to get the sized and aligned buffer allocated. At least for discussion portability is not the issue regarding performance on a single o/s type.
It's hard to imagine how an i/o to a system buffer, followed by a copy to a possibly misaligned user bufffer is going to be faster that i/o to a user buffer which is sized and aligned to allow transfer of an integer number of sectors. I hear what you say, but looking at the physics I have to think the measurement is wrong or the direct i/o is slowed by delays in the o/s.Note that what you get with this feature is less than what you get from adecent VM & buffer subsystem.
For a real compare you need to test Solaris vs. Linux on the same hardware (I did not say or mean identical hardware, but SAME hardware), and test O_DIRECT vs. normal system buffered.
When I added O_DG_UNBUFFERED support into star 11 years ago, it turned out thatDG/UX with O_DG_UNBUFFERED was still _much_ slower than reading from a standard fd on Solaris.
That's also not terribly relevant to the performance in the same o/s and hardware. I'm sure you're correct, but how does it apply to the topic? There is no question that in Linux doing O_DIRECT will improve the performance of programs using system buffers.
-- bill davidsen <davidsen@tmr.com> CTO TMR Associates, Inc Doing interesting things with small computers since 1979