Re: /tmp as tmpfs and consequence for imaging software
Ben Hutchings <ben@decadent.org.uk> writes:
> On Sun, 2011-11-13 at 04:04 +0100, Carlos Alberto Lopez Perez wrote:
>> On 12/11/11 23:25, Josselin Mouette wrote:
>> > Le samedi 12 novembre 2011 à 23:12 +0100, Samuel Thibault a écrit :
>> >> Adam Borowski, le Sat 12 Nov 2011 23:08:08 +0100, a écrit :
>> >>> You need to increase the swap size by the amount you'd use for /tmp.
>> >>
>> >> Well, the idea of such case is precisely to *not* use swap, but real
>> >> disks. Such software already know how to manage its memory and
>> >> disk-backed memory (thusly stored in /tmp)
>> >
>> > Practically speaking, the only significant difference is that files are
>> > not forced to disk as early. Otherwise, if you have a large enough swap,
>> > pages of a file on a tmpfs that are not used enough will be swapped. And
>> > pages of a file on a regular filesystem that are used enough will be
>> > kept in the buffer cache.
>> >
>> There are another differences:
>>
>>
>> When the system is swapping heavily, if you are not using a preempt
>> kernel the hole system will become so unresponsive while the swapping
>> process is taking place that even your mouse pointer will stop moving.
>> And Debian kernel is no preempt.
>
> Linux does not busy-wait for disk I/O and will keep switching between
> tasks regardless of whether preemption is enabled. Further, since
> version 2.6.31-1~experimental.1, all official kernel packages have
> PREEMPT_VOLUNTARY enabled.
>
> The problem you are describing is that physical memory becomes almost
> full with the working set, dirty pages and write buffers and it is not
> possible to write data to disk fast enough to keep up with tasks that
> generate it. As this happens, the cache may have to shrink and reads
> will then miss the cache more often. Further, the kernel must block
> tasks that write to disk rather than allocating more write buffers for
> asynchronous writeback. Suddenly many of your tasks become blocked on
> the disk I/O queue. (Note that Linux 3.2 should improve I/O scheduling
> in this case so that the slowdown isn't quite so bad.)
The problem there is in what linux decides to swap out.
With a filesystem it will write the dirty buffers to disk in the
background and then drop the clean pages from the cache quite
consistently. This leaves the code involved with moving the mouse
pointer alone and functioning smoothly.
On the other hand with tmpfs you get a different code path that swaps
out seldome used pages. Somehow that tends to consider pages needed to
move your mouse pointer as seldome used and swap them out (just drop
them since they are mmaped from the binary). Then when you move the
mouse it needs to swap in the missing pages and THAT blocks.
Don't ask me how or why but that is what I do see hapening. Something is
quite screwed up with the metric what to swap in that regard. Swapping
out tmpfs should have a verry low score (swap out first) imho. Much
lower than swapping out code pages that are used every second.
>> So, if having /tmp with tmpfs will make your system to swap more often
>> (huge files on /tmp) then any performance gain by tmpfs will be buried
>> by this swapping hell.
>
> But a very similar problem can occur with I/O to a conventional
> filesystem. And it's worse in some ways - the kernel has to write
> filesystem blocks back in the right order to keep the filesystem
> consistent, which is not a concern for tmpfs.
>
>> Also, if you are near to run out of virtual memory (RAM+SWAP) for
>> whatever reason: few ram, many apps open... an application can trigger
>> the OOMKiller by simply writing data into /tmp if you are using tmpfs.
>
> In theory, yes. However the size of tmpfs is limited.
Assuming you have increased your SWAP by the size of the tmpfs to
compensate for /tmp now using RAM+SWAP you can only ever get that effect
in cases where the OOMKiller would have already been triggered with /tmp
on disk.
>> > OTOH, for a wide range of applications that do a lot of small writes,
>> > using tmpfs is a huge gain.
>> >
>>
>> Linux already has a disk cache buffer that works very nice for this case
>> of doing lot of small read/writes into a file. Also, when its time to
>> flush the data to disk, or read data no previously cached, the kernel IO
>> scheduler will take care of optimizing it in order to reduce the disk
>> spinning and improve the performance.
>
> That doesn't really help that much once the system is short of memory,
> though.
Plus the real gain is creating small temporary files (like gcc when not
using -pipe). Or any other operation involving a flush/fua for
filesystem consistency including fsync() calls.
>> So... while is true that tmpfs is faster than using the disk for /tmp,
>> it isn't such big deal.
>>
>>
>> And IMHO I don't think that this performance gain outweighs so clearly
>> the problems exposed that justify making tmpfs on /tmp the default on
>> Debian.
>
> Ben.
I think the problems you describe are quite uncommon. Yes, there are use
cases where tmpfs for /tmp isn't the best solution but I think most
people do not place 1.2GB files in their /tmp and benefit greatly from
tmpfs.
MfG
Goswin
Reply to: