[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Moving /tmp to tmpfs makes it useful

2012/5/25 Thorsten Glaser wrote:

Ok, we have guesses, let's do some TESTS...

>> Can you name some popular real-world program that write
>> only small files and become noticeably faster when /tmp is on tmpfs?

> Most shell scripts using << (here documents). mc, out of all things,
> as long as the temporary files (e.g. of a tarball) fit inside it.
> And countless others. The majority, in any case. What you want is
> to optimise for a corner case.

Do they become faster? My tests show they don't (see below).

I'm asking for *popular* apps, that create files in /tmp, *never put
large files* there, and become *noticeably* faster with /tmp on tmpfs?
And remember that there're popular apps that break because of that,
so to make sure it's still a good thing all 3 conditions should be met.
We're talking about defaults, not about corner cases after all...

> I say that every tool that doesn’t honour when the user sets
> TMPDIR=/var/tmp is broken (but that’s nothing new, as it has always
> been like that), and that the user should set that in some corner
> cases like yours.

Browsers, filemanagers, flash video, cd burners... Do you basically
suggest every debian user to think whether it's a corner case or not
and manually select TMPDIR *every time* they start browser? ;)

> You reduce the number of disc writes quite a bit, as all current
> filesystems use journalling and write more additional metadata than
> paging would ever use.

I believe you're not talking about some small short-lived temporary files,
because they never actually get to disk, kernel filesystem cache is smart
enough for that (unless you've changed dirty_writeback_centisecs and
dirty_expire_centisecs to some low values).

So you expect to reduce number of disk writes for *large* files
"quite a bit". I'm curious how much is that? I wanted to test:
  # time cp * /mnt/temp/
  # mount tmpfs /mnt/temp -t tmpfs -o size=10G
  # time cp * /mnt/temp/
watching `iostat -k 1` but failed. My 2GB RAM was filled pretty quickly
and the system almost died. Anybody feels the power to compare time and
number of kB_wrtn by swap and extfs for the file that's at least twice
as large as physical RAM?

> And I’m suggesting that file managers that extract archives apply
> some heuristics (size of the archive vs. size and current usage of
> /tmp) to determine whether they should use /tmp or /var/tmp.

So you suggest every file/archive manager developers to change their code
(with a not so trivial change) and ignore FHS because debian decided to
change some variable from "no" to "yes" by default? And what's this all
for? What's the benefit?

It does not make them faster:
  # mkdir /mnt/temp
  # mount tmpfs /mnt/temp -t tmpfs -o size=10G
  # cd /mnt/temp
  # time tar xf ~/linux-3.4.tar.bz2
  real    0m27.100s
  user    0m24.327s
  sys     0m2.100s
  # cd .. ; umount temp ; cd temp
  # time tar xf ~/linux-3.4.tar.bz2
  real    0m28.072s
  user    0m22.103s
  sys     0m4.767s
It varies from 26 to 31 seconds between runs, but the result is obvious —
no noticeable difference. User won't notice the speed, but will notice
problems when system becomes heavily swapping or when some apps
break because of not enough space in /tmp.

> ... apply some heuristics ...

Just curious, what heuristics? I've attached a small archive to this
email, can you estimate its unpacked size without unpacking it? ;)

>>I really can't think of any popular application that write a lot of
>>small-only files and can benefit from /tmp being on tmpfs.

> $GUI_BROWSER_DU_JOUR with XTaran’s unburden-home-dir, for example.

`unburden-home-dir` is a bad example:
(a) It's a real corner case (it's the first time I've heard about it)
(b) It can store cache in any directory, not just /tmp
(c) Those files may be really large, so it may also break or make your
system unstable because of tmpfs. And, finally,
(d) it still does not benefit from /tmp being on tmpfs. It's not making
anything noticeably faster because of tmpfs, is it?

PS: there's a major difference between "most programs will still work"
and "most programs will become faster".

You're telling that in many cases apps will still work somehow with /tmp
being on tmpfs. And YOU'RE RIGHT, they will. And if they won't there're
workarouns like TMPDIR. That's true.

What I'm saying is that most apps will work *better* with /tmp being on
real disk, system will use less RAM and be more stable, user won't have
to use any workarounds and *won't lose anything*.

Thanks for reading this HUGE message.


Attachment: archive.bz2.gz
Description: GNU Zip compressed data

Reply to: