Re: /tmp as tmpfs and consequence for imaging software
]] bastien ROUCARIES
Hi,
please don't cc me on mailing list posts. It's rude and against Debian
list policy, even more so when I've set mail-followup-to.
| Le Sunday 13 November 2011 16:24:18, Tollef Fog Heen a écrit :
| > ]] Bastien ROUCARIES
| >
| > Hi,
| >
| > | No it is not true. Science and imaging software are better to use true
| > | disk baked file. For instance, if I want ot invert a big matrix they
| > | are pretty good algorithm that force only some part of the file to be
| > | keep on disk. They known better than kernel when to put somepart on
| > | the data on the slow disk.
| >
| > I doubt the authors of said software is better at writing VMs than the
| > kernel authors are. Even if they are as good, the software isn't in a
| > position to know the needs ot the whole system, just what the particular
| > application needs.
|
| Yes they are particularly for inversion of matrix bigger than memory. And moviemaker software work on file bigger than memory.
| Moreover it is not a problem of VM it is a problem of:
| - information: kernel lack information on the structure of the problem: how can the kernel known that your piece of data is
| positive definite (ftp.numerical.rl.ac.uk/pub/talks/jas.berkeley.28III2007.pdf)
Trying to write a VM in userspace (which the abovementioned PDF
describes) quickly runs into the problem of the kernel swapping out
pages underneath your feet. The easiest way to solve this is to not try
to manage it explicitly, but just allocating lots of memory or mmap-ing
a big file and letting the kernel do the paging in and out.
[...]
| > | Using tmpfs under /tmpfs you break assumptions on the life expendancy
| > | of memory object.
| >
| > How is this broken? It's not like /tmp is magically cleaned more often
| > just because it's a tmpfs.
|
| No but offlining is not made by application request but by kernel. It
| is fine in lot of case but not for all the case.
What do you mean by «offlining»?
| > | And you slow down this kind of software (that work perfecly for 40
| > | years).
| >
| > If the architecture of a piece of software is the same as it was 40
| > years ago, it's not particularly well adapted to today's machines, since
| > it won't know how to take advantage of virtual memory, multiple cores,
| > etc.
|
| Not sure for instance see http://www.tau.ac.il/~stoledo/taucs/ or high
| performance eigen solver have not been updated since 15 years. But
| work well and are efficient.
«September 4, 2003: Version 2.2! Multithreading using Cilk.»
2003 is not 15 years ago. Also, 15 years is not 40 years. ;-)
Also, it talks about «tens of gigabytes» as if that's a huge
system. It's not, it's a pretty small system; you can get machines today
that has 1T of memory in a single chassis and that's before you start
hitting the SSDs. The PDF you referenced talks about doing test runs on
a machine that's about as capable as the three-year old laptop I'm
writing this on, which doesn't seem to be relevant at all to how those
problems ought to be solved.
| Moreover the tmpfs solution has no quota like on disk file and this could lead to dos (see
| http://0pointer.de/blog/projects/plumbers-wishlist-2.html for a solution and they are recent post on lkml).
Sure, that's a valid criticism of tmpfs. It's something that's
relatively easy to fix, though.
| The problem is that you propose band aid and not long term
| solution. Apps known better when kernel theirs need.
I haven't proposed any solution, and if apps need to know whether their
non-permanent working data is in physical memory or in a memory mapped
file on a physical disk, something is broken somewhere. The right fix
is to fix that, not introduce workarounds in other parts of the system.
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
Reply to: