Re: Moving /tmp to tmpfs makes it useless
On Fri, May 25, 2012 at 02:22:24AM +0300, Serge wrote:
> Q: /tmp on tmpfs increases apps performance.
> A: What apps? Real apps don't write files during performance-critical
> operations. Even if they do, they write large files. And large files are
> written faster when they're written on real disk, rather then swapped
> out and slow down the entire system (see the "Who uses /tmp" part).
> The apps that can really benefit from tmpfs are too rare. And we're
> talking about default settings and most common cases.
OK, some benchmarks were requested in this thread in a few places.
Can't find them again since they got lost in the volume of mail. I
resisted doing any so far because I didn't feel that they would be
particularly informative, and also because the results would be fairly
predictable, and additionally because while tmpfs /can/ improve
performance, it's only one of the reasons why we might want to use it.
---- Summary ----
These tests were all performed on current unstable using a core2 quad
core system with ext4 and swap on LVM on a 1 TiB MD RAID1 PV, and
Btrfs internally using RAID1 over 2 1TiB partitions. The system has 8
GiB RAM and 16 GiB swap, and a 4.8 GiB tmpfs on /tmp (using the
experimental initscripts default of 20%VM). For all results, n=20
with error ± SEMean. Stats were done in R. 5 passes were performed
before recording data to ensure consistency.
1) Checkout of several tags in a git repository
tmpfs 11.17s ± 0.05s
ext4 18.51s ± 0.60s
btrfs 15.81s ± 0.29s
2) Building a package (sysvinit)
tmpfs 20.08s ± 0.03s
ext4 21.08s ± 0.03s
btrfs 20.52s ± 0.03s
3) Unpacking of a large zip file
tmpfs 12.45s ± 0.01s
ext4 19.69s ± 0.52s
btrfs 12.79s ± 0.32s
4) Unpacking of large uncompressed tarfile
tmpfs 1.16s ± 0.00s
ext4 17.61s ± 0.41s
btrfs 5.12s ± 0.20s
So the outcome is fairly obvious. I/O bound jobs perform superbly on
tmpfs; in examples 3 and 4, the times are for how long it takes to
unpack 1.2 GiB of data. Without the overhead of uncompression, making
it largely CPU bound, tmpfs is the fastest by far. The package build
is a combination of both I/O and CPU bound jobs, so the differences
aren't so big, while git is more I/O bound, again showing tmpfs to be
faster. So if you're doing a lot of I/O, tmpfs is unquestionably
faster. If you're almost entirely CPU bound, the benefit is marginal,
but still measurable. If you're doing a mixture, you do see some
benefit, but you'll need to profile your specific case to determine
exactly how much.
Now, these are just four simple tests. They deliberately use hot
caches, using several warm up runs to ensure that. No unnecessary
swapping or disc activity should occur. If you have a system which is
swapping, of course the tmpfs performance will drop, just as if you
have a lot of filesystem I/O, the I/O performance will also drop.
These are rather harder to test accurately and consistently, so I
haven't done this at present. If anyone wants to, feel free.
The other caveat is that due to the use of hot caches, and lack of
explicit fsyncs, this will actually artifically inflate the performance
of the disc-based filesystems in the common case, since all of the data
being processed is in the buffer/page cache. It's only measuring
writes. So for most workloads, the performance of tmpfs is most likely
much better than reported here (by a factor of at least two, probably
quite a bit more). These could be added; again, if anyone wants to
do that, please feel free.
Regards,
Roger
---- Methods and results ----
1) checkout of git repository
#!/bin/sh
set -e
for pass in $(seq 1 5); do
sh -c "for tag in v3.0 v3.1 v3.2 v3.3 v3.4; do git checkout \"\$tag\"; done"
done
for pass in $(seq 1 20); do
/usr/bin/time -f '%e' -a -o "$1" \
sh -c "for tag in v3.0 v3.1 v3.2 v3.3 v3.4; do git checkout \"\$tag\"; done"
done
> d
tmpfs ext4 btrfs
1 11.31 22.42 17.15
2 11.30 22.13 15.35
3 11.04 19.63 15.46
4 11.24 19.12 15.51
5 11.50 18.00 14.96
6 11.01 22.55 15.15
7 11.17 16.29 15.11
8 11.04 23.57 17.42
9 11.20 15.21 15.17
10 11.68 18.20 15.86
11 10.74 17.81 15.22
12 11.30 18.25 15.43
13 10.96 21.94 15.43
14 10.80 14.61 20.39
15 11.27 15.34 14.99
16 11.28 18.04 16.46
17 11.65 17.35 15.53
18 10.97 16.46 15.16
19 10.79 16.73 15.51
20 11.22 16.46 14.91
> sapply(d, mean)
tmpfs ext4 btrfs
11.1735 18.5055 15.8085
> sapply(d, semean)
tmpfs ext4 btrfs
0.0583243 0.6044565 0.2855330
2) Building a package (sysvinit)
#!/bin/sh
set -e
for pass in $(seq 1 5); do
sh -c "dpkg-buildpackage -us -uc"
done
for pass in $(seq 1 20); do
/usr/bin/time -f '%e' -a -o "$1" \
sh -c "dpkg-buildpackage -us -uc"
done
> sb
tmpfs ext4 btrfs
1 20.11 21.06 20.56
2 20.13 21.28 20.58
3 20.15 20.90 20.46
4 19.82 21.01 20.52
5 19.94 21.03 20.83
6 20.06 21.09 20.43
7 20.06 21.19 20.48
8 19.94 20.82 20.38
9 20.16 21.00 20.29
10 20.03 21.16 20.48
11 20.10 21.24 20.45
12 19.92 21.09 20.45
13 19.89 21.00 20.47
14 20.04 21.11 20.55
15 20.36 21.42 20.43
16 20.27 21.09 20.53
17 20.28 20.98 20.69
18 20.02 21.05 20.52
19 20.21 21.08 20.67
20 20.12 21.02 20.54
> sapply(sb, mean)
tmpfs ext4 btrfs
20.0805 21.0810 20.5155
> sapply(sb, semean)
tmpfs ext4 btrfs
0.03099639 0.02973479 0.02623251
3) Unpacking of a large zip file
#!/bin/sh
set -e
for pass in $(seq 1 5); do
sh -c "rm -rf lsm; unzip lsm.zip"
done
for pass in $(seq 1 20); do
/usr/bin/time -f '%e' -a -o "$1" \
sh -c "rm -rf lsm; unzip lsm.zip"
done
> unzip
tmpfs ext4 btrfs
1 12.44 16.67 12.68
2 12.44 18.57 12.56
3 12.50 18.13 12.82
4 12.45 17.14 12.82
5 12.43 19.72 12.79
6 12.50 19.43 12.55
7 12.35 17.74 12.76
8 12.46 21.73 12.85
9 12.43 21.01 12.86
10 12.43 19.80 12.78
11 12.59 17.18 12.96
12 12.45 21.86 12.77
13 12.46 21.88 12.84
14 12.39 21.27 12.84
15 12.35 17.61 12.76
16 12.45 22.21 13.21
17 12.50 26.03 12.72
18 12.43 18.60 12.77
19 12.49 19.45 12.84
20 12.47 17.71 12.58
> sapply(unzip, mean)
tmpfs ext4 btrfs
12.4505 19.6870 12.7880
> sapply(unzip, semean)
tmpfs ext4 btrfs
0.01199726 0.52173249 0.03233541
tmpfs 12.45 ± 0.01
ext4 19.69 ± 0.52
btrfs 12.79 ± 0.32
4) Unpacking of large uncompressed tarfile
#!/bin/sh
set -e
for pass in $(seq 1 5); do
sh -c "rm -rf lsm; tar xf lsm.tar"
done
for pass in $(seq 1 20); do
/usr/bin/time -f '%e' -a -o "$1" \
sh -c "rm -rf lsm; tar xf lsm.tar"
done
> tar
tmpfs ext4 btrfs
1 1.16 18.61 4.81
2 1.16 17.87 5.73
3 1.17 17.36 5.05
4 1.15 17.97 5.51
5 1.16 16.97 5.16
6 1.16 18.50 6.83
7 1.15 19.72 4.21
8 1.15 16.31 5.39
9 1.15 19.51 5.23
10 1.15 19.16 4.71
11 1.16 18.04 6.80
12 1.16 17.25 3.42
13 1.15 18.72 4.43
14 1.16 14.17 4.26
15 1.16 16.85 5.30
16 1.15 19.47 4.81
17 1.15 19.39 4.90
18 1.16 16.27 6.77
19 1.15 12.56 4.06
20 1.15 17.56 5.03
> sapply(tar, mean)
tmpfs ext4 btrfs
1.1555 17.6130 5.1205
> sapply(tar, semean)
tmpfs ext4 btrfs
0.001352386 0.405560755 0.202557951
> semean
function(x) {
sd(x) / sqrt(length(x))
}
--
.''`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools
`- GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800
Reply to: