On Wed, Mar 05, 2008 at 06:16:33AM +0000, Kees Cook wrote: > Hi, > > I finally got some time to run some benchmarks. I checked the results[1] > into the "hardening" svn tree, in case other people want to contribute > more stuff. > > On Wed, Jan 30, 2008 at 08:46:55PM +0100, Moritz Muehlenhoff wrote: > > Video: > > mplayer with the -benchmark option in conjunction with -nosound and -vo. > > mplayer doesn't compile with PIE due to the various ASM routines. (I've > noted this failure mode in the wiki[2] now.) However, with everything > else enabled (including FORTIFY_SOURCE), there was no measurable > difference (it was below the percentage difference between runs): > > runtime in seconds > Mplayer Normal Hardened > 1 10.87 10.807 > 2 10.873 10.824 > 3 10.854 10.963 > 4 10.809 10.84 > 5 10.877 10.838 > avg 10.8566 10.8544 diff: -0.02% > error 0.19% 1.00% > > > HTML rendering: > > Mike Hommey once blogged about benchmarking the ACID test: > > http://web.glandium.org/blog/?cat=17 > > I followed that to: http://celtickane.com/webdesign/jsspeed2007.php > The differences between runs were too high for me to use, so I skipped > this for now. > > > Nexuiz: > > | To run the benchmark: start Nexuiz & open the console (`) issuing: > > | timedemo demos/demo1.dem The results will be stored in: > > | ~/.nexuiz/data/benchmark.log > > This one showed a possible difference: > > nexuiz Normal Hardened > 1 66.68 68.113 > 2 66.802 66.93 > 3 66.758 67.03 > 4 66.728 67.051 > 5 66.859 67.037 > avg 66.7654 67.2322 diff: 0.70% > error 0.14% 1.31% > > So, for nexuiz, with all hardening enabled in i386, there was a > less-than-1-percent reduction in speed. Though the error margin for the > hardened runs were still larger than the measured slow-down. > > > Not sure about XML benchmarks. > > I did parse/render tests with inkscape on i386. Some of that is XML, but > I figured it was heavy CPU, which might be worth something. Note that > inkscape already compiles with all hardening options (excepting PIE), > so the "hardened" time differences are entirely due to PIE. This one > turned out similar to nexuiz, but with less error. Again, less than 1 > percent slow-down was seen. > > inkscape Normal Hardened > 1 48.163 48.503 > 2 48.227 48.535 > 3 48.267 48.647 > 4 48.335 48.431 > 5 48.199 48.587 > avg 48.2382 48.5406 diff: 0.63% > error 0.20% 0.22% > > I also ran inkscape and nexuiz tests on x86_64, and there was no > measurable difference. I'm unclear if this was due to the extra > registers, or just that that CPU was much faster and the difference > vanished into the noise. Thank you very much for those. Though what did you built using -fPIE FORTIFY_SOURCES and so on ? only the tested applications ? or their build-deps as well ? Because I don't expect mplayer to be slowed a lot if you don't rebuild its ogg/mp3/mpg/... as well :) Same goes for inkscape. -- ·O· Pierre Habouzit ··O madcoder@debian.org OOO http://www.madism.org
Attachment:
pgp_Wl27LBcId.pgp
Description: PGP signature