*To*: debian-devel@lists.debian.org*Subject*: Re: Introducing security hardening features for Lenny*From*: Don Armstrong <don@debian.org>*Date*: Wed, 5 Mar 2008 01:29:01 -0800*Message-id*: <20080305092901.GZ2110@volo.donarmstrong.com>*Mail-followup-to*: debian-devel@lists.debian.org*In-reply-to*: <20080305061633.GY27247@outflux.net>*References*: <20080129211624.GA3982@galadriel.inutil.org> <20080129213714.GH30093@artemis.madism.org> <slrnfpvaak.5kd.jmm@inutil.org> <20080129224532.GB5769@artemis.madism.org> <20080130001619.GH16366@outflux.net> <slrnfq1l1f.3bh.jmm@inutil.org> <20080305061633.GY27247@outflux.net>

On Tue, 04 Mar 2008, Kees Cook wrote: > mplayer doesn't compile with PIE due to the various ASM routines. (I've > noted this failure mode in the wiki[2] now.) However, with everything > else enabled (including FORTIFY_SOURCE), there was no measurable > difference (it was below the percentage difference between runs): > > runtime in seconds > Mplayer Normal Hardened > 1 10.87 10.807 > 2 10.873 10.824 > 3 10.854 10.963 > 4 10.809 10.84 > 5 10.877 10.838 Just for future reference, it'd probably be better to run more than 5 tests of each population in the future, as 5 tests means you'll only detect very large differences in performance at any reasonable level of signifigance. FE: > t.test(x=c(10.87,10.873,10.854,10.809,10.877),y=c(10.807,10.824,10.963,10.84,10.838)) Welch Two Sample t-test data: c(10.87, 10.873, 10.854, 10.809, 10.877) and c(10.807, 10.824, 10.963, 10.84, 10.838) t = 0.0722, df = 5.561, p-value = 0.945 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -0.07382831 0.07822831 sample estimates: mean of x mean of y 10.8566 10.8544 > This one showed a possible difference: > > nexuiz Normal Hardened > 1 66.68 68.113 > 2 66.802 66.93 > 3 66.758 67.03 > 4 66.728 67.051 > 5 66.859 67.037 While there may be a possible difference here, it's not significant, given the sample size: > t.test(x=c(66.68,66.802,66.758,66.728,66.859),y=c(68.113,66.93,67.03,67.051,67.037)) Welch Two Sample t-test data: c(66.68, 66.802, 66.758, 66.728, 66.859) and c(68.113, 66.93, 67.03, 67.051, 67.037) t = -2.0899, df = 4.154, p-value = 0.1023 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -1.0779888 0.1443888 sample estimates: mean of x mean of y 66.7654 67.2322 But useful data nevertheless.[1] Don Armstrong 1: I won't even begin to discuss how many times I see benchmarks without SEM or sd reported. -- I'd sign up in a hot second for any cellular company whose motto was: "We're less horrible than a root canal with a cold chisel." -- Cory Doctorow http://www.donarmstrong.com http://rzlab.ucr.edu

**Follow-Ups**:**Re: Introducing security hardening features for Lenny***From:*Kees Cook <kees@outflux.net>

**References**:**Re: Introducing security hardening features for Lenny***From:*Kees Cook <kees@outflux.net>

- Prev by Date:
**Re: Introducing security hardening features for Lenny** - Next by Date:
**Bug#469456: ITP: elisa-plugins-good -- Elisa plugins from the "good" set** - Previous by thread:
**Re: Introducing security hardening features for Lenny** - Next by thread:
**Re: Introducing security hardening features for Lenny** - Index(es):