Re: Introducing security hardening features for Lenny
On Wed, 05 Mar 2008, Kees Cook wrote:
> On Wed, Mar 05, 2008 at 01:29:01AM -0800, Don Armstrong wrote:
> > Just for future reference, it'd probably be better to run more than 5
> > tests of each population in the future
>
> Getting larger data sets will be rather time-consuming -- especially
> for nexuiz which I didn't figure out how to automate.
Yeah; the converse is that since you're really interested in major
differences, 5 samples should be able to detect those.[1]
> > t = 0.0722, df = 5.561, p-value = 0.945
> > 95 percent confidence interval:
> > -0.07382831 0.07822831
> > sample estimates:
> > mean of x mean of y
> > 10.8566 10.8544
>
> Which of these outputs should be paid attention to?
The p value and the convidence interval are the two things that are
really useful. The first tells you that assuming the true difference
between the means is zero, you would expect to see a difference like
this or larger 94.5% of the time. The second tells you that with 95%
confidence the true difference between the means is between -0.07 and
0.07.
Don Armstrong
1: There are sample size calculators where given power (or 1-beta),
alpha, and the difference you wish to detect will give you the number
of samples required.
--
When bad men combine, the good must associate; else they will fall one
by one, an unpitied sacrifice in a contemptible struggle.
-- Edmund Burke "Thoughts on the Cause of Present Discoontents"
http://www.donarmstrong.com http://rzlab.ucr.edu
Reply to: