On 2003-08-28 19:40:08 +0100 Andreas Barth <aba@not.so.argh.org> wrote:
* Branden Robinson (branden@debian.org) [030828 20:35]:What's your threshold of statistical significance? I'd like to know for the purposes of commentary on my "final" survey reponse summary, which Ican produce about 12 hours from now if I'm still awake.5% (means: the conclusion is to at least 95% true, and to less than 5% false) to make a "probably", 1% to settle it.
Back to school with you! If you are speaking of classical hypothesis tests, a 5% significance level means that there is at most a 5% probability of this test result occurring by pure chance. Speaking of the conclusion being "95% true" normally indicates that you've failed to test correctly, or don't know what you're doing.
I'm not 100% sure, as I can't remember exactly what was being used, but generating a p-value from a good ol' chi-squared test and allowing people to pick their own level is probably safest. I think the table used earlier was 3x4, so 6 degrees of freedom, although we may have some classes that hold too few values. I don't know if either salstat or vista are currently in Debian, but R is (if a little more awkward to get started with) and should be able to do the test. Failing that, it's not too hard by hand. Give the raw data too and anyone sufficiently interested can do their own analysis.
Disclaimer: I hold one degree in this stuff and studied for another, but don't use it all that often just now, so get yourself a copy of "Statistics Without Tears" or "Statistics Explained" and check if you want to be bulletproof.
--
MJR/slef My Opinion Only and possibly not of any group I know.
http://mjr.towers.org.uk/ jabber://slef@jabber.at
Creative copyleft computing services via http://www.ttllp.co.uk/