On Thu, Apr 21, 2005 at 12:57:08AM +0100, MJ Ray wrote: > I have written a short page describing some surveys in > and on debian. I also include some pointers to advice on > producing good surveys. I'd welcome comments, corrections > and suggestions, either on- or off-list. > > The page lives at http://people.debian.org/~mjr/surveys.html I'm rather fond of "How to lie with statistics" as an introduction to what it's all about, not least because it teaches extreme cynicism about anything that smells like a statistic. But possibly even more pertinantly, and in no particular order: - A survey is the worst possible way to answer any question, short of guesswork or religion. It should be employed only when you have no better options, and the result should be treated with extreme scepticism. - A survey will not convince anybody who doesn't already agree with you. It's probably a waste of time. Think again before you bother. - You're supposed to be trying to prove yourself wrong. If you succeed then you've learned something. If you fail despite your best efforts then either you're right, or you suck. Odds are in favour of you sucking. If you try to prove yourself right then you're pretty much guaranteed to succeed, and you've learned nothing. If you don't try to prove anything then you probably won't. - Eliminating bias is impossible; instead, eliminate a need to worry about it by biasing it against whatever answer you're trying to get. That way if you still get the answer, it actually means something. - The most likely answer is normally "I don't know". That's what you should get most of the time. If you don't, you're doing something wrong. - A survey that is sent out to a mailing list in the hope that people will respond to it has a built-in sampling bias that you can drive a truck through. Do not let people self-select for sampling; it won't work. (Hint: one of the things your pilot study should cover is how many of the people you select for sampling actually respond, and why the rest didn't) -- .''`. ** Debian GNU/Linux ** | Andrew Suffield : :' : http://www.debian.org/ | `. `' | `- -><- |
Attachment:
signature.asc
Description: Digital signature