[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Surveys in debian

On Thu, Apr 21, 2005 at 12:57:08AM +0100, MJ Ray wrote:
> I have written a short page describing some surveys in
> and on debian. I also include some pointers to advice on
> producing good surveys. I'd welcome comments, corrections
> and suggestions, either on- or off-list.
> The page lives at http://people.debian.org/~mjr/surveys.html

I'm rather fond of "How to lie with statistics" as an introduction to
what it's all about, not least because it teaches extreme cynicism
about anything that smells like a statistic.

But possibly even more pertinantly, and in no particular order:

 - A survey is the worst possible way to answer any question, short of
   guesswork or religion. It should be employed only when you have no
   better options, and the result should be treated with extreme

 - A survey will not convince anybody who doesn't already agree with
   you. It's probably a waste of time. Think again before you bother.

 - You're supposed to be trying to prove yourself wrong. If you
   succeed then you've learned something. If you fail despite your
   best efforts then either you're right, or you suck. Odds are in
   favour of you sucking. If you try to prove yourself right then
   you're pretty much guaranteed to succeed, and you've learned
   nothing. If you don't try to prove anything then you probably

 - Eliminating bias is impossible; instead, eliminate a need to worry
   about it by biasing it against whatever answer you're trying to
   get. That way if you still get the answer, it actually means

 - The most likely answer is normally "I don't know". That's what you
   should get most of the time. If you don't, you're doing something

 - A survey that is sent out to a mailing list in the hope that people
   will respond to it has a built-in sampling bias that you can drive
   a truck through. Do not let people self-select for sampling; it
   won't work. (Hint: one of the things your pilot study should cover
   is how many of the people you select for sampling actually respond,
   and why the rest didn't)

  .''`.  ** Debian GNU/Linux ** | Andrew Suffield
 : :' :  http://www.debian.org/ |
 `. `'                          |
   `-             -><-          |

Attachment: signature.asc
Description: Digital signature

Reply to: