I was recently asked what is the best tool for creating online surveys. The question struck me, because whether it's SurveyMonkey, Wufoo, Zoho, or some custom thing – the tool doesn't really matter if you mess up the questions and execution of a survey.
As these tools have dropped to $0, the diligence behind the survey process seems to have followed. Here are five tips for anyone creating an online survey:
1. Beware sampling bias. Calculate the sample size you need based on population to make sure you have significant results within an appropriate confidence interval.
- Calculate your sample size.
- Learn about survey bias.
- Check out some tatistical significance calculators.
When people were asked whether they would “favor or oppose taking military action in Iraq to end Saddam Hussein’s rule,” 68% said they favored military action while 25% said they opposed military action. However, when asked whether they would “favor or oppose taking military action in Iraq to end Saddam Hussein’s rule even if it meant that U.S. forces might suffer thousands of casualties,” responses were dramatically different; only 43% said they favored military action while 48% said they opposed it. (From a January 2003 Pew Research survey.)
4. A variable incentive works and doesn’t skew the results as long as the prize doesn’t conceptually conflict with the intention of the survey. For example, giving a cash award for a survey of homeless people about how they spend money could be a bad idea.
5. Using “Average” to summarize questions is easy, tempting, and – on average – misleading. For example, if Bill Gates reads this article, I could easily say that the average reader of my blog is a millionaire.
Did you hear about the statistician who had his head in an oven and his feet in a bucket of ice? When asked how he felt, he replied, “On the average, I feel just fine." (From every geeky stat professor out there.)
Learn how to create simple surveys and questionnaires in Salsa
Online survey tips from SurveyMonkey
Full survey design process
Thinking Fast and Slow