Some tips on questionnaire design

In market research on April 11, 2012 by sdobney

The growth of DIY market research – that is research designs and surveys conducted by people whose normal day job isn’t market research using sites like Zoomerang or SurveyMonkey (or even now via Google) extends the reach of research and means more customer feedback. However, it’s common to see a number of problems with questionnaires designed by inexperienced questionnaire designers, so a few tips might be useful (we also have our Questionnaire Wizard as a starting point for questionnaire design with typical questions for common surveys).

The main thing about questionnaire design is that you have to think of it from the view of the person filling in the survey. If you’re asking about the CodsWaddop AE320 and you want to know if it should have a THP engine or a PQD engine, these terms and reference points may be very meaningful to you in the business, but completely unknown to the people you want to talk to. But it’s very easy to ask a question of the form “What do you like about the THP on the AE320?” and leave a big open-ended box expecting someone to fill it in.

The key is therefore to design a questionnaire with structure trying not to make assumptions about what someone knows and using language that is straightforwards and avoids TLAs (three letter abbrieviations) or other internal jargon unless it is clearly explained. Secondly, it can be very difficult for people to answer a open-ended question with no help. What do you like about…? How could X be improved…? These can be useful, but these types of open questions are best answered after getting someone warmed up and, where possible, giving him or her some clear options – it’s generally quicker and easier to tick boxes.

So the normal way a survey is designed is to run in what is a ‘wine-glass’ type of shape. As someone comes into the start of the survey you ask general broad questions that are simple to answer. As he or she becomes comfortable, the questions become more specific until you get to the key points you want to research. Then you broaden out again to capture things like demographics.

Since most DIY questionnaires are for online use, it’s worth thinking about ‘the fold’. On a typical computer screen, anything above ‘the fold’ is what can be seen without needing to scroll down the page. Things below the fold require scrolling and may be ignored, not seen or skipped. So if you can limit questions and question length so that you stay above the fold – this means no scrolling and makes the survey easier to follow, even if it means putting questions across multiple pages.

Also remember that you’re interested in the respondent’s point of view. If you ask a question like ‘Should we change our packaging to cardboard?’ are you asking from your point of view, or from theirs. So make the question more like ‘For you personally, would you prefer if the packaging was cardboard or plastic?’

Once you get a questionnaire written, then test it on someone, preferrably on someone unrelated to the project or question you are testing. Sit and listen to them as they go through the questionnaire and ask them to point out anything they don’t understand and check to see if they felt the questionnaire covered the topic properly. This is known as piloting and is an essential part of checking the questionnaire.

For some sensitive areas such as pricing or the importance of factors, it can be difficult to get good answers from simple questions. People always say everything is important, or always say that they would only pay a low price. There are techniques and methods to improve the quality of these types of questions and sometimes the order of the questions, or asking questions in particular ways (eg asking a preference or a trade-off) can give more insight.

In some cases people like to ask lots of scale-based questions or agree-disagree questions – for instance you want to ask a rating for each of your competitors. If you want to do this, look very carefully at how the questions appear on screen. Large grid based questions with lots of statements and lots of boxes look very daunting and will drive people away from your survey if there is page after page of this type of question.

Finally, remember some short and sharp questionnaires can be more powerful than long dreary surveys if the analysis is done well. For everyone’s benefit it’s best to keep surveys short and to the point, by making every question sharp and focused. Ask yourself for each question, what sort of answers will you expect to get back and how would you analyse and understand those answers. A mass of unfocused open-ended answers are often less powerful than simple questions about what someone has bought and what from a list of features they really need. For more information and advice on DIY research, see our main website


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: