Double guessing the purpose of a survey

In market research on April 11, 2012 by sdobney Tagged: , ,

A very common occurance in research surveys is to have respondents say what they think we should have asked in the survey. This isn’t a bad thing since in a questionnaire we’re really trying to gather the opinions of the consumer or customer and if he or she has a particular view of the world that we are not capturing, we’re not necessarily getting their full opinion. For this reason we always ask a final thoughts question at the end of the survey. But from a respondent point of view, you don’t always see the real purpose of the questions being asked.

For instance in a recent survey, someone was concerned about the packaging of the product we were researching, and wanted to know why we hadn’t included questions about packaging or package design. Unfortunately the survey in question was really to test brand awareness so for us as research designers, in this case packaging wasn’t what we wanted to investigate. We wanted to know about the familiarity with a set of brands and have something we could compare to other studies on the same subject.

But this tendency to try to understand the questionnaire in terms of ‘what do they want to find out and why?’ from the respondent’s perspective means that the research designer has to realise that even in the simplest surveys, respondents are trying to tell you what they think you want to, or ought to, hear. And if the questionnaire doesn’t allow these concerns and issues to be raised, the respondent can feel dissatisfied with the whole process. The classic example of this is actually in customer satisfaction research, for example for a bank where a customer might answer lots of questions about the service they received at the counter, but feel cheated if they aren’t also asked questions about how they see the general behaviour of the bank or the bank’s charging policies.

The need to think of how the respondent is ‘reading’ the subtext of the questionnaire is important. If he or she knows that you are interested in one particular brand it may change how they respond or react to that brand. There is a general tendency to try to be helpful – for instance to say you know more about something because someone is asking you. Researchers try to control for this by masking which brand or product they are enquiring about early on in the questionnaire so as to gather unbiased reports, but even here respondents will try to guess who the survey is for based on the types of questions being asked.

A second issue for working including prices is that respondents can try to negotiate with the questionnaire.  That is they might be asked how much they would pay and give a lower than true figure as a negotiating starting point. Often it’s useful to test higher price points than the maximum someone has given because they will say they will pay higher prices later.

In some markets, this tendency to read a subtext can make it difficult to explore difficult or sensitive decisions. For instance in a market where sales are made on an indirect basis via a sales channel, discussions of moving to a direct sales model are very sensitive and often the cause of much channel conflict. The business will want to explore different sales and marketing options, but the danger is that the customer reads the discussion as the decision and is left worried or confused. The care with which questions are framed and included and emerge in the survey means a great deal of thought needs to go into not just what the commissioning company wants to know, but also how to frame the questionning so as to get a full view from the respondent without leaving impressions that might not be true. For more information and advice on survey research, see our main website


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: