Porky polls: How to dissuade customers from lying in surveys

There aren’t many studies examining how frequently people lie. It is lunacy to assume that respondents will truthfully answer questions about how truthful they are.

Neither is it socially desirable to brand yourself a liar – even in an anonymous questionnaire. Nevertheless, a report published some years ago by ORB International, tried to determine how many of us fib in questionnaires. While researchers found that 80% of people in the UK say they always provide truthful answers to surveys, respondents had a lower opinion of others. Just 9% said they were prepared to trust published polls.

This anecdote is familiar to people in the business of gathering sentiment for policy, market research or customer insight. People generally think that they are truthful, while others are not. And on the latter point, they are correct. When it comes to revealing details about themselves, very rarely do respondents tell the whole truth.

Value-action gap

There is a perceivable disconnect between what people do, and what they say they do. This pertains to any action that is connected to ethics, aspiration or self-worth. In recent years, the most pronounced example of this can be found in research on how much people do to care for the environment.

While everyone would agree that they care deeply about the myriad issues facing the planet, this sentiment probably doesn’t match up with how often individuals recycle, engage in environmental activism, or spend more on green products.

Market research giant GfK refers to this sort of thing as a value-action gap. In a global study - called Who Cares, Who Does - in collaboration with Europanel and Kantar, it found that 65% of consumers try to buy products that are packaged more sustainably. However, only 29% regularly manage to avoid plastic packaging.

Separately, Kantar found that people overestimate how much they engage in socially desirable activities, exaggerating how much they vote, recycle or opt for organic produce at the supermarket. Meanwhile, they downplay how often they smoke, binge on fast food or watch reality TV.

Empathic surveys

There is a way to inspire more truthful answers. Very simply, it is to design questionnaires that inspire less shame from respondents. Someone who enjoys a cigarette only occasionally and in social scenarios might not label themselves a “smoker” if they are asked in a survey. They are more likely to give a straight answer with options such as “I am currently cutting down” or “I only smoke sometimes”. Giving people the opportunity to make excuses for themselves gets more truthful answers, but more detailed ones too.

Another technique is called deflection priming. This is like a qualitative warm up act, when you ask one question to lay the groundwork for a truthful answer to the next. For instance, many are tempted in surveys to exaggerate the level of education they had, saying that they attended university when they did not, according to Jon Puleston, a researcher at Kantar. The solution, he writes, is to create a soft intro to the hard question. So instead of asking “did you attend university?” in the first instance, first you ask, “did you enjoy school?”. This helps people give a truthful answer, because it explains – partially – why they did not attend higher education.

Social listening

There are of course a number of other options for brands and marketers wanting to know more about their customers. Interviews and focus groups can provide valuable feedback and data, but don’t hold up well in B2B environments, and could result in the same lies as surveys do.

Web analytics and A/B testing can give insight into how people behave online. While this kind of testing does give clear data about how a customer responds to a site, it doesn’t provide any qualitative data about what is driving that behaviour

Social listening is also an increasingly important gauge of customer satisfaction and sentiment, providing an unfiltered view of a brand. The downside of social listening is that those with the most to say will dominate the conversation, whereas a customer who is generally happy with their experience of a brand, may not be compelled to take to social media to express that opinion.

Each of these methods has value for a brand or a business but to create a full picture of the customer and what drives them, it may be important to use one or more in conjunction with surveys.

As it becomes more crucial for brands to cut to the truth of why people spend, those who design studies will need to get better at spotting the lies in statistics.

At Bulbshare, our researchers are well-versed in creating surveys that encourage truth - and we use in-built functionality that encourages instinctive responses and more accurate data.