Survey bias is an often-neglected part of the survey creation process. If left unchecked, it can make your results irrelevant and insignificant. So, what exactly is survey bias, what are the main causes and how can you eliminate it?
WHAT IS SURVEY BIAS?
Survey bias encompasses any error due to the design of a survey. It is not just about the way you phrase your questions but also the types of questions you use and even the way you order your questions. You might not realise it but the way you create your survey can have a significant impact on the results so below we look at some of the main sources of survey bias you can address.
The way your questions are worded is one of the most common ways to insert bias in to your survey. You could phrase your questions in such a way that they elicit either a positive or negative reaction from the respondent. You want to aim to keep the wording as neutral as possible. This can often be difficult if you are unaware of your own bias or perhaps have a lack of knowledge on the survey topic. This could lead to you confusing respondents or you may not consider things such as public opinion or respondent sensitivity.
To avoid this, try to make your questions brief with a single focus. Keep them jargon-free and don’t use any emotive language. If you are not an expert in the area then ask an expert to look over the survey for you and try to research the topic as much as possible when designing your questions. It is the survey writers job to avoid questions that lead or confuse the respondent so you must remain impartial.
At Wizu we offer an extensive range of question types from open text questions, multiple choice, sliding scale, star ratings and more. The type of question you choose could gain a very different response. Each question type has its own strengths and weaknesses so it is important you take these into account when making your selection. You also need to look at the range of answer options you are giving the respondent. Are you limiting them and perhaps not getting their genuine response. These are all considerations to look at when deciding which question type is right for each area you want to discuss.
How you structure your survey and the order that you place your questions can impact the responses you get. Even any introduction messages can have a significant impact. You don’t want to upset or offend your respondent on the first question as this could affect your completion rates or could impact the responses to future questions. If you do need to ask sensitive questions, then try asking these later in the survey. You want them to be interested at the start of your survey so you can gain the information you want.
It is important that your survey looks good and works well but you need to ensure your styling does not impact your responses. A well-designed survey can help avoid respondent fatigue and using your branding can help increase legitimacy. This can include everything from your colour scheme, font, images used and more. You also need to be aware that different colours and images can affect respondents in different ways. Try to avoid styling in a way that suits a particular demographic and ensure all messages and questions are easy to see.
Clean language is a technique that is often used in psychotherapy that has more recently been utilised as a research interview technique. Clean language questions try to minimise content that comes from the questioner’s MAPS. MAPS, or Metaphors, Assumptions, Paradigms or Sensations, can direct the respondent’s attention away from their own representation of an experience.
It is also worth noting that sometimes survey bias is used intentionally. Ultimately you need to decide the goal of your survey and if you do want to provoke a reaction or lead respondents in a certain direction then you can utilise a lot of these issues to achieve the desired effect. It is the unintentional survey bias that you need to watch out for. If you are making big business decisions based on your results, then you want to make sure the responses are accurate and not led by your own bias.
About Martin Powton
Wizu is a chatbot for conversational surveys offering a unique and innovative way to measure and improve customer experience. Wizu allows companies to invite their customers to a conversation with a chatbot that will be personalised to them. The bot uses the latest in Artificial Intelligence and text analytics technology to understand customer emotion offering a deeper insight than traditional surveys and offering a more engaging and interactive survey experience.