Survey Bias: Interviewer Bias, Its Types, & Preventive Techniques

Welcome to the third and final installment in our Survey Bias series where we talk about all the different sorts of survey biases that can occur during a survey. We’ve covered different stages of conducting a study and have noted that bias can occur at any stage: planning, execution, and inferences.

Three unique types of survey biases have emerged as the main culprits that can affect the validity and generalizability of a survey or research. These are:

• • Sampling Bias
• • Response Bias
• • Interviewer Bias

We’ve covered the two earlier in the series and today we tackle the last one: interviewer bias.

Interviewer bias is also known as researcher bias and (similar to the sampling and response biases) further divides into sub-categories.

Before we go there, let’s take a quick refresher about our broad topic: survey bias.

What is Survey Bias?

Bias, in general terms, refers to unfair support or opposition extended to a person, thing, or goal due to personal reasons and opinions.

In surveys and research, bias is any deviation from the truth. As we’ve already said, “Any act or decision that can potentially influence the survey results and drive them away from the true picture is known as survey bias.”

Important Note:

While experts agree that survey biases cannot be fully eliminated from research (uncontrollable external factors can always play a role), the goal is to minimize unintentional bias as much as possible and exclude intentional bias completely.

Remember, a biased study will always produce inaccurate results that would be invalid and could not be applied to the broader public.

Interviewer Bias

Interviewer bias occurs when survey creators or interviewers consciously or unconsciously try to influence the survey process, resulting in skewed findings.

Based on what causes it, interviewer bias can take many forms. Some major ones are listed here:

• Non-verbal cues

When the interviewer uses nonverbal behaviors — body language, facial expression, and gestures — to either intentionally or unintentionally influence the respondent to produce certain answers.

Example:

The interviewer nods or raises an eyebrow or shifts uncomfortably in the seat when hearing a certain response or asking a certain question. This can lead the respondent to align their answer to fit the interviewer’s apparent expectation.

• Tone and attitude

When the interviewer’s tone or attitude displays their approval or disapproval of a certain question or answer. It happens when the interviewer becomes animated when talking about a certain topic or shows disinterest when hearing something else. Respondents who match the interviewer’s preferences may alter their responses to fit the displayed preferences.

Example:

In an awareness and usage survey, a respondent may choose not to elaborate on their negative experience if the interviewer shows displeasure about the topic. This may result in a study that highlights positive experiences and downplays negative ones.

• Probing or prompting

This is the type of interviewer bias that results from asking leading or suggestive questions that steer towards a certain response. Interviewers may unintentionally provide hints that influence a respondent on what to say.

Example:

During an HR survey, the employee is asked about instances of successful conflict resolution with a difficult boss. The condition of ‘successful’ may tell employees to only focus on experiences that result in a positive outcome.

• Personal bias

When the interviewer allows their personal opinions, beliefs, or prejudices to influence the research process in any way or to interpret the final results in a certain light. It is also often referred to as favoritism or discrimination.

Example:

Instead of simply asking, “What is your stance on same-sex marriage?”, the interviewer leads with, “Do you believe in traditional marriage values?”

These loaded or emotionally charged questions are a huge no-no. They may give you the answers that you’re looking for but they won’t show you the real picture.

• Reporting bias

This type of interviewer bias involves sharing survey results that either partially or completely omit certain findings. This happens when the study wants to downplay a certain reality or when they are unsatisfied with the results.

Example:

If a researcher studying Gen-Z’s social media habits and mental health has a preconceived negative notion about this dynamic, they may emphasize data that highlights this correlation while oversimplifying more nuanced findings.

How To Avoid Interviewer Bias?

Mary K. Salazar, a University of Washington professor has talked of 8 preventive techniques that address correcting interviewer bias. Here is a quick and simplified version of those suggestions:

1. Better training of interviewers: Train your researchers better. Ask them to sit in field interviews, give them the study material, and show them model interview tapes. Your training sessions must focus on teaching about research integrity and how to set personal prejudices aside.
2. Clarify the interviewer’s role and expectations: When the respondents don’t know what the interview is about, they may give confusing answers. So without leading the respondents, introduce your research topic to them and what you, as an interviewer, are attempting to learn.
3. Avoid using emotionally loaded words: All kinds of leading or loaded questions must be avoided at all costs. Always strive to rephrase your questions and remain gender-neutral too.
4. Start big and narrow it down: Start with broader topics and narrow them down to specific questions. It will also help your respondents understand the context and prepare them to answer the most crucial questions with more understanding.
5. Ask open-ended questions: A research interview is the best setting to get as comprehensive a picture as you can from your core audience. So ask open-ended questions and see what they have to say.
6. Pay attention to question timings: Are you giving some questions more attention or time? Because if you do, your respondents may follow suit. It can result in a skewed study where one area of the topic has more nuanced data than the other. So, try to divide your interview time equally among questions.
7. Be careful of the shift of topics: If you’re shifting too quickly between topics, the respondents might think these are unimportant questions or you’re in a hurry. In both cases, you’ll get surface answers with no complex thoughts.
8. Summarize with care: Try not to add your personal commentary to the results when summarizing. Your personal opinions may make you see patterns when there are none. Strive to remain objective and follow where the data takes you.

Conclusion

Biases can creep into the most well-intentioned surveys, so our job as researchers is to try and minimize these occurrences.

We hope our Survey Bias series will help you stay on the right track when designing your business surveys and research studies. Tell us what you think of the series, and any topic you’d want us to do next!