When designing a survey, it is important to omit as much bias as possible. A biased sample will produce biased results. Attempting to exclude all bias is almost impossible; however, if you understand that it may exist, you can instinctively discount some of the answers. When designing a survey questionnaire, it is important to include elements that make the survey pertinent and relevant to the target audience. The survey questionnaire should be designed to maximize the response rate and minimize errors or bias.
The term bias is used for any trend in the collection, analysis publication, interpretation or review of data that can lead to conclusions. There are two important sources of bias to consider in survey design: non-response bias and reporting bias. Non-response bias will occur when non-respondents to a survey question tend to differ systematically from respondents when data is collected. One example is seen where there is an under-representation of disadvantaged people and over-representation of affluent people in a survey that contributes to non-response bias. For example, a survey using this data might reflect figures which would represent healthy or unhealthy lifestyles, depending on the segment of the population being questioned. Reporting bias happens when people downplay a particular aspect of their lifestyle or behavior, or overstate aspects. For example, people might under-report the amount of alcohol they consume because they drink more than they think, or it might be that they do not want to admit to drinking as much as they do.
Some surveys are developed to intentionally or unintentionally seek out people or events that support the survey taker’s bias, prejudice or program. Some survey takers will commonly commit this fallacy because of laziness or sloppiness. It is very easy to simply take a sample from a source where information happens to be easily available rather than taking the time and effort to generate an adequate sample and draw a justified conclusion.
Because each question is measuring something, it’s important to make sure each question is clear and precise. All survey questions must be written so each respondent will interpret the meaning exactly the same way. One must keep questions short, as long questions can be confusing and stressful for respondents. Also, one must try not to use loaded or leading questions as the response given by the participant will be biased.
An important consideration throughout questionnaire construction is the impact of our own bias. We often have some idea what we are seeking, and the way we build the survey instrument can inadvertently reveal our biases. For example, if we create a new item and distribute it for free to a variety of users, we may decide to send out a follow-up questionnaire to see if the users found the item useful. Great care needs to taken in the design of questions for the survey so that the respondent is not swayed to confirm our desired result.