Questionnaire-Related Error

Comprehension

Taking steps to ensure question comprehension by respondents is often the first stage in developing a questionnaire that minimizes error due to misunderstanding or misinterpreting questions. In general, the more complex or ambiguous the task, the more error there will be. Questions may contain words unfamiliar to the respondent, may have complex syntax, or may assume a higher level of respondent cognitive ability than is warranted. Sometimes researchers use specialized terminology rather than colloquial language. Respondents may also not pay close attention and may miss part of the question.

One difficulty is that many words in natural language are ambiguous, and researchers often must use vague quantifiers that have an imprecise range of application. Some examples include categories of words denoting frequency (e.g. never, sometimes, pretty often, very often), probability expressions (e.g. very unlikely, somewhat unlikely, likely), amounts (e.g. none, some, many, all), and strength (e.g. extremely dissatisfied, very dissatisfied, somewhat dissatisfied). There is evidence that respondents have varied associations with words that denote frequency, regularity, or size. For example, How often … ? results in variability based on how respondents interpret the word often. It is advisable to use more specific quantifiers, such as On how many days … ? or How many times in the last week/month/year … ?

Simple words or categories that appear to have common understandings, such as household or sibling, can be interpreted differently. When possible, such terms should be defined so as to minimize variability in how respondents interpret them. For example, household can include persons who use the address as their permanent address but who are not physically living there, such as college students who live in dormitories, individuals on military duty, or individuals who may reside at that address for only part of the year. Sibling may include biological siblings, half siblings, and adopted or stepsiblings.

...