Bogus Question

A bogus question (also called a fictitious question) is one that asks about something that does not exist. It is included in a survey questionnaire to help the researcher estimate the extent to which respondents are providing ostensibly substantive answers to questions they cannot know anything about, because it does not exist. Bogus questions are a valuable way for researchers to gather information to help understand the nature and size of respondent-related measurement error.

Examples of how a researcher can use a bogus question abound, but they are especially relevant to surveys that measure recognition of, or past experience with, people, places, or things. For example, in pre-election polls at the time of the primaries, candidate name recognition is critical for understanding the intentions of voters. Thus, the name of a fictitious candidate could be added to the list of real candidates the survey is asking about to learn how many respondents answer that they know the fictitious (bogus) candidate. Similarly, when people (e.g. surveys of teenagers) are asked about the use of illegal substances they may have used in the past, it is advisable to add one or more bogus substances to the list of those asked about to be able to estimate the proportion of respondents who may well be answering erroneously to the real survey questions.

Past experience has shown that in some cases as many as 20% of respondents answer affirmatively when asked if they ever have “heard about X before today,” where X is something that does not exist. That is, these respondents do not merely answer that they are “uncertain”—they actually report, “Yes,” they have heard of the entity being asked about. Past research has suggested that respondents with lower educational attainment are most likely to answer affirmatively to bogus questions.

The data from bogus questions, especially if several bogus questions are included in the questionnaire, can be used by researchers to (a) filter out respondents who appear to have answered wholly unreliably, and/or (b) create a scaled variable based on the answers given to the bogus questions and then use this variable as a covariate in other analyses. Researchers need to explicitly determine whether or not the needs of the survey justify the costs of adding bogus questions to a questionnaire. When a new topic is being studied—that is, one that people are not likely to know much about—it is especially prudent to consider the use of bogus questions.