Balanced Question

A balanced question is one that has a question stem that presents the respondent with both (all reasonably plausible) sides of an issue. The issue of “balance” in a survey question also can apply to the response alternatives that are presented to respondents. Balanced questions are generally closed-ended questions, but there is nothing inherently wrong with using open-ended questions in which the question stem is balanced.

For example, the following closed-ended question is unbalanced for several reasons and will lead to invalid (biased) data:

First, the question stem presents only one side of the issue in that it notes only one position taken by some people in the general public. Second, the response alternatives are not balanced (symmetrical), as there are three “agree” choices and only one extreme “disagree” choice. Third, the four response alternatives have no true midpoint; this is a further aspect of the asymmetrical (unbalanced) nature of the response alternatives.

In contrast, a balanced version of this question would be as follows:

This wording is balanced because it poses both sides of the issue. It also has a symmetrical set of response alternatives, with two choices for “agree” and two similarly worded choices for “disagree.” Furthermore, it has a true midpoint, even though that midpoint does not have an explicit response alternative associated with it. If the researchers wanted to add a fifth response option representing the midpoint, they could add, “Neither Agree nor Disagree” in the middle.

In writing survey questions, researchers can further balance them by using randomized variations of the ordering of the wording in the question stem and in the ordering of the response choices. In the second example presented here, one version of the stem could be worded as shown and a second version could have the information reversed, as in, Some people believe that American troops should remain in Iraq until the country is more stable, whereas other people believe that they should be withdrawn from Iraq as soon as possible. The response alternatives could also be randomly assigned to respondents so that some respondents received the four response choices shown in the second example, and the other half of the respondents could be presented with this order of response choices: Strongly Disagree, Somewhat Disagree, Somewhat Agree, or Strongly Agree.