Rating

In constructing a survey questionnaire there are a number of response formats that can be used in asking respondents about their views on various topics. One of the most commonly used formats in survey questionnaires is the rating, or rating scale.

As the name implies, ratings are regularly used in surveys to evaluate an object along some dimension, such as performance or satisfaction. In a typical rating question, respondents are asked to make judgments along a scale varying between two extremes, such as from “very good” to “very poor,” or “extremely positive” to “extremely negative,” and the like. The following illustrates a common rating scale question: How would you rate the job (PERSON'S NAME) is doing as governor… Would you say she is doing an excellent job, a good job, a fair job, a poor job, or a very poor job? As another example, the item, Some people don't pay much attention to political campaigns. How about you… would you say that you have been extremely interested, very interested, not too interested, or not at all interested in the political campaigns so far this year? is a rating scale in which respondents are asked to assess their interest in campaigns among choices ranging from “extremely” to “not at all.” A third example is the item, Now I'd like to ask you to rate the city as a place to live and work on a scale of 0 to 10, with 10 being the highest rating you could give and 0 being the lowest.

A rating question provides the topic to be evaluated, the dimension along which it is to be evaluated, and the scale of the evaluation. As the previously mentioned examples illustrate, rating questions vary widely in terms of the dimensions on which objects are evaluated (job performance, interest in political campaigns, place to live and work), the scales used (“excellent” to “very poor,” “extremely interested” to “not at all interested,” 0 to 10), and the number of response options (5, 4, and 11, respectively).

The various components of a rating question affect the results produced by such items. For example, in evaluating presidential performance, different—although very likely similar—results would be obtained if the item asked the respondents to assess such performance as excellent, good, fair, poor, or very poor than if it asked them whether they strongly approved, somewhat approved, neither approved nor disapproved, somewhat disapproved, or strongly disapproved of the president's performance. Even a seemingly minor change in the options provided—for example, changing “excellent” to “very good”—can yield different results and sometimes these differences are not negligible.

Another important factor in rating questions is the number of response options. It is generally considered that five options are the most respondents can understand without some type of visual aid, although some surveys use more, such as the “rate on a scale of 0 to 10” example described previously. Related to this is whether respondents should be provided with an odd or even number of response categories. Rating questions with an odd number of categories provide respondents with a true “middle” option, whereas items with an even number of categories force respondents who feel “in the middle” to lean in one direction or the other. The order in which the options are presented, from “most positive” to “least positive” [Page 692]or vice versa, can also make a difference in the results obtained. For any of these considerations, there are no technically right or wrong ways in which to ask a rating question. Each of the decisions made in developing an item should be guided by the research question of interest.

...