Evaluation of Science Communication

Audience Definition

Who should science communication efforts target? Many have documented the gap in scientific literacy that lies between the general populations of [Page 285]many countries and scientific professionals. Some scholars argue that this gap threatens societal health and needs to be narrowed, at least in part so that sound policy on scientific research can be formulated through democratic means. That stance suggests the need for science communication efforts that address a relatively wide audience. Other researchers have argued that the audience for science communication efforts is always going to be a relatively small minority of a general population due to limits in educational background and specific interest. From that perspective, recommended projects might only be intended to speak to a relatively select group.

Defining who makes up a target audience is nonetheless vital to project evaluation. If a project is intended to address beliefs about genetically modified food among a general population, an appropriate evaluation design will focus on effects that are apparent for that whole group. Alternatively, evaluation designs that permit assessment of interactions, for example, differential project effects according to demographic segmentation or other group characteristics, might be most appropriate if a project is more narrowly focused. For example, if a project is only intended to improve attitude toward science education policy among active voters, then looking for effects among the general public might miss important outcomes.

A different type of science communication evaluation effort involves reporters or policymakers as an intended audience rather than lay people. The academic literature documents a number of such efforts in recent years, many of which stem from concern about the chasm between scientists and journalists. Although important differences between scientists and journalists exist in terms of educational background and scientific understanding, some recent evidence suggests that the chasm is not an insurmountable one. Interviews with epidemiologists and stem cell scientists, for example, have revealed widespread interaction with reporters and editors and a drop in general apprehension about peer reaction to such interaction compared with the past. Lingering concerns remain, however, both on the part of scientists who are worried about being misquoted and misrepresented and journalists who do not always get the information they need for stories from interviewed scientists. Regardless of these specific findings, it is clear that the audience definition for science communication projects involving scientists or journalists (or both) differs in important ways from general public efforts. Identification of a sampling frame is sometimes easier for such efforts; a survey of a census of identified reporters rather than a sample is sometimes possible due to the smaller audience size, for example. At the same time, generalizability limitations sometimes arise with such investigation; a study of science journalists working for major metropolitan newspapers may or may not generalize to general assignment reporters for local TV news stations due to differences in story length and focus.

...