Evaluation: Mixed Methods

Mixed Methods in Formative Evaluation

Most health communication processes are built on social marketing, a framework that emphasizes an audience-centered consumer orientation and strategies to attractively package the social product and use the optimum combination of campaign components that will achieve the pragmatic goals. The early stages of these processes are referred to as “formative evaluation” and attempt to understand the health problem; the audience's relevant knowledge, attitudes, and behaviors; any competition; and costs and benefits of the behaviors being promoted. Qualitative methods are often used at this formative stage and include key informant interviews, focus groups, and behavioral observation. At this stage, research helps identify intended audiences and focal behaviors, specify significant intermediate response variables, ascertain channel exposure patterns, and determine receptivity to potential message components.

Focus group sessions are conducted by a moderator who stimulates extensive, open-ended discussion of selected issues in a small group setting. Key informant interviews are in-depth interviews conducted with those who have unique knowledge of the intended audience, perhaps because they interact and/or observe them regularly. In-depth interviews may also be conducted with individuals from the intended audience, especially if the topics are very sensitive or the individuals difficult to recruit for focus groups. In the case of individually tailored messages, a unique form of formative research is used where the actual audience first completes an online survey instrument measuring numerous variables. The computer program then matches this individual assessment input with appropriate portions of content drawn from a database to create a tailored message that is then delivered on-screen or on printed pages.

Quantitative methods can also be used at this early stage in the planning process. Epidemiological databases are frequently consulted to describe the health problem or risk affects and what current behaviors are. Surveys such as the National Health and Nutritional Examination Survey (NHANES) or the National Health Interview Survey (NHIS) are examples of sources for these data. It is not uncommon for these surveys to fail to include all the information the communication planner needs, so a formalized survey of midsize samples representing intended audience segments might be designed to measure a broad array of variables, usually via questionnaires. These quantitative results may serve as a pretest to be compared to the same survey administered after the intervention to assess effectiveness.

...