Prevention Evaluation

Levels of Evaluation and Results

Although textbook definitions and titles may vary, there are three levels or types of prevention evaluation. A comprehensive evaluation includes impact, outcome, and process evaluation. These three levels directly correlate with program goals, objectives, and activities. Impact evaluation involves the long-term, global effects of the program and will include incidence and prevalence data as well as risk and protective factor data. Outcome evaluation describes the immediate or direct effects of the program strategies and includes data on knowledge, awareness, attitudes, and skills. Process evaluation includes any combination of measurements obtained during the implementation of program activities to control, assure, or improve the quality of performance or delivery of strategies that are implemented to achieve established goals and objectives. Data collected will include a description of appropriate educational and other activities, numbers served by the program, participant satisfaction with the activities, and the actual planning process by the sponsors.

Evaluation Design

Evaluators or program staff must determine an evaluation design or plan with regard to the time, resources (financial and personnel), population, and setting. Consideration must be given to the type of data (qualitative or quantitative) that is necessary, as well as to what exactly is being evaluated (the variables). Finally, a determination must be made by the evaluator or staff regarding the final audience to receive the evaluation results (e.g., community stakeholders, funding agency).

There are three types of evaluation designs commonly used in health and prevention programs: experimental, quasi-experimental, and nonexperimental. Experimental design has the greatest control over various factors involved with the program that may have an influence on results. Random assignment to two or more groups, either experimental (the groups receiving the program or intervention) or control (a group similar to the experimental group), with measurement in both groups, is the key feature of this design. Measurement will involve a pretest-posttest or a posttest only. This design will produce the most reliable and defensible program results. A quasi-experimental design uses two or more groups also, the experimental and a comparison group. A comparison group is as similar to the experimental group as possible, such as two classrooms of students with similar characteristics. However, there is no random assignment and the program staff cannot control for all factors that may influence the results. A pretest-posttest measurement is used with this design. The nonexperimental design (no comparison or control group) has little control over factors that may affect the results of a program.

...