Evaluation Research

Evaluation research originally emerged in the education and human services arenas as a means for improving social programs, but it now focuses more on determining the overall effectiveness of programs. Both approaches are valuable in that they yield evi-denced-based research to secure funding or develop and implement effective programs.

One type of evaluation research, formative evaluation, deals with influencing the beginning stages of program development. The evaluation researcher collects and interprets data about how a program operates in its early stages and then translates this information into concrete suggestions for improvement to be shared with program staff. For example, if a local school board requests a formative analysis of a new Holocaust Education program, the researcher would carefully review the program's goals, objectives, curriculum, and instructional materials as well as collect data from students, teachers, and administration about how the program functions. After analyzing and interpreting the data, the evaluator could offer program improvement suggestions in such areas as curriculum, instructional materials, and overall program management.

[Page 343]The second type of evaluation research, process evaluation, determines whether a given program was implemented as designed. Understanding the sources of variation in program delivery allows evaluators to understand better how a program works. Process evaluations are especially important when trying to understand multisite programs such as the Gang Resistance Education and Training program. The focus is not to determine if a program works, but rather to determine how it works and what variations in its delivery could potentially contribute to its overall effect.

Summative evaluation, the third type of evaluation, focuses on determining the program's outcomes. In other words, did the program work? Did it meet its professed goals and objectives? This type of evaluation research resonates strongly with funding agencies. For example, in the 1990s, the National Cancer Institute funded several health promotion programs reaching out to Hispanic women to raise awareness, increase knowledge, and improve access to cancer screening opportunities. For funding eligibility, each proposed program had to include a summative evaluation component. The proposal design comparing the intervention group with another group had special appeal to the funding agency.

Program evaluations, regardless of type, typically encompass a sequence of stages. The initial two stages involve understanding the essence of the program. More specifically, stage 1 involves the formulation of the program's goals and objectives, while stage 2 involves developing an understanding of the program's delivery, setting, and participants. Stage 3 focuses on designing the evaluation. The design can include both qualitative and quantitative research methodologies. Choice of methodologies depends on several factors, including purpose of the evaluation and type of information needed. The fourth stage involves collecting, analyzing, and interpreting the data. The fifth and final stage of the evaluation involves utilizing the evaluation results to improve the program or to verify that it works.