Process Evaluation

Researchers evaluating programs gather information about program outcomes by answering questions such as “Did the program achieve its goals?” and “Was the effect of the program substantial?” This kind of evaluation (sometimes called summative evaluation) is clearly important. Equally important is the evaluation of the processes the program followed in trying to achieve its goals. A process evaluation is also called an implementation evaluation [Page 408]and sometimes a formative evaluation. Outcomes evaluations tell you what was accomplished; process evaluations tell you how it was accomplished. Knowing the “bottom line” is necessary but not sufficient. To use what was learned from one program to plan or to improve other programs, you need to know about both outcomes and processes. Process evaluations are widely used in most organizations that conduct evaluations, public and private.

Evaluation researchers typically use a wide range of methods, and this is especially true of process evaluations, which usually collect both qualitative and quantitative data. Data from observations, documents, and interviews are often central to a process evaluation. A process evaluation usually takes an in-depth, case study approach. Documents such as meeting minutes and reports are important for establishing context and time lines. Interviews are frequently the key method for understanding program delivery. Interviews are conducted both with program officials and with the individuals and groups served by the program.

Process evaluation focuses on the early stages of development and actual implementation of the program, including any changes in goals and methods, such as adjustments to the context in which the program worked. Although summative assessment of the impact of the program is not the point of a process evaluation, judgments of quality are frequent, especially as they pertain to the administration, coordination, and delivery of services. Process evaluations often address the program's work in evaluating itself: What is the program's process for collecting, recording, and interpreting evaluative data about its activities? What is being learned, and to what use is it put?

Typical questions addressed by process evaluations are as follows:

• Is the program being implemented as planned?
• Who actually participated?
• Is the program reaching the kinds of people it was meant to reach? Are the participants willing? How long did they participate? How many dropped out?
• How does the program fit into its environment?
• What problems were encountered? Did they lead to alterations in program delivery or objectives?

Understanding exactly how a program operates is essential for three purposes: (1) improving the program as it is implemented, (2) drawing lessons for other programs, and (3) interpreting the results of outcome evaluations. For example, a program might work well because it devised a new method of delivery in response to a problem. A process evaluation should discover this; an outcome evaluation would probably miss it.