Program Evaluation and Auditing

The structure of policy making is a continuous multistage process during which programs or policies are designed, implemented, evaluated, and modified. Program evaluations are the part of the process focused on determining whether [Page 1360]intended objectives have been met. In this way, program evaluation is an institutional tool for implementing accountability and control. To a greater degree, program evaluation enables the policy process to be continuous by providing information on the necessity of program changes or modifications.

There are three general types of program evaluations—auditing, formative, and summative. The specific type of evaluation needed largely depends on the phase of the program and the requirements of the organization engaging the evaluation.

Auditing is a type of program evaluation focused on the accounting of procedures or funds used in the implementation of a program. In terms of procedures, auditing examines the activities of a program and verifies whether correct procedures were followed. Fiscal auditing accounts for the funds used in the implementation of a program and similarly determines whether they were used appropriately. Both types of auditing are tightly linked to the accountability and control aspects of program evaluation. Auditing may occur at any stage of a program's implementation.

Formative evaluations occur during a program's implementation. These evaluations periodically monitor the program's progress in achieving preset goals or objectives. Formative evaluations allow users to identify trends and cumulative effects. Moreover, they determine if changes in program delivery are necessary. In this way, formative evaluations aid in the development of programs by identifying whether changes in policies or procedures are needed.

Summative evaluations are different from formative evaluations based on the phase in which they are conducted. This type of evaluation is typically conducted at the end of a program or after a period of time during which the program should have achieved some level of result. Summative evaluations, the most common type of evaluation, provide information to decision makers on whether the program achieved its goals. Summative evaluations are an important tool in ensuring accountability, as they identify whether the funds used were an efficient or effective allocation of resources.

The literature reflects a lack of agreement on the purpose of program evaluations. Generally, program evaluations are seen as the process of measuring and assigning a level of merit or worth to a program. However, others disagree and see program evaluation as a branch of scientific inquiry interested only in results, leaving judgments of value and worth judgments to others. These distinctions are important as they guide the roles of the evaluator, which are classifiable into three categories—methods, valuing, and use.

The methods approach subscribes to the tenets of social inquiry. Evaluators using this approach identify program evaluation as an outgrowth of scientific inquiry. Therefore, these evaluators are concerned with generalizability and internal and external validity. They are also interested in the reduction of bias and errors. Those using this approach leave judgments of merit, value, and worth to the users of the information.

Evaluators following the valuing approach see themselves as having an additional role in the process. Unlike the methods approach, this category makes the distinction between evaluators and researchers. This approach views the role of the evaluator as not only researcher, but also as an evaluator of those findings. This is based on the belief that evaluators have the knowledge, skills, and abilities important to making valid judgments. This is contrary to the methods approach holding the evaluator as an objective participant. Evaluators using this approach are active participants in assigning a level of value or worth to the evaluated program.

...