Evaluation Research

Programs and Projects

Programs are activities or groups of activities undertaken by a government to serve the public. Programs are usually continuing activities such as routine service delivery in local governments. Projects, on the other hand, are combinations of human and nonhuman resources pulled together in a temporary organization to achieve a specified purpose.

Inputs and Outputs

Outputs are the amount of products and services delivered (completed) during a reporting period. Outputs result from the type of processing or technology employed by the agency. Thus, number of students vaccinated is an output resulting from the delivery of immunizations. The work of most organizations involves chains of processors (sub-units, individuals, machines, etc.) that convert resource inputs (money, person-hours of effort, etc.) into outputs. The output of one processor may become the input of another processor in sequences that produce the final output.

Outcomes

Outcomes are events, occurrences, or changes in conditions, behavior, or attitudes that indicate progress toward achievement of the mission and objectives of a program. There may be intermediate outcomes and end outcomes. An intermediate outcome might be the percentage of persons completing smoking cessation training who have continuously ceased smoking for six months. An end outcome could be the percent of reduction in reported cases of lung disease that can be attributed to smoking following an extensive anti-smoking campaign. Improvements in some service quality characteristics are treated as intermediate outcomes because they are important to the public. Examples are timeliness in delivery of a service, accessibility, or convenience; accuracy of assistance; courteousness; safety; and customer satisfaction.

Efficiency and Productivity

The ratio of the amount of input to the amount of output (or outcome) is labeled efficiency; for example, the dollar cost per customer assisted. The reverse of this ratio is labeled productivity (customers assisted per dollar). These are equivalent numbers. Efficiency and productivity are important criteria for judging efficacy in evaluation research.

Demographics and Workload Characteristics

Demographic and workload characteristics are most important in evaluation research to discriminate among affected persons, areas, or objects. Thus, age and medical problems of senior citizens who receive [Page 295]“meals on wheels” service may be important controls on effectiveness of program delivery or other outcomes studied. Workload characteristics, especially where there is variance in quality, quantity, or other differences, are important similarly. Subtle differences in inputs vitiate research design because not all participants receive exactly the same treatment.

Research for Program Planning and Development

This is the category of getting started in evaluation research. Its purpose is to design programs in conformity with intended goals. It seeks answers to questions such as, What is the extent of the target problem population? What research and development program planning and implementation issues need attention?

The specification of target populations often is difficult. Too rigorous or too narrow a definition may result in exclusions from a project of target units that have exceptional or unusual levels of need for the program services, or whose potential for benefiting from the program is high. Too broad a definition may result in uneconomical investments in targets with low potentials for benefiting from the intervention.

...