Evaluation Research

Evaluation Research

A distinction is often made between basic (or academic) research and evaluation (or applied) research. Evaluation research is applied in that the aim is to produce knowledge that will contribute to greater understanding of the effect of a defined activity. This activity may be referred to in a number of ways such as intervention, initiative, and policy. An intervention is a specified, but not necessarily specific, activity. It may already be in place or may be a new type of activity. Evaluations are often set up when interventions are being initiated or when unexpected problems arise. It is considered as important to find out what interventions do and do not work and also how things work or what prevents them from working. In this [Page 304]way, lessons can be learned and taken forward in future attempts to improve the social world in which we live. Developing knowledge of how things work is where qualitative research has a particular contribution to make to evaluation.

Evaluation Design

Evaluation research differs from basic research in that it is usually set out as such at the start, with aims and objectives (defined in the research design) that are closely related to the intervention. A wide range of types of evaluation design exists. However, two main types that are defined by the contexts in which they take place can be identified: program and organizational evaluation. Program evaluation is applied to intervention programs carried out primarily to address social problems in a population or in a community setting. For example, a study may be set up to evaluate a program of activity aimed at reducing levels of crime. Depending on the nature of the intervention, the program and consequently the evaluation can become complex, and multiple evaluation studies may be conducted in parallel. Organizational evaluation is research that is carried out in organizations to examine the factors that influence the production and delivery of goods and services. The focus is on performance and productivity. An example here would be an evaluation of a police communication skills training initiative to improve relations between police and the public.

Quantitative methods have traditionally dominated in evaluation due to the emphasis on outcome. For example, the success of a crime prevention initiative is likely to focus, initially at least, on whether crime had been significantly reduced at the end of the intervention period. Influenced by the use of experimental method in the natural sciences, the “gold standard” approach is the controlled trial. This involves “before and after” comparison between a site where the intervention has been applied and a site where it has not. However, this methodology has limitations in social settings because social variables are difficult to define and control. Consequently, a more realistic approach to evaluation has emerged that highlights the importance of understanding the process of social change or, in other words, the context and mechanism involved. The goal of evaluation in this approach is to understand the relationship among context, mechanism, and outcome.

...