Evaluability Assessment

Introduction

Programme evaluation is a common practice in public and private organizations in the western world. It is an essential step, and the final one when actions are carried out to solve a social problem. Rossi and Freeman (1993) defined programme evaluation as the systematic application of social research procedures for assessing the conceptualization, design, implementation, and utility of social interventions programs (Rossi & Freeman, 1993: 5).

However, before evaluating a programme, we must inquire about the need for the evaluation. Wholey (Wholey, Scanlon, Duffy, Fukumoto & Vogt, 1970; Wholey, 1983, 1987) described this question as evaluability assessment.

The concept of evaluability assessment emerged in the context of the problems Wholey and his colleagues experienced when they were working at the Urban Institute of Washington in the early 1970s. These problems were of two main types. First, stakeholders' resistance to co-operate in the evaluation, and second, the limited use of evaluation outcomes for the improvement of the programmes (Wholey et al., 1970). In short, evaluation was difficult due to stakeholders' resistance and was not useful because its outcomes did not help to bring about changes in the social interventions.

Subsequently, Wholey (1983, 1987) developed the concept further, identifying four problematic areas that increase the difficulty of programme evaluation. Such areas are:

• 1
  A lack of definition of the problem addressed, the programme intervention, the expected outcomes of the programme, and/or the expected impact on the problem.
• 2
  A lack of a clear logic of testable assumptions linking expenditure of programme resources, the implementation of the programme, the expected outcomes (to be caused by that programme), and the resulting impact.
• 3
  A lack of agreement on the evaluation's priorities and its intended uses.
• 4
  The inability to make decisions on the basis of evaluation information.

Wholey himself established concrete problems to identify the evaluability assessment process associated with these four areas. Such problems are poor programme definition, misjudgement to implement the programme, lack of establishing realistic objectives, and contradictory presence of non-expected effects. In this regard, we can use the definition reached by the author:

Evaluability assessment is a diagnostic and prescriptive tool that can be used to determine the extent to which any of these four problems exists [Page 379]and to help ensure that such problems are solved before decisions are made to proceed with any further evaluation. (Wholey, 1987: 78)

Evaluability Assessment: Uses

Smith (1989) identified two steps in the carrying out of evaluability assessment, and three possible uses of it, depending on moments and objectives. According to the steps, firstly, evaluability makes a contribution to the guarantees and technical credibility of the programme. Secondly, evaluability assessment determines the plausibility and feasibility of the programme and its evaluation. In conclusion, the two old problems that Wholey and his team pointed out are discussed: the knowledge of the easiness and feasibility of the evaluation and the knowledge of the utility of the evaluative process.

As regards the uses of evaluability assessment, the first of them should be its use as a summative tool or as a preliminary step in evaluating the effectiveness or the programme's impact. The second use is as a formative tool to decide what can be changed to make the programme more evaluable. The third use is as a planning tool to define objectives, identify actions for attaining those objectives and find the appropriate resources for implementing such actions.

...