Advocacy in Evaluation

Advocacy, defined in many of the debates as taking sides, is seen as one of the more intractable matters in contemporary evaluation. For example, Linda Mabrynotes, “Whether reports should be advocative and whether they can avoid advocacy is an issue which has exercised the evaluation community in recent years. The inescapability [from a constructionist stance] of an evaluator's personal values as a fundamental undergirding for reports has been noted but resisted by objectivist evaluators [who have] focused on bias management through design elements and critical analysis. At issue is whether evaluation should be proactive or merely instrumental.” M. F. Smith, considering the future of evaluation, writes, “The issue (of independence/objectivity versus involvement/advocacy) is what gives me the most worry about the ability of our field to be a profession.”

Mabry's and Smith's concerns seem well justified. Granted, the issue of advocacy in evaluation is complex in that examining one aspect (for example, the nature of objectivity in social science) leads to another aspect (such as methods of bias control), then another (such as the role of the evaluator as an honest broker, a voice for the disenfranchised, or something else), then another as quickly. The issue is high stakes in that the approach taken may lead to different evaluation processes, designs, measures, analyses, conclusions, and, probably, consequences. With a few exceptions, there appears to be little systematic comparative study on this point. The issue of advocacy is divisive in our field in that beliefs about advocacy seem deeply and sometimes rancorously held. Last, despite the development of evaluation standards and guidelines, in practice, there is no more than chance agreement and considerable unpredictability on what experienced evaluators would do in quite a few ethical situations involving advocacy.

Often, advocacy issues are posed as questions: What is the role of the evaluator? Is impartiality a delusion? In particular, do the Guiding Principles of the American Evaluation Association place a priority on Principle E (Responsibilities for General and Public Welfare) before Principles A through D, if all cannot be satisfied equally? Does credibility require objectivity? These questions were prominent in the1994 examination of the future of evaluation, based on statements of more than 25 leading evaluators, and they became even more so in the 2000 statements.