Philanthropic Evaluation

Purposes of Evaluation

Philanthropic evaluation generally aims to measure two things: the effectiveness of a grant or grantee program or the social impact or effectiveness of the funder. According to a 2002 study conducted by the Center for Effective Philanthropy, evaluations are the most commonly used tool for assessing overall foundation performance. In the study Indicators of Effectiveness, 72% of foundation chief executives said they use grant, grantee, and program evaluations to assess the performance of their foundations in achieving social impact and operational goals.

However, attempts to assess funder effectiveness through evaluation of grants can be burdensome to grantee organizations and fail to yield the data a funder seeks. In some cases, the information that funders require to evaluate their own social impact is not the same information that would help grantees evaluate their own programs. Such evaluation requirements cost grantees time and energy in collecting and reporting information that is useless to them. In many circumstances, the data are not being used to inform the funder's strategy or practice. Funders are becoming increasingly aware of this disconnect and are working to align their information needs with that of their grantees. Some funders (for example, the Roberts Enterprise Development Fund) ask grantees what information they would find useful to track and report and use that for their own evaluations.

Challenges of Evaluation

Funders lack accountability measures and external incentives for evaluation. Grantmakers are ambivalent and perhaps confused by the concept of accountability. In contrast to many highly regulated fields, the parameters surrounding funder accountability are limited and basic.

Performance information is limited, collected inconsistently, and not focused strategically on the outcomes of funders' work. Recently, funders have moved to assume greater responsibility for the outcomes of their work. The efforts, however, tend to be more focused on funder processes and grantee outcomes, leaving the linkages between the two undetermined.

The organizational learning agenda generated excitement but failed to link to institutional mission and the needs of fields. Over the past several years, funders have embraced ideas associated with learning organizations. Grantmakers have developed agendas focused on learning from the past, including studying “best practices” and sharing project lessons among staff. However, the relationship between learning and accountability has assumed a highly polarized tenor. In some organizations, learning and accountability are seen as competing functions. The issues are especially contentious when the same personnel, often evaluation staff, are responsible for both functions. In these situations, program staff members fear being exposed by the process of evaluation. They find it difficult to discern when an evaluator is functioning in learning mode or accountability mode, “who the evaluators work for,” where their loyalties lie, and whether they can be trusted.

Evaluation is expected to fill a diversity of roles—as a learning device and as a device for judging accountability, effectiveness, management, and strategy. The ambiguity about the proper role of evaluation heightens program officers' natural fears about having their work observed and assessed. Staff members generally are unsure about the purposes of evaluation, how the process will unfold, and how the data ultimately will be used. Moreover, they tend to see that evaluation often operates in seemingly contradictory roles, and they react with understandable suspicion.

...