Unearthing secret excel spreadsheets – why measuring what matters, matters

“What’s on your secret excel spreadsheet?” This was the question posed to a room of Brazilian civil society practitioners, who had been convened by Candid in 2019 to discuss the topic of measurement in their work. Over 2018 and 2019, the GFCF had similarly coordinated a separate learning group of 15 community philanthropy partners to explore the same topic. While this process had revealed that these organizations are motivated to assess the effectiveness of their work and that many are already doing so, it had also revealed that most of these organizations maintained “secret excel spreadsheets.” Beyond simple outputs and dollar amounts collected for donor reporting, these spreadsheets capture much more complex information pointing to the roles these organizations play in their communities in effecting social change. 

We have just published Measuring what matters, a consultation paper designed to advance a conversation about measurement in civil society. It draws from the processes that Candid and the GFCF have been conducting over the last several years, as well as similar work conducted by Philanthropy for Social Justice and Peace (PSJP). The goal is to find more meaningful approaches to organizational learning and accountability, and to begin to bridge the gap between what people working at the local level see and feel is important (ie. the contents of those secret spreadsheets), and what funders want to hear is happening. To put it more simply, to begin measuring what matters.

Between a number of face-to-face meetings (when those were possible) and even more Zoom calls, more than 130 individuals representing donor organizations, INGOs, activist groups, community philanthropy organizations and support bodies have already contributed to the paper with their practical experiences. The paper has grown as part of an iterative “create and adjust” process spanning the last two years, and we foresee adapting it further as the ideas laid out in the report are tested around the world.

The paper builds a prototype measurement approach that aims to capture the richness of what happens in civil society – a universal view among those who contributed to the paper was that new methods of evaluation and measurement should be developed which better capture and communicate the nuance of this work. At the same time, bespoke measurement approaches are often out of step with what many funders want to see in their grant reports. This mismatch is a near-universal experience, as revealed by some of the following representative comments from contributors to the paper:

“We have one donor who is just after hard-core outcomes: how many roads and wells, etc. Now we are in the process of educating them and others and saying that there is a whole lot more to the process of social change. There is a gap between what we see as value and what they see as value.”

“There is a push from our donors to measure quantity. But we deliver a quality service, and it’s more about listening than money. ’Too invisible’, say our donors.”

“Often funders ask for metrics that don’t make sense in our context and are not built with a community. This is often more of a hindrance than help. Community members often react negatively to this.”

As a result of these and many other similar comments, Measuring What Matters argues that what is known in scientific circles as the “hierarchy of evidence” is in need of reform. While the hierarchy is a useful guide for evaluating interventions with clear goals, easily measured variables, and a counterfactual, it is not appropriate – and can prove harmful – if applied to complex initiatives working to build emergent solutions in conditions of uncertainty, particularly when there are limited resources and little scientific expertise on tap.

Rather than scrapping traditional evaluation methods, participants in the process felt that a more useful approach was to pioneer a parallel measurement system that demonstrates how civil society emerges and develops, how it meets challenges on the way and what it achieves as a result. This is significant, as such a system would make people in communities, where the work is being done, feel more seen, heard and understood. Many people felt that the GFCF’s Assets, Capacities and Trust Framework[1] was highly suitable for this purpose and worked well for many who want to #ShiftThePower.

Join us in measuring what matters!

The paper is a work-in-progress. Up to this point, our conversations have resulted in increased understanding about the challenges involved in measurement, identification of some promising areas for exploration, guiding principles for any system of measurement, and two recommended action steps. We would like to widen the circle even further of organizations and individuals thinking about these issues, and experimenting with the ideas laid out in the report.

Join us – with your ideas, feedback and of course those secret excel spreadsheets – for one of two webinars on 12^th November, to find out more about how you can get involved:

• Webinar 1, Thursday 12^th November at 11am GMT: Click here to register
• Webinar 2, Thursday 12^th November at 3pm GMT: Click here to register

[1] The GFCF first presented the Assets, Capacities, and Trust Framework as the “three‑legged stool” of community philanthropy in 2010. Despite differences of context, institutional focus, origins, etc. it became clear through the GFCF’s grantmaking that partners around the world strongly identified with the Framework, naming the following objectives as being critically important to their work: a) building local assets, b) strengthening the capacities and agency of communities and c) building trust.