An Evolution of Evaluation in Grantmaking With a Participatory Lens
October 9, 2023 | 7 min Read
SHARE ARTICLE:
We all want to know whether our work makes a difference. Among grantmakers, there tends to be a lot of focus on impact and outcomes, as well as metrics to measure impact. Grantmakers want to know if their funding has created the change they have envisioned. But is this the right question?
Here, we explore for whom change is desired and who is defining and measuring that change. We offer some practical tips, some examples of funders doing this work, and some resources.
Power Imbalance in Traditional Evaluation
As grantmakers, we tend to monitor and evaluate our strategies and programs using metrics that we deem important. Our intention is to understand whether grants have had the impact we envisioned. It is our hope, as funders, that financially resourcing a movement, an organization, or an individual can lead to positive change. On its face, evaluation seems like a neutral activity, designed to help us understand what’s happened, and to change course where needed.
But, monitoring, evaluation and learning (MEL) frameworks and approaches may not be as objective as they seem. They are created with the biases and worldviews of those who developed them. The data collected is usually owned by the grantmaker, not questioned, and not shared back with the grantee or any larger community. For issues with this, check out Vu Le’s 2015 post “Weaponized data: How the obsession with data has been hurting marginalized communities.” And, the focus of MEL is generally on evaluating grantee projects or organizations, rather than on the grantmaker and its practices.
All too frequently, the grantmaker alone is determining, leading, and benefiting from MEL processes with no input or collaboration from the people, organizations or community impacted.
MEL, as it turns out, is not neutral, but yet another place where power differentials show up.
Consider:
Who defines objectives and “success”?
Who decides what is measured?
Who manages the monitoring and evaluation?
For whom is monitoring, evaluation, and learning being done?
Who owns the information collected?
Where does the learning go?
For many grantmakers, the answer to these questions is our own institutions. We define success, we decide what to measure, we collect the information, we own the data, and we don’t share the learning from the process.
A Shared and Flexible Understanding of Impact
As practitioners of and advocates for participatory philanthropy, we believe there’s a better way. A way that can be more fair, improve the institution of philanthropy, and deliver a shared and flexible understanding of impact. Like many other activities in participatory philanthropy, this approach considers the process to be as important as the outcomes. As the art critic John Berger said in his book Ways of Seeing, “We only see what we look at. To look is an act of choice.” Indeed, if we commit to shifting power, we need to see and understand things differently. As Ceri Hutton wrote for the Baring Foundation, “PGM [participatory grantmaking] funds denote a different paradigm of funding and monitoring and evaluation need to be ‘re-evaluated’ in the light of this.”
The practice of participatory evaluation aims to disrupt power dynamics, and to generate knowledge as a result of collaboration. It promotes mutuality instead of extraction. Participatory evaluation is based on the premise that everyone has knowledge; everyone has biases; and the people closest to an issue know the most about it.
“Participatory approaches to M&E invite us to think about which belief systems are privileged in our work, whose knowledge and priorities matter most, and why we do the work we do in the first place. If our ultimate aim is to support real-world impact and transformative change through M&E, then the participation of affected communities is a precondition. You cannot have empowerment without participation: empowerment is not something we ‘do’ to other people, but is itself a participatory process that engages people in reflection and inquiry to understand the power they have, and to take action for change as they define it.”
Just as in grantmaking, in MEL, there can be a spectrum of participation of community members. This can range from them being consulted to owning the results.
In the Equitable Evaluation Framework™, advancing equity is at the center of all activities. Practitioners are tasked with understanding how their assumptions or mindsets about objectivity, rigor and evidence; resources; roles; definitions, perceptions, and decisions; relationships; and productivity and accountability all may bias their MEL approaches.
“The pursuit of equity necessitates those who have amassed tremendous wealth and engaged in philanthropic endeavors to acknowledge and reflect upon the ways in which privilege (and thus racism) have been key contributors to that wealth. For evaluation as a field, the notion of equity challenges what practitioners accept as valid, rigorous, and objective. Equity asks us to consider multiple truths (some perhaps more important than others); to weigh the complexity of our current society, the multiple communities that exist within that society, and the multiple identities we each carry; and to discover new and multiple definitions of validity.”
Many grantmakers, and perhaps particularly participatory grantmakers, have embraced learning and reflection approaches. Many of these grantmakers gather participants in the spirit of sharing learning amongst different stakeholder groups, usually grantees and/or community decision makers, to consider their experiences and their take on impact. These convenings are often driven by the needs and goals of participants.
At the global Disability Rights Fund (DRF), for example, the first MEL framework was developed at a gathering of staff, advisors, and donors—the majority of whom were activists with disabilities from the Global South. Co-development led to conceptualization of participation of people with disabilities in advocacy for legislative, policy and government program change as an impact—in and of itself—regardless of outcome. This resonated with “nothing about us without us,” and with the mandate of the Convention on the Rights of Persons with Disabilities that persons with disabilities be involved in decision-making. In a world in which people with disabilities have been excluded from most daily activities, the process of participation was seen by the disability community as the most important impact. [See pages 47-48 of the report Grassroots Grantmaking: Embedding Participatory Approaches in Funding by Hannah Patterson, to learn more about the DRF/DRAF evaluation framework.]
Fenomenal Funds, a feminist funder collaborative supporting women’s funds, gathered guidance for their MEL framework from their member funds and donor partners, which included a desire for immediate learning, the capacity to capture complexity, and working with a participatory and transparent approach. This input led them to adopt an Emergent Learning framework and feminist evaluation principles, which recognize evaluation as a political act and the information generated as key to advocacy. Their approach underlines the need for those practicing MEL to iteratively question all assumptions about what constitutes knowledge, who has it, and how it gets used.
Other participatory grantmakers have developed new MEL tools to better reflect a power shift to grantee perspective on impact. RAWA Fund, for instance, has used grantee self-assessment based on narrative, story-telling methodologies, as a basis for evaluation. As Moukhtar Kocache described in Deciding Together, this challenges “a pre-established framework, which means we never really see what groups do and don’t do. All we know is what they tell the donors they’re doing. We’re engineering responses, rather than being open to what happens.”
FRIDA | The Young Feminist Fund reframed traditional MEL language, outlining a MEL Framework guided by three core organizational values: participatory collaboration, accessible language and understanding, and collective feedback (all of which are explained in their framework). The FRIDA framework encourages a non-linear approach and embraces the leadership of all people.
“FRIDA’s Strategic MEL Framework is our attempt to ‘queer’ (challenge + change) the language, structures, and tools normally used to think about monitoring and evaluating programming, development outcomes and resource accountability. Our decision to ‘queer’ traditional approaches to MEL is a political stance. FRIDA believes that transforming the practice of MEL will make understanding goals, outcomes, and indicators more accessible, approachable and user-friendly for the collective FRIDA community.”
Making Evaluation About Us as Well
In addition, it’s becoming more standard practice for grantmakers to turn the evaluation lens on themselves. Many grantmakers ask for feedback from grants applicants, grantees, and community decision makers, using tools like Center for Effective Philanthropy’s Grantee Perception Report. Wikimedia Foundation, the non-profit that stewards Wikipedia, regularly asks for feedback from stakeholders and publishes what they share, along with any changes that are made to the programs from that feedback.
As grantmakers, we can reconsider traditional monitoring, evaluation and learning approaches from start to finish. Here are some elements to rethink:
When defining the goals of grantmaking on which your MEL system will be based, involve the grantee partners or broader community you serve. Ask what they want to see as a result of grant funding and how they will know that it has happened? What are the changes that are meaningful to them? Their goals and indicators of success may be different from yours.
Ensure that the process is directed by the value of “nothing about us without us,” the motto of the disability rights movement. Ask how the community wants to be involved at every step. Jointly define additional values with your community, which might include principles such as Be collaborative, Uphold non-extractive practices, Do not tokenize, or Involve a diversity of community members.
Work with community participants to identify roles and set parameters around what they do and don’t want to do as part of MEL. Compensate them for their contributions generously.
Identify and challenge usual power relationships. Think about the donor role as facilitator and the community as owner of the results.
Instead of using results to evaluate or judge grantees, consider findings as learning opportunities that can be used to change grantmaker strategy as well as help the community advocate for larger changes in their environment.
Naomi Falkenburg encourages us to reconsider where the focus of evaluation should lie: “participatory approaches require us to look critically at power and ultimately to redistribute it. Participatory M&E is therefore not just about ‘them’, but also very much about ‘us’.”
Know Where You Are–And Where You Want to Go
Consider your evaluation processes–both for your grantees as well as for your organization–to be reflections of the relationship you have with your community. If you are addressing a shift in power, having a framework to help you figure out where you want to go, set benchmarks, and understand best practices can help with desired changes. For a participatory framework to help you take next steps, check out our webinar “Participation in Philanthropy: How Will I Know If I’m Doing It” where we discuss a self-assessment tool we developed that can help funders see where they are and identify opportunities to expand participation by people with lived experience in their work, including within their MEL processes.
Manage your entire grantmaking workflow, from online grant applications through payments
Find out how Blackbaud Grantmaking™ fits your organization.
