The Unfulfilled Promise of Promoting Evaluation Culture by VOPEs: The Case of the MENA Region*

June 13, 2020

*Translation of the blog post is also available in French here as done by the wonderful colleague, Sana Ben Salem.

Since its establishment in 2011, the Middle East and North Africa Evaluation Network (EvalMENA) held eight regional evaluation conferences. Some 750 evaluators, academics, nonprofit professionals, lawmakers, and policymakers from across the region and beyond attended these conferences during the past nine years. Most evaluation professionals in the region know of the Network. In case an evaluator from the region did not attend one of the Network’s annual conferences, there is a good chance that she or he is a member of its longstanding and rather dynamic mailing list. Additionally, while only two national Voluntary Organizations for Professional Evaluation (VOPEs) were established before the initiation of EvalMENA (Morocco and Egypt), five national VOPEs were founded following its commencement (Palestine, Jordan, Tunisia, Algeria, Lebanon, and Egypt’s 2nd VOPE). Other groups exist in other countries (Sudan and Mauritania). All national VOPEs have since joined EvalMENA and now constitute its official VOPE membership. The Network has also been aptly represented in the global evaluation structures. Namely, EvalMENA representatives sit on the boards of the International Organization for Cooperation in Evaluation (IOCE), EvalPartners, and the International Development Evaluation Association (IDEAS). 

Despite these achievements, very few outside the evaluation circles know about the Network, its mission, and its role in the society overall. This lack of awareness is not unjustified; for the past decade, EvalMENA has mostly targeted its members, affiliates, and followers but failed to connect with the broader community through advocacy and outreach. 

Besides providing an intellectual space and opportunity for their members to learn, exchange, and share opportunities with one another, most regional (and national) VOPEs, including EvalMENA, set promoting evaluation culture as a key objective for themselves. Enhancing and promoting evaluation culture can be understood in three levels:

1. Evaluator and evaluation community level: creating a cadre of knowledgable, committed, and motivated evaluators who advocate for evaluation and its role in their organizations and sphere of influence overall.
2. Organization Level: enhancing the recognition, demand, and use of evaluation within organizations and institutions to advocate for evidence-based (or at least, evidence-informed) policymaking.
3. Society Level: raising awareness about evaluation and its added-value to widen the appreciation and demand for evaluation by citizens and contribute to strengthening the transparency, accountability, and learning culture among the general public.

EvalMENA has primarily worked on the first level, and, to a lesser degree, on the second level. For the third level, almost a decade has passed without a notable achievement. Case in point, the Network was established the same year that witnessed the spark of Arab Spring revolutions. Nonetheless, EvalMENA did not implement any activity – that I am aware of – in solidarity with these social movements and demands of democratic change since then, neither did it even issue one public statement in this regard as one observes in the case of VOPEs in other countries and regions (e.g., AEA, CES/SCE, and EES ).

One must point out that most evaluation scholars who examined evaluation culture almost entirely focus on the organization/institution level (and to a lesser degree, on the evaluator/evaluation community level). They too rarely consider the third level, i.e., Society Level –let alone anything near Donald Campbell’s “Experimenting Society” concept. See, for example, Chelimsky, E. (1997), Owen, M. J. (2003), Mayne, J. (2008), Mayne, J. (2010), Mayne, J and Rist, R. (2006), Hanwright, J. and Makinson, S. (2008), Tudawe, P. I. and Samranayake, M. R. (2008), Jacob et al. (2015), and Dahler-Larsen, P. and Boodhoo, A. (2019). This gap in the literature about the evaluator role in advocating for evaluation culture on the society level stems from, in my view, the narrow lens through which mainstream evaluation scholars view the evaluator role. They often underestimate, overlook, and reject to recognize the crucial role of evaluators as advocates and activists. A more detailed discussion about this topic can be found in this previous blog post. 

Returning to VOPEs, though, this shortcoming, I believe, is not an EvalMENA-specific phenomenon. It applies to multiple other VOPEs to the best of my knowledge. I have observed remarkably similar trends in numerous other VOPEs I came to learn about through volunteering with EvalPartners, IOCE, and EvalYouth for the past several years. Literature and anecdotal evidence also indicate parallel or comparable experiences (see, for example, Voluntary Organizations for Professional Evaluation (VOPEs): Learning from Africa, Americas, Asia, Australasia, Europe, and the Middle East). 

I am concentrating on EvalMENA because it is the Network with which I am most familiar and have first-hand experience. I joined the Network soon after its inception upon founding its third official VOPE member, the Palestinian Evaluation Association (PEA), which was the first newly established national VOPE to join after the Network’s launch. Since then, I served as a member on its Board for three years, helped organize most of its annual conferences, and designed and implemented many of its projects. 

There are several rationales why I believe we have failed so far to achieve one of our key objectives: 

Firstly, EvalMENA, as in the case of several other under-funded regional VOPEs, relies heavily on a handful of overworked volunteers. While the volunteer base has grown in recent years to include some young and emerging evaluators from several countries in the region, a limited number of the network founders still dominate the critical decision-making processes. Young evaluators’ contribution was most visible in the last EvalMENA conference – EvalMENA2020 – which was one of the most diverse conferences so far in terms of participants’ backgrounds and topics. National VOPE presidents constitute the EvalMENA’s Board, which is chiefly inactive. Young evaluators’ and volunteers’ contribution is mainly restricted to the demanding and labor-intensive conference organization. They almost have no say in the critical decision-making mechanisms –e.g., network registration and governance structure, membership modality, programming beyond annual conferences, and overall strategic directions of the Network among similar critical issues. 

Secondly, EvalMENA implemented merely a handful of projects during the past nine years. Its primary focus has been almost exclusively on organizing the annual regional conference. The conference presents a unique platform for evaluators and “non-traditional stakeholders” (e.g., policymakers and lawmakers) to discuss and agree on ways, strategies, and potential interventions to foster EvalMENA and national VOPE effort in promoting evaluation culture on the national level. Nevertheless, national VOPEs rarely implement any activities with the potential national partners on the ground beyond the conference. In most cases, VOPE leaders and policymakers/lawmakers who meet at an EvalMENA conference only meet again during the next one with little to no coordination during the period between the two conferences. 

Thirdly, the few projects the Network has implemented were all funded by external donors (often INGOs), channeled through one of IOCE’s or EvalPartners’ programs, mainly the Peer-to-Peer Support Program. No other major activity – other than the Winter School for Young and Emerging Evaluators (organized by EvalYouth MENA chapter and UNFPA Arab States Regional Office in 2019) – was implemented by EvalMENA beyond the annual conferences and IOCE- or EvalPartner-funded projects. Additionally, all previous conferences – except the last one (EvalMENA2020) – were fully funded by external donors and did not include a participation fee. This financial dependency, coupled with the lack of individual and VOPE membership fee scheme, contributed to creating a network that is entirely dependent on external funding and, inevitably, heavily donor-driven in its programming and interventions. Because donors do not include promoting evaluation culture as a central objective in their call for proposals, VOPEs do not design interventions that incorporate this objective. IOCE’s and EvalPartners’s funding schemes do not usually explicitly include promoting evaluation culture as an objective and instead focus on the top-down objective of “strengthening national evaluation/M&E systems.”

Finally: EvalMENA – and indeed most national VOPEs in the region – suffer from a limited involvement and initiative of evaluators beyond a few highly active founders and volunteers in the case of EvalMENA and mostly board members in the case of national VOPEs. EvalMENA and all its member national VOPEs do not have topical interest groups (TIGs), as in the case of other VOPEs. I believe that the low level of member involvement and initiative is due to the lack of a sense of ownership among members – as they do not pay membership fee and do not elect the Board members – and because they cannot join specific TIGs where their voice can be heard. Although several attempts to establish TIGs were made in previous years, they were not maintained due to the lack of volunteer commitment, although, in my opinion, they failed due to the lack of follow-up on the part of the EvalMENA leadership. 

The rudimentary governance structure of the regional VOPE, the feeble national VOPEs, the dependency on external funding, and the limited individual evaluator involvement in the national and regional VOPEs are among the main factors contributing to the weak advocacy role of EvalMENA and its member national VOPEs. These factors are also contributing to the inability of these VOPEs combined to achieve progress in promoting evaluation culture in their countries and the region overall.

Evaluation associations that do not invest in promoting evaluation culture within their societies – while focusing merely on the career interests of their members – are not living up to their potential and constitute a burden on societies that host them. They contribute to the toxic inequality predicaments these societies suffer by focusing on the narrow interests of a few privileged members. Hence, a utilization-focused evaluation of national VOPEs’ and EvalMENA’s accomplishments, shortcomings, role, objectives, governance, and strategic direction is necessary if they intend to alter the status quo and ensure their relevance and utility as genuine and effective agents of change in their countries and the region.