News archive

Sign up to the ECB e-newsletter

Get our free quarterly email newsletter direct to your inbox.

ECB Project on Twitter

Share and Bookmark

Published on 16 November 2010

The earthquake that struck Haiti on January 12, 2010 created one of the most devastating humanitarian crises in years. 300,000 people are thought to have lost their lives, hundreds of thousands were injured, and almost a million were left homeless. Though Haiti is not one of ECB’s five country consortia, the magnitude of the humanitarian response created a natural momentum for inter-agency collaboration.

Joint evaluations are often more resource intensive than single agency evaluations, but they are quite useful for several reasons. They allow agencies to see the big picture and assess coverage and coherence. Joint evaluations enable NGOs to learn from their peers and build relationships that may enable further coordination and collaboration. Finally, accountability and transparency are improved because agency strengths and weaknesses are more easily identified.

The ECB Project has supported several joint evaluations in recent years and collected a body of knowledge on these experiences. Based on this knowledge, ECB has developed a tool, entitled What We Know About Joint Evaluations of Humanitarian Action: Learning from NGO experiences.

An independent consultant, Paul O’Hagan, was hired as the evaluation team leader. Paul was known for his experience with inter-agency evaluations, accountability, and participatory approaches. Angela Rouse (CARE ECB Agency Manager) and Katy Love (ECB Project Officer) joined Paul on the evaluation team to capture the learning from the process, and update the What We Know tool. Save the Children, CARE and other INGOs expressed interest in participating in the joint evaluation, seeking to learn from peers. They came together in Port-au-Prince to determine the scope and methodology of the evaluation. Creating the terms of reference was by no means an easy process as agencies came to the table with different interests and priorities, as well as different levels of resources and staff time. At the end of this process, it became clear that only Save the Children and CARE would proceed.

The Steering Committee decided to use a participatory methodology for the evaluation developed by Paul O’Hagan and Gerry McCarthy entitled People First Impact Methodology. The evaluation team began by working with 30 national staff, each with different sector experience, from CARE and Save the Children. Over the course of two days, this enthusiastic group was trained to listen without bias, communicate with impact, lead focus group discussions, and facilitate group sessions.

The focus group discussions in Léogâne, Carrefour and Port-au-Prince ...allowed us to experience a deeper communication with Haitian communities, especially with Haitians who are stressed out and frustrated after this natural disaster. Using this method also allowed us to see that there are social groups that have not received aid.” Esson Dorisca, Save the Children, national staff participant in the JE

A literature review and key informant interviews were also held with a cross section of stakeholders, together with a validation workshop to share learning with the wider humanitarian community. Several staff from other ECB agencies, and partners attended, including Mercy Corps and World Vision. The findings of the evaluation are captured in the evaluation report.

CARE and Save the Children had not worked closely in Haiti, and this positive experience left them wanting to explore further opportunities to work together. This evaluation also spurred country directors from several ECB agencies to start discussions on potential joint initiatives, specifically around training and accountability.

The What We Know tool will be revised in the coming months to reflect the lessons captured from the Haiti and recent Kenya experiences. Some key lessons include:

• The right people must be at the table from the beginning. Be clear on who the decision makers are and what role stakeholders at the global level will play.
• Agencies should commit to the principle of the joint evaluation and design it together. All steps of the evaluation should be undertaken together.
• An external consultant can facilitate a process to agree on the terms of reference (ToR) early on. Establishing the ToR was the largest hurdle for the team in Haiti, and they would have moved much faster if the ToR had been agreed from the start.
• A joint evaluation can involve many NGOs, civil society organizations, and government officials. Agencies conducting joint evaluations should not feel constrained to limit participation to their direct partners.While only two agencies participated in the Haiti evaluation, the process and product would have been enriched by the participation of other agencies.
• A joint evaluation is the perfect starting point for collaboration. A joint evaluation is a chance to learn together and to build relationships. It can be the beginning of a longer term relationship. 

Joint evaluations aren’t easy; in fact they are quite complex. But the collaboration that ensues is well worth the challenge. View our other Joint Evaluation results on our JE database.

Contact us for further information and don’t forget to read our What We Know About Joint Evaluations tool and the final evaluation report.

Written by Katy Love, with special thanks to Paul O’Hagan, Joint Evaluation Team Leader, and Angela Rouse, CARE ECB Agency Manager for their support in writing this story.

Related news

6 October 2011: Inter-agency Standing Team improves accountability

Our new Accountability & Impact Measurement (AIM) Standing Team is a skilled inter-agency group that is championing accountability and impact measurement across agencies and regions. The AIM Standing Team is available for deployment to support field staff with assessments, evaluations, monitoring missions and members can train teams to implement accountability principals in practice. Read more