
Standard program evaluation methodologies may not be sufficient to determine whether collaboration among institutions is working and making a difference.
Over the past several years of operating our own networks and providing advice to others, IISD has noticed that there is not yet a consistent discipline or framework for the monitoring, assessment and evaluation of collaboration among groups of institutions.
We have observed that network and partnership evaluations often begin with a process to solicit from members their original expectations and their observations about whether their particular collaboration accomplished what it set out to do. The difficulty with that approach is that the collaboration is not assessed against any external benchmarks—against what network or partnership management experience tells us could be optimal performance measures. We believe that there is sufficient understanding now of how networks and partnerships can and should be managed, that it is no longer enough to assess them against the expectations (either high or low) of those involved.
IISD's primary area of interest is in developing simple and powerful network evaluation models, although we are also working with other experts on the review of partnerships. We thank those who have made their evaluations public, for others to learn from.
We are contributing to the work of the U.K.-based group BPD (Building Partnerships for Development in Water and Sanitation) on the evaluation of partnerships. The paper, "Turning partnership evaluation on its head: considerations for partnership practitioners" is forthcoming at www.bpdws.org.
Knowledge Networks: Guidelines for Assessment (2004) (PDF - 349 kb)
This paper explores five major elements of network performance and related indicators of success: effectiveness; structure and governance; efficiency; resources and sustainability; and life-cycle analysis. The paper includes a sample process for undertaking an assessment of a knowledge network.
Measuring While You Manage (2001) (PDF - 337)
This paper details the strengths and weaknesses of common measurement and evaluation frameworks (SWOT analysis; Results-based Management; Logical Framework Analysis; Outcome Mapping; Appreciative Inquiry; and human resource performance assessment) and proposes a simple network evaluation model that incorporates the best of each.
A selection of evaluations undertaken by IISD
Evaluation of UNESCO's Community Multimedia Centres: Final Report (2006) (PDF - 348 kb)
UNESCO's Community Multimedia Centre initiative promotes sustainable local development through community-based facilities that combine traditional media like radio, television and print with new information and communications technologies (ICTs) such as computers, the Internet and mobile devices. This evaluation assesses what can be learned from the pilot phase that will both strengthen the CMC model and improve UNESCO's programming for future CMCs, in particular through the scale-up initiative. It was conducted by the International Institute for Sustainable Development between April and October 2005.
ENRAP II: Knowledge networking for rural development in Asia: A Mid-term Review (2005)
The International Fund for Agricultural Development and the International Development Research Centre commissioned a mid-term review of the second phase of ENRAP: Knowledge Networking for Rural Development in the Asia/Pacific Region. The report analyzed the functioning of the ENRAP program and suggested adjustments and lessons learned. In addition, the authors provided insight into what ENRAP might do in future with respect to the use of information and communications technologies and knowledge-sharing within and across its projects in the region.
International Forum for Rural Transport and Development: Network Evaluation Synthesis (2004)
The International Forum for Rural Transport and Development (IFRTD) is a global network of individuals and organizations interested in rural transport issues in developing countries. IISD was commissioned to: assess the relevance of IFRTD's mission and mandate; assess how effective IFRTD is functioning as a network; indicate the extent to which IFRTD is having an impact; assess how the different stakeholders in the network are accountable to each other; and assess the sustainability of the network in the long-term. This evaluation was done in partnership with U.K.-based researcher Madeline Church, introducing her model of threads, knots and nets to IISD's network research and evaluation models.
The Sustainable Development Communications Network, 1996-2001: An Evaluation (2002) (PDF - 431 kb)
The SDCN evaluation was IISD's first attempt at applying new network evaluation models. It used a self-assessment approach to evaluation using an agreed performance framework, questions and a facilitated process. While this was a consultative process, it was not a collaborative process. The evaluation of outcomes, conclusions and recommendations are those of IISD, based on consultations with members, related correspondence and five years of network documentation.