Monitoring and Evaluation

Monitoring & Evaluation (M&E) of adaptation aims to assess progress in implementing adaptation, looking at both process and outcomes. Process refers to achievements in terms of implementing adaptation policies, plans or actions, while outcomes are the changes that result from these achievements, usually in relation to communities, ecosystems or vulnerable groups (Price-Kelly, et al., 2015). For more information, please see the Guidebook for Developing National Adaptation Monitoring Systems. The guidance that follows is complementary to the guidebook, focusing on specific considerations for vertical integration in M&E. The purpose is to ensure that sub-national results and lessons learned are captured and integrated into ongoing decision making and future planning, and that national-level results and lessons learned are shared to inform planning and implementation at sub-national levels.

Figure 5 presents a potential model for vertical integration in M&E, which consists of linked M&E systems at national and sub-national levels. The overarching framework for M&E will most likely be developed at the national level, with appropriate institutional arrangements (orange arrow) to facilitate linkages with lower levels. Generally, this will include capacity development, such as guidance and technical assistance for establishment of M&E systems at the sub-national levels, as shown by the yellow arrow. The community-level system will ideally incorporate participatory M&E methods, including reflection on lessons learned. Locally generated information is fed to sub-national authorities, where it is aggregated and synthesized, together with sub-national monitoring data and lessons learned. As shown by the grey arrows, the national level M&E system captures an overview of the information generated at sub-national levels, along with information on national-level actions and learning. This is synthesized and communicated downwards to the sub-national levels to inform implementation and updates to plans. As in the previous dimensions, ongoing sharing of information on climate observations and projections, changes in the context and what works for adaptation at different levels is integral to the vertical integration process (shown by the red arrows). M&E informs both implementation of adaptation actions and reviews and updates to plans, in particular to enable increased integration of adaptation in development plans.

Figure 5. Vertical integration in M&E.

The following sections present guidance on integrating sub-national actions and learning into national M&E systems.

Enabling Factors for Vertical Integration in M&E

Have institutional mechanisms been established for linking national and sub-national M&E systems?

Mechanisms for linking national and sub-national M&E systems must be established from the outset, with actors from the different levels involved in developing the system, including identification of indicators and agreement on methodologies for data collection and analysis. It will be important to clearly delineate roles and responsibilities for adaptation M&E, identifying specific functions for particular actors at the different levels of application. This includes responsibilities for sharing monitoring information with other levels; for example, local organizations could take responsibility for conducting household surveys and communicating the data to sub-national authorities. Ideally the system for monitoring adaptation will build on existing M&E systems, such as those used to track development progress or implementation of sustainable natural resource management strategies.

D1: Monitoring the NAP process

How will ongoing generation and exchange of information between national and sub-national actors be facilitated?

Adaptation is an ongoing process of learning, testing, adjustment and integration of new information and knowledge. This process occurs at multiple levels, from the community level up to national policy-makers. As different actors learn about what works and what does not, it is important to ensure that these experiences are shared. Mechanisms for this type of information exchange often exist at the national level, for example through multisectoral working groups. You should consider how existing mechanisms can be complemented to ensure that information and knowledge are shared not just at the national level, but also among sub-national stakeholders and between sub-national and national actors. This can be linked to the M&E system, but goes beyond the collection and analysis of data to facilitate interpretation, reflection and co-generation of knowledge among different actors. Through this process, actors at different levels will increase their ability to integrate the lessons into policy and practice going forward.

D1: Monitoring the NAP process

How will capacity of sub-national actors to monitor and evaluate adaptation be built?

In addition to considering how adaptation will be monitored across different levels of implementation, you will also need to consider how capacity will be built to ensure that M&E frameworks and methods are applied consistently and systematically by the relevant actors. Sub-national actors may require capacity development in, for example, data collection and management in order to effectively monitor actions at their level and feed the information into the national M&E system in a way that allows for aggregation at national level. This investment in capacity must be viewed as a core part of the M&E system, and resources allocated to ensure that it occurs. Taking a collaborative approach to M&E, with actors at different levels coming together in dialogue, can be a very effective way to build capacity and ensure sharing of information across levels.

C3: Enhancing capacity for planning and implementing adaptation

Capturing Sub-national Results and Lessons Learned

What information is required from sub-national levels to illustrate national progress on adaptation?

Given that much of the implementation of adaptation actions will occur at sub-national levels, it is critical that these efforts are captured and incorporated in the national-level M&E system. As part of the design of the M&E system, you will need to consider what information is needed from sub-national levels (and in what format) in order to illustrate progress on NAP implementation. This will likely involve a mix of quantitative and qualitative information that demonstrates advancement in terms of both process and outcomes. The types of information required could include progress in developing community or sub-national government adaptation plans and/or integrating adaptation in local development plans, resources committed to adaptation efforts at the local level, evidence of increased capacity of sub-national actors to plan and implement adaptation and/or numbers of people reached by adaptation activities, for example.

D1: Monitoring the NAP process

How will information on sub-national adaptation processes and outcomes be collected, aggregated and synthesized?

Once you have decided what information is needed from sub-national levels, you will need to determine how it will be collected, aggregated and synthesized at the national level. This generally involves the identification of indicators to be monitored by sub-national actors, often linked to a broader set of questions designed to evaluate impact and capture learning from implementation. Sub-national metrics for adaptation must be designed taking into account the process of aggregation and synthesis. The simplest approach to aggregation is to use standardized indicators at different levels. However, this may obscure some of the context-specificity of local processes and outcomes. To address this, you may consider identifying key themes at the national level, and allowing sub-national actors to identify specific indicators within these themes. A less data-oriented approach and more learning-focused approach could involve synthesis of information from different levels, identifying common themes and lessons (Leiter, 2015). The right approach in your country will depend on the structure of your NAP and the information and knowledge you wish to obtain. Your M&E system will need to clearly define the indicators and questions to be answered at different levels, as well as how data will be collected, analyzed and transmitted to the national level for aggregation and synthesis.

D1: Monitoring the NAP process

“The NAP process emphasizes the need to integrate climate change adaptation into all relevant sectors and levels of government (i.e., both horizontally and vertically). Therefore, while an M&E system that includes monitoring a NAP process may apply at the national level and be coordinated by a national entity, it will likely need to aggregate informationfrom different sectors and/or scales.”

Price-Kelly et al., Guidebook for Developing National Adaptation Monitoring Systems (2015, p. 29)

How will sub-national experiences and learning be captured in updates to the NAP and related processes?

As the NAP is an iterative process, it is assumed that there will be reviews and updates over time. Ideally, these updates will be aligned with revisions and updates to key development plans and strategies and/or sectoral strategies for climate-sensitive sectors, to facilitate increasing integration of adaptation. From a vertical integration perspective, these updates must be informed by sub-national experiences and perspectives, recognizing that much of the implementation of adaptation actions will occur at these levels. This will likely involve synthesis of learning generated through the knowledge sharing mechanisms and engagement of sub-national actors in the update process. This iterative process also provides opportunities for enhancing the integration of adaptation into sub-national development planning.

D3: Iteratively updating the national adaptation plans

Ensuring Results and Lessons Learned Inform Sub-National Action

How will progress and lessons learned be used to inform ongoing adaptation processes?

The information generated by M&E systems is useful not only to demonstrate progress, but also to inform ongoing processes of planning, integrating and implementing adaptation. The dissemination of synthesized results and lessons learned to sub-national actors is important to ensure that they have the best information available on effective adaptation approaches, as well as things that haven’t achieved the desired results and why. This could include, for example, information on specific technologies or practices and how they have performed in different contexts and against different weather conditions, or on processes that have been effective in building adaptive capacity of local stakeholders. With this information, sub-national actors can adjust their implementation strategies accordingly and incorporate the learning into future planning.

D3: Iteratively updating the national adaptation plans

Morocco: Regional Adaptation M&E Systems

Morocco has taken a practical and iterative approach to developing regional adaptation M&E systems, beginning in three of the country’s 12 regions: Souss Massa Drâa, Marrakech Tensift Al Haouz and Tadla-Azilal. The participatory process involved several workshops where key stakeholders agreed a common vision for adaptation in the region and developed climate change impact and vulnerability chains for key sectors such as water, biodiversity and forests, agriculture and tourism. Based on this, indicators were selected through multistakeholder dialogues, including indicators to monitor both adaptation processes and outcomes.

Morocco’s regional environment and sustainable development monitoring systems (SIREDDs) are integrated information systems incorporating a number of modules addressing different issues. They are managed by the Regional Observatories on Environment and Sustainable Development that have been established in each region. The adaptation indicators have been integrated as a specific module (which also includes climate change mitigation, extreme events and damage and loss indicators). This module incorporates a database that captures a mix of standard indicators that will be aggregated at the national level and region-specific indicators which are tailored to the particular context of the region in question. The climate change module of the SIREDDs monitors changes in vulnerability in key sectors, as well as progress on implementing adaptation measures and the resultant impacts. In addition, it provides tools for decision makers, including a decision dashboard and spatial-temporal monitoring of key climate indicators (S. Maamouri, personal communication, September 21, 2016). Lessons learned on implementing adaptation are also analyzed (Hammill & Dekens, 2014). Some challenges have been encountered in rolling out these systems, notably the recent changes to the regional governance structure which have changed the territories covered by some regions and necessitated a redistribution of roles and responsibilities in relation to the SIREDDs. Despite this, establishment of the systems is progressing and expansion to the remaining regions is expected in the next couple of years.

At the same time, Morocco is moving forward in developing regional climate change plans in two of the three regions covered by the adaptation M&E systems, as part of its NAP process. The development of these plans has been informed by data from the SIREDD, which has provided a good basis for understanding hydro-meteorological hazards and climate vulnerabilities in the regions. When the plans are finalized, the climate change module of the SIREDD will be reviewed and the indicators will be adjusted or revised to ensure they are aligned with the objectives in the plans. Other next steps include the strengthening of linkages with the commune level (the lowest administrative structure in Morocco) and a system for compiling and communicating the information at the national level, including to report on adaptation progress against the NDC and in the annual State of the Environment reports (Y. Jaouhari, personal communication, August 11, 2016).