How do we “Track Progress” and “Enhance Ambition”?
The simple answer to this question is, indicators. From the moment you were born—when doctors took your temperature, assessed your blood pressure and monitored your oxygen levels, to your school years when you anxiously awaited exam results, to your annual performance rating from your boss—indicators have been a central part of your life. They are a snapshot in time, and if designed well, tell you where you are and if you are moving in the right direction.
Establishing a good set of indicators is easier said than done. Especially in the context of global monitoring of climate change mitigation efforts. The Paris Agreement establishes a general framework for bringing countries together as part of a global solution. It establishes processes for all countries to communicate their commitments, and to subsequently track progress. But the truth is, what we have now is a transparency framework to a framework agreement. Countries must still come together to write the detailed rules for how to report actions, monitor progress, and demonstrate that collectively we are on a trajectory to meet the overall temperature goals of the Paris Agreement.
This detailed rule writing is underway. At the May 2017 negotiations in Bonn, Parties drafted skeleton guidance outlines on the content of and accounting for nationally determined contributions (NDCs), as well as an enhanced transparency framework on reporting and reviews. Indicators were a key part of these discussions.
This rule writing step in the negotiation process is critical. In this post, I argue that we need to monitor progress in meeting the objectives of the Agreement through the reporting and use of well-defined indicators (e.g., total GHG emissions, GHG emissions per GDP or per capita, technology penetration rates). I discuss approaches to generate indicators and ways to use them. And finally, GHGMI seeks your help. We request your expert input on the questions raised in this post, which will be included in a submission by GHGMI to the UNFCCC negotiation process on this topic.
So, let’s dive in.
What is an indicator?
According to the Organization for Economic Cooperation and Development (OECD), an indicator is a quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor. Indicators are relevant to many fields. Indicators are at the core of all types of monitoring and evaluation systems, whether for the economy, the environment, children’s literacy, or any other field. They can measure physical, financial, social, environmental or other dimensions. Well-designed indicators can serve as a key part of an evidence-base for a strong accountability system.
But what makes a good indicator? The literature[1] outlines numerous criteria. A common theme is that a good indicator is SMART: specific, measurable, achievable, reliable and timely (Table 1).
Table 1. Common Criteria for What Makes a Good Indicator
Why does the Paris Agreement need indicators?
One might reasonably ask whether the Paris Agreement needs indicators. Nearly all countries have published an NDC describing their intended contribution to global climate mitigation efforts. The Agreement requires all countries submit a national GHG inventory on a regular basis (Article 13.7(a)).
Isn’t this enough?
The short answer, is no. Without indicators, the international community can set up some sort of review committee and judge the information provided by a country and make determinations based on whatever quality, detail, and format of information the country provides. But, such a non-standardized process becomes highly subjective, and the assessment moves out of the technical realm into a more political one. A focus on more objective and standardized indicators protects the Paris Agreement from unnecessary criticism and makes the reporting and compliance process far more efficient (versus ad hoc and burdensome because each national review must be designed and judged). Objective, semi-standardized indicators also help drive higher ambition because they facilitate comparative analysis across countries and over time.
Unlike Kyoto Protocol commitments, NDCs come in a wide diversity of forms. Many are not in terms of total national GHG emissions, but are instead based on other structures such as specific policies, sectors, or intensity rates. So, national GHG inventories alone (as currently submitted) are not sufficient to assess most NDCs. The Agreement recognizes this problem. It requires each country not only to submit a national GHG inventory, but to also regularly provide “information necessary to track progress made in implementing and achieving its nationally determined contribution” (Article 13.7(b)) and to “account for their NDC” (Article 4.13). Yet, the Agreement is silent on how this tracking and accounting is to be performed, as well as how it is to be reviewed for compliance. More specifically, the following two factors create new challenges for tracking compliance under the Paris Agreement:
- Each country defines its own NDC.
Countries have full flexibility to elaborate their specific contribution to the global effort that is appropriate for their national circumstances. And, they retain the flexibility to change how they specify their NDC over time. In the initial round of NDC submissions, countries proposed a variety of types of NDCs, with some countries proposing more than one type of target:
- an absolute reduction below a base year (either economy-wide or sector-specific),
- a reduction from a business-as-usual (BAU) scenario,
- a reduction below an intensity level (e.g., GHG emissions per unit of GDP or per capita),
- a peaking target,
- a collection of policies and/or measures, and
- other (e.g., carbon neutrality target).
While the production of an annual or biennial GHG inventory could provide evidence that a country is on track to meet an absolute target (the heart of the compliance regime under the Kyoto Protocol), it may not necessarily provide the information needed to assess whether specific sector policies are being implemented. National inventories, as currently structured, do not include information on projections to assess a peaking target, and do not generally include sector-or category-specific intensity levels. Something more than a regular inventory submission is needed to demonstrate compliance with the diversity of existing NDCs.
- There are no IPCC Guidelines for NDCs.
This point may seem obvious, but the challenge it introduces shouldn’t be understated. We are moving from a world where UNFCCC reporting and review activities were relatively standardized, sometimes at the expense of flexibility, to one where the flexibility of a country to define their commitment is of paramount importance.
Under the Kyoto Protocol, each developed country had an absolute emissions target. They submitted a GHG inventory on an annual basis following the UNFCCC reporting guidelines and used agreed methods from the Intergovernmental Panel on Climate Change (IPCC) to estimate and account for emissions. The use of the scientifically rigorous IPCC methods, along with a robust set of reporting guidelines and decisions, provided uniform and standardized national reporting of GHG emissions. Given this uniformity, reviewers have been readily able to assess whether national reporting is in keeping with these UNFCCC and IPCC guidelines.
National reporting has recently been expanded to include biennial reports and biennial update reports, for developing and developed countries, respectively. The content of these new biennial submissions includes information on national circumstances, policies and measures, and projections. The reporting requirements became a bit more flexible (emphasizing more what a country “should” do then what they “shall” do). There is currently nothing akin to the 2006 IPCC Guidelines on how to analyze and report data in these new submissions.
Under the Paris Agreement, reviewers are going to be asked to assess a country’s progress, and to do this, they will need a framework, some set of indicators, to be able to objectively assess whether the country is on target to meet its goal.
And do not forget, a country’s obligations under the Agreement extend beyond mitigation. Indicators are needed to track progress across mitigation, adaptation, finance, technology transfer and capacity building activities (see Table 2).
Table 2. Paris Agreement Articles that Contain Outcomes that Warrant Indicator Sets
Approaches for generating indicators for the Paris Agreement
How should negotiators address questions about the type and format of information countries report to demonstrate progress, and how it is reviewed? The use of SMART indicators within NDCs should be a pillar of these discussions.
The ability of countries to come together in Paris was thanks to the flexibility offered all countries to take actions appropriate for their national circumstances. As we design the international framework for monitoring implementation of the Agreement, that flexibility should be respected. But, safeguards must also be put in place so that both individual country commitments and the overall objective of the Agreement are met. Well-designed indicators can and should be a core part of this safeguard.
The following five approaches are options we have for generating indicators (see Figure 1):
Figure 1. Possible Elements of Indicator Framework for Paris Agreement
- Development of UN decision language to require reporting of minimum indicators.
A minimum indicator set should be regularly reported by all Parties, regardless of the type of NDC a country communicated (e.g., absolute target, peaking target, or an intensity-based target). This fundamental set of indicators ensures that the international community has the information it needs to assess global progress towards achieving the overall temperature goal (Article 14.1).
In addition to indicators reported by all Parties, decision language could specify indicators that must be provided if a specific type of NDC is used by a country, for example: current and projected emissions/GDP or per capita for intensity-based targets, or the penetration of technologies or percentage of the population affected by a policy in the case of a policy and measure target. A review of Parties’ formal submissions[2] prior to the May 2017 negotiations provide suggestions from developed and developing countries on the type of information that should be reported in NDCs (e.g., baseline projections, GDP trends, BAU baseline, intensity trends, etc.) (Table 3)
Table 3. Minimum Indicators for Tracking Implementation of NDCs
Table References: UNFCCC aggregate information report, IEA 2016 (Enhancing transparency of climate change mitigation under the Paris Agreement: lessons from experience. Climate Change Expert Group. Paper No. 2016(4). Formal submissions by Parties to the UNFCCC.
The challenge with relying solely on decision language to develop an indicator framework is the race to the bottom. In order for agreement to be reached, all countries would have to agree on these common indicators, which results in either no agreement, or inclusion of only the most basic of indicators. While it is helpful to have the reporting of some indicators based in decision language, this likely won’t be sufficient to fully track progress.
- Country-driven indicators.
The full extent of the diversity of future NDCs is difficult to predict. Therefore, it is even more difficult to propose prescriptive indicators in advance. Countries should be encouraged or, for some NDC types, required to define, justify, and report on objective country-specific indicators. The process for developing country-driven indicator sets will further encourage the improvement in capacity to further understand mitigation drivers and achievement or failure of intended outcomes. This learning-by-doing and experimental approach is particularly relevant for NDCs that are a collection of policies and/or measures. The mitigation impacts on any one policy, or even a group of policies, may not be transparently expressed in the national GHG inventory, as there are so many activity drivers affecting national emissions and policies can have overlapping or reinforcing effects.[3]
With country-driven indicators, it will be important for countries to consider the SMART principles above when defining the indicator to ensure it tracks the desired outcome. It is also important for countries to provide clarifying information when reporting the indicator (e.g., definitions, data sources, assumptions) so others understand the scope and relevance of the indicator. Without such information, country-reported indicators may fail to deliver.
- International third-party report.
Data collection to support indicator reporting can be resource intensive and therefore challenging for many countries to implement, particularly in the early years of the Paris Agreement. Fortunately, some relevant indicators are already tracked under the auspices of other international agreements such as the 2030 Agenda for Sustainable Development, by other international organizations, such as the International Energy Agency (IEA) and the OECD, and by national and regional statistical agencies. The World Bank also publishes the World Development Indicators and an Atlas of Sustainable Development Goals (SDGs). Although there are some useful indicators in these reports for tracking economic progress, overall CO2 emissions, deforestation, and energy efficiency improvements, they were not developed with consideration of the breadth of reporting under the Paris Agreement. A third party could build on these indicators, identify additional indicators, and produce a State of Implementation of the Paris Agreement to paint a picture of the global progress made. The collection and publication of Paris Agreement relevant indicators on a regular basis by such a third-party organization could provide an objective assessment, enhance global confidence in the Agreement, and drive overall ambition by giving countries assurances that their neighbors are doing their part.
A key challenge to this option is resources. Who would lead such an effort? It would likely require a large multi-stakeholder effort to review existing indicators to see what is available and where gaps exist. Although, the challenge is not insurmountable.
The next two processes address what information should be reviewed or assessed, rather than what types of indicators should be reported. To fully develop an indicator framework, it is important to consider not only what is reported, but the review of that information and how indicators can be modified and improved over time. Are the selected indicators reliable enough to provide reviewers evidence that the Party either is, or is not, on track to meet its target? Can the country provide the information to report the indicator in a timely manner? Are new data sources required for the country to apply the indicators?
- Multilateral assessment.
Under Article 13.11 of the Paris Agreement, the information submitted by Parties will undergo a technical review, as well as a multilateral consideration of progress. Assuming that the process of multilateral consideration of progress will share at least some common elements with the current processes of Multilateral Assessment and International Consultation and Analysis for Annex I and non-Annex I Parties, respectively, these international questions and answer processes will provide a valuable opportunity for other countries to confirm, or question and challenge, the specific indicators chosen by each country to measure progress.
- Regional peer reviews.
Existing regional groups can support implementation and enhancement of indicators for the Paris Agreement. For example, the OECD has been reviewing environmental performance in member countries since the early 1990’s. The African Peer Review Mechanism was established in the early 2000’s as a forum for sharing experiences and best practices and identifying capacity building needs as countries in the region strive for overall sustainable development and regional integration. Such existing, or newly formed, regional groups could play a contributing support role for countries as they implement the Paris Agreement, by identifying regionally relevant indicators and by pooling resources to collect data for indicators.
These five processes are not mutually exclusive, and the indicator framework under the Paris Agreement will include one or more of, if not all, of them. In all cases, it is important to remember that the challenge of designing, implementing and assessing indicators to track a country’s progress against a goal is not new in international diplomacy and treaties. Objective and verifiable indicators are sought after in many regimes,[4],[5],[6] and either formal or informal cooperation should be sought with these other organizations early and often. It is worth reviewing the lessons learned from these other groups, including the 2030 Agenda for Sustainable Development, the IEA, the OECD, and the Sustainable Development Solutions Network. We can also examine further corporate initiatives to gain a more financial insight, such as the International Finance Corporation. We should consider what role the Climate Change Experts Group could play in this arena, as well as the IPCC. The global community could build on these, and other, efforts and develop a gap analysis to assess what is already being collected on a regular basis and what is a known gap. We could also strive to understand and respond to the technical and political challenges faced by countries in other ongoing indicator efforts.
Next Steps
The international community came together in late 2015 with a common goal, recognizing the importance of limiting the global temperature to avoid the worse impacts of climate change. In late 2016 and early 2017, they reconvened to begin drafting the rulebook for how to track and report progress towards their national commitments. Time is short. The rulebook is to be developed and agreed by COP24 in late 2018. Countries will also likely start drafting their 2nd NDC starting in late 2018.
Without clear, well-defined indicators, it will be difficult to know if the Paris Agreement is being effectively implemented. It is likely that we will have no reliable way to know if a country is on track to meet its contribution, or if it ever does meet its goal. It will be ambiguous if the second NDC of a country reflects higher ambition than the first. We will not be able to conclude if the global community really has kept emissions below a threshold level.
And this is where you come in.
Through GHGMI, you can have a direct input to this critical negotiation process. Like with our hosting of the world’s largest workgroup on ISO standards, GHGMI gives a voice to the global community of GHG experts and practitioners. As part of the Paris Agreement negotiations, we have been asked to submit views on several topics where indicators could play a prominent role moving forward. Two topics that are of particular interest to GHGMI are on:
- What types of information should be provided in NDCs to facilitate clarity, transparency and understanding, as well as to support accounting of NDCs, and
- The modalities, procedures and guidelines for the transparency framework (Table 4).
Table 4. Upcoming Submissions Related to Indicators
GHGMI will submit its views on one or both of these topics, and we offer you an opportunity to influence the process with your recommendations and expertise.
Please, click here to access the online survey.
This survey is designed to solicit your feedback on the points raised in this post. We welcome your comments, proposals, and critiques. GHGMI will then consider the collective experience from you and others in our extensive global network of experts when answering the UNFCCC’s call for views. Please provide your feedback by 15 August 2017.
Acknowledgements: The author would like to thank Michael Gillenwater, Molly White and Kathryn Goldman for offering their expertise and insight during the drafting of this post.
[1] The literature is rich; some examples are: http://www.oecd.org/dataoecd/7/47/24993546.pdf, http://pdf.wri.org/environmentalindicators_bw.pdf, https://www.nap.edu/read/10404/chapter/5#23, https://publications.iadb.org/bitstream/handle/11319/5447/Indicators%20to%20Asses%20the%20Effectiveness%20of%20Climate%20Change%20Projects.pdf, http://www.rscproject.org/indicators/index.php?page=what-are-the-characteristics-of-a-good-indicators-or-indicator-sets, https://rapidbi.com/history-of-smart-objectives/
[2] The submissions referenced were due on 31 March 2017 and are can be found under “Item 3: ‘Further guidance in relation to the mitigation section of decision 1/CP.21 on: (a) features of nationally determined contributions, as specified in paragraph 26; (b) information to facilitate clarity, transparency and understanding of nationally determined contributions, as specified in paragraph 28; and (c) accounting for Parties’ nationally determined contributions, as specified in paragraph 31’.
[3] World Resources Institute. 2014. Greenhouse Gas Protocol: Policy and Action Standard, section 3.2.11.
[4] The Inter-Agency and Expert Group on Sustainable Development Goal Indicators under the 2030 Agenda for Sustainable Development, available online.
[5] Global indicators developed to be monitor implementation of the Busan Partnership for Effective Development Cooperation, available online.
[6] Sustainable Development Solutions Network. 2016. Indicators and a Monitoring Framework for FfD: Proposals for Follow-up and Review of the Addis Ababa Action Agenda, available online.
Leave a Reply