Additionality is assessed against a counterfactual. True or False?

August 24, 2012, by Michael Gillenwater

Continuing on the theme of widely held fundamental misconceptions in the carbon management community (see previous blog posts here and here), today I am going to write on a matter of terminology I find particularly irksome: the use of the term “counterfactual” in additionality discussions.

Probably one of the most frequently cited assertions relating to additionality is that it cannot be proven because it is assessed against a “counterfactual” baseline. Well, I am here to tell you that this line of argumentation is flawed. Certainly, I admit there is a lot that can and should be improved with how additionality and baselines are assessed under all the existing GHG offset programs. (I have previously elaborated on some potential areas for improvement here.). However, the use of “counterfactual” implies that you cannot prove additionality and that it is a completely unreliable determination. This is a mischaracterization of the actual issue.

In my writings, I have previously argued that the term “counterfactual” has been misconstrued when discussing baselines or additionality. A counterfactual is defined as something that is contrary to the facts or not reflecting or considering relevant facts. But it is feasible to ground the models we use for assessing baselines and additionality using good science and proper causal inference techniques. In other words, observations (i.e., facts) from related cases or experiments can be used to develop models that predict behaviors in similar or identical situations. Assuming the models that produce these predictions are based on rigorous studies, I would argue that the outcomes are not well described as being “contrary to relevant facts.” These models are unlikely to be perfect representations of the case being assessed, but, unless conducted with no consideration of good causal inference methodologies, the assessment of additionality can still be based on observed facts from similar cases and other empirical research. As a consequence, I have previously recommended that the term “unobserved” be used rather than “counterfactual” to describe the baseline used for assessing additionality.

But, my thinking has now gone further. I no longer think that “counterfactual” is simply a misconstrued or misused term for talking about additionality. It is actually flat out WRONG.

Sweating the small stuff

It may seem trivial to work up a sweat over semantics. But in my opinion if there was ever a topic requiring precision and care in how we talk about and conceptualize it, it is additionality and baselines in the context of offsets.

For a proposed project or class of similar project activities, additionality is assessed relative to a predicted baseline, which represents a scenario under identical conditions except for the absence of the recognized intervention created by the offset program (e.g., such as the price signal from the potential to earn offset credits). Although it may be possible to observe the behavior of an actor under the influence of an intervention and another similar actor under near identical circumstances where the intervention is absent, it is rarely possible to simultaneously observe the behavior of the same actor under the same conditions both with and without the intervention present.

In a typical GHG offset project cycle, the assessment of additionality follows the development of a predictive baseline model. This assessment occurs at the proposal stage, prior to a project’s implementation. Therefore the baseline for this purpose is not a backward looking counterfactual but instead a forward-looking prediction. Although additionality is characterized as an assessment against a counterfactual baseline throughout the literature on CDM and other GHG offset programs, it remains that something that is set in the future and cannot accurately be considered a counterfactual. In other words, at the time it is actually conducted, additionality is not a counterfactual assessment, but rather it is a prediction.

In contrast to the forward-looking additionality assessment, a GHG offset project’s emission reductions are calculated against a backward-looking counterfactual baseline.

The simple lesson here is: stop calling additionality a counterfactual. It is not.

Baseline bingo

So now that we have identified two applications for baselines (i.e., assessing additionality ex ante and calculating emission reductions ex post), we can think about another question: for a given project, should the two baselines used for assessing additionality and calculating emission reductions be the same and what are the implications of them being different?

[Warning: the following discussion is probably best treated as an advanced topic for more intellectually ambitious readers. So, just to be clear, you have been warned.]

First, it is critical to understand that if a project proponent lacks certainty in the baseline that will be used for determining additionality and calculating emission reductions for their proposed project, this uncertainty will affect their perception of the strength of the offset program intervention. In other words, the more uncertainty in the baseline that a project proponent perceives, the less likely the project proponent is to change their behavior and as a consequence their proposed project is less likely to be additional.

Beyond having this negative feedback on the perceived strength of the offset program intervention, this uncertainty will also lead to one of a number of undesirable outcomes that will reduce the credibility of the overall process. The table below summarizes these outcomes. In the table, A/BL is the baseline used for assessing additionality and ER/BL is the baseline used for calculating emission reductions. Actual is true baseline, which we do not observe, yet it does exist, theoretically. As shown in the table, not maintaining consistency between A/BL and ER/BL effectively assures that there will be a higher error rate in additionality determinations and/or crediting relative to setting them equal.

Table: Possible outcomes with respect to stringency of baselines for additionality determination (A/BL) and emission reduction crediting (ER/BL).

possible

So although we could play games with baselines to try and make them more conservative, thinking we are improving the environmental integrity of the offset program, these efforts are likely counterproductive (from a game theory perspective). Project developers, knowing that their ability to earn offset credits has been altered, will then simply alter their behavior accordingly. Instead we should focus our efforts on coming up with the best baseline prediction we can (i.e., getting our prediction as close to actual as practical) and using that single baseline for both ex ante and ex post applications.


8 responses to “Additionality is assessed against a counterfactual. True or False?”

  1. Mike Wara says:

    Michael,

    A very thoughtful post.

    Great point about the terminology – I’m not sure it changes the substance of typical concerns about long-term baselines, but your point is a good one.

    I’m confused about the baseline bingo discussion/table: doesn’t the table assume that A/BL and ER/BL are precisely specified and known by all participants? There is of course some uncertainty about Actual but this is intrinsic. So how does the design of the program with respect to stringency related to uncertainty? I would take your discussion of uncertainty (second to last paragraph) to mean that we should avoid contexts/sectors/project types where there is significant uncertainty about the Actual Baseline – a point with which I absolutely agree.

  2. Adam says:

    Dear Michael,

    To paraphrase The Princess Bride: I do not think these words mean what you think they mean! Your definition of counterfactual is the common usage (an adjective), but the term is properly used in this context as a term-of-art in formal logic (and a noun, to boot). The formal logic definition happens to correspond exactly to how you’ve just described the additionality calculation (save for the matter of tense, but that’s inconsequential in the world of formal logic).

    If you’re interested, the Stanford Encyclopedia of philosophy has an excellent write-up on the topic: http://plato.stanford.edu/entries/causation-counterfactual/. Should you consult it, I think you’ll agree that you really are talking about counterfactuals in additionality calculations.

  3. Pierre says:

    Hi Michael,

    Thanks for the terminology discussion related to additionality. Couple of questions and comments on your essay. First comment is that “project developers” are often not the owner/operator of a source, hence their behavior is not at issue with the additionality test. They will have a harder time selling the project to the owner/operator if greater uncertainty exists with either passing an additionality criterion or expected flow of project revenues. Second comment is that I question the “negative” feedback you describe. If greater uncertainty is perceived by either project developers and/or the owner/operators of an emission source, wouldn’t that make business-as-usual (BAU) behavior more likely…and hence any offset revenue from a project that reduces emissions below BAU – more likely to be “additional” than without that level of uncertainty in the baseline(s). Finally, a question related to your theoretical “actual” in the table…I’m not clear on what it means for actual emissions to be “more stringent” or “less stringent” – I see actual emissions being just that…some cardinal figure that is what it is (not more or less, stringent).

  4. Derik says:

    I’m confused!

    It sounds like you’re saying:
    1. additionality baselines are not counterfactual.
    2. ER baselines ARE counterfactual.
    3. But both A/BLs and ER/BLs should be set equal, and as close as possible to the “actual” baseline.

    Question 1: is the “actual” baseline counterfactual or not counterfactual?

    Question 2: how can you set a “forward-looking prediction” and a “backward-looking counterfactual” to be equal to each other (unless they are, in fact, the same thing determined in the same way)?

    To my thinking, the actual baseline is indeed a counterfactual (per Adam’s comment above) and logically both the A/BL and ER/BL are as well. But maybe I’m missing something.

  5. It seems the more esoteric these blogs get the more commentary they produce.

    For Derik, I will admit that the table is tough to describe. And I will not pretend to done so optimally in a quick blog post. To your first question the key issue is that when assessing additionality, the baseline is a prediction. I view counterfactuals being in the realm of past tense. Otherwise we have to think of every prediction or forecast as a counterfactual. Even after having gone through Adam’s philosophy link (which is great by the way…although I did consult a logic textbook prior to writing this blog), I did not see anything that changed my view here. I agree that from a formal logic standpoint these are similar. But, I did not argue that this distinction between prediction and ex post counterfactual has logical analysis implications.

    But words do matter, and the term “counterfactual” has been used as a linguistic bludgeon to discredit additionality assessments. So my main point is that you can criticize the process and/or need for assessing additionality, but then at least lets make it clear what we are doing. We are making a prediction of future behavior. What will a project developer do (future tense). Maybe that is equally discreditable.

    To Derik’s second question, the way you make them equal is I think pretty obvious. You just have one process done ex ante and then use the same baseline for crediting ex post (as is generally done now). There are not two processes. The impetus for this blog post was, in part, proposals by some in the community to have two separate processes. Another alternative is to update the baseline after the project is implemented when you know more about the state world than you did ex ante.

    The more general question is whether to use two different baselines. One for assessing additionality and then a different one for crediting, as is done with the cement industry standardized approach and is often recommended by those hoping to come up with some magic formula to improve the credibility of offset programs (e.g., by making one or the other baseline more “conservative”). The point I’m trying to make with the table is that these attempts do not necessarily achieve the hoped ends.

    For Pierre, good point. I use the term project developers as referring to those making the decisions with respect to the proposed project. I am open to another term. We can call them project investors, but then some investors are silent on actual decision making as well. We can call them project investors and developers if that is helpful. On uncertainty, I am not sure where we are getting crossed up. The point is that with a standardized approach you are trying to predict where and how much the intervention “signal” is going to effect behavior. That signal is a function magnitude and probability/uncertainty. If the uncertainty in the signal increases then the overall signal strength (the project developer’s perceived strength of the intervention) goes down. And from the perspective of the entity developing the standardized approach, the weaker the signal the harder to separate the signals effect from the noise.

    In the table, I will admit it is a little confusing. I welcome ideas on how to present it better. A colleague recommended doing it graphically, but I just have not gotten around to playing with it. The actual represents where the actual baseline is relative to the two assumed baselines. You can think of it as a cardinal value if that is helpful.

    For Michael, I think you are probably correct (although I am not sure, because I have considered all the potential implications) that in terms of the mechanics of the additionality assessment process it does not change much. But given the depth of confusion on the topic of additionality and baselines, I find there is an almost endless need to find ways to explain it better and more precisely. And how well we explain and understand it does have significant policy repercussions.

    I would agree that where there is uncertainty in the actual baseline above some threshold that these classes of projects are not well-suited for an offset mechanism.

    As for the table, see if my comments above help. Again, I agree that its it not the perfect way to communicate the point…which is relates to the discussion in the debates over standardized approaches that advocate using different baselines for assessing additionality and issuing credits. My point is that doing so does not appear to buy you anything.

    More discussion and debate is welcome. I don’t pretend to having all the answers here.

    Thanks
    mg

  6. Derik says:

    “Otherwise we have to think of every prediction or forecast as a counterfactual.”

    …but isn’t the whole point of additionality to make a prediction about what would happen *in the absence* of a specified intervention (per your articles)? Doesn’t that make it an explicitly counterfactual prediction, as opposed to a standard prediction about what actually will happen in the case of the intervention (esp. if that intervention has already taken place)?

  7. Derik,

    I was wondering if someone would pick up on that point. Nice work! At least I am starting to affect your thinking.

    Your very good point is something I sidestepped in my short blog. It was one more layer down in complexity that I thought was getting too deep. But now that you brought it up, lets go there.

    Whether there is an element of counterfactual analysis really relates to how you conceive of the intervention. If it is, as you and Mark Trexler have described previous, as the creation of the offset market, then we are talking about the intervention requiring us to construct an alternate ahistorical world. In other words, a counterfactual, and then predict that alternate world into the future for the project developer(s)/investor(s) in question. So this is a combination of ex post counerfactual and ex ante prediction. However, if we take history as a given and only conceive of the intervention as something, like expectations around a price signal, then we can still use the present real world as our starting point (no ex post counterfactual) and only project forward what would happen in the absence of the intervention.

    As I pointed out in my papers on additionalty, it matters greatly how we conceive of the intervention because it has huge implications for the assessment challenge. If we can avoid defining the intervention in a way that requires we “rewrite” history, then that will make our lives much easier. For example, those that argue additionality should be based on what people would do in the absence of climate change happening are really asking us to construct such a totally different virtual world that it starts to get ridiculous. How could I possibly predict what a project developer today would do if the last 150 years of climate change and science around it had never happened? Doing the same thing as if the carbon market had never developed in the first place is only slightly less impractical.

    mg

  8. Derik says:

    Michael,

    Thanks. I’ll have to put that in my pipe and smoke it for a while. 🙂

    Derik

Leave a Reply

Your email address will not be published. Required fields are marked *