Making Policy in an Uncertain World
In one of the best commencement speeches ever, the Australian composer, lyricist, and comedian, Tim Minchin told students at University of Western Australia, “We must think critically and not just about the ideas of others. Be hard on your beliefs. Take them out onto the veranda and hit them with a cricket bat.”
I was reminded of that advice when I participated in the San Francisco March for Science back in April. There were great signs and inspiring speakers, but on many topics — including climate change — there was also no shortage of strong claims made by people who were not following Minchin’s rule: claims that we could easily switch to 100% renewable energy, that such a switch would “create jobs”, that nuclear power must not (or must) be part of a climate strategy, and that there is a known threshold of atmospheric CO2 concentration beyond which catastrophic effects would follow.
None of these claims stands up to close scrutiny. Not that they are clearly wrong, but that they are not supported by existing research. Working on the economics of energy and climate change, similar wince-inducing statements of “fact” are everywhere:
- The CEO of a multinational oil company who claims that “the market” will solve climate change without government regulation. Evidence: cheaper natural gas from fracking has reduced coal burning and lowered GHGs. Problem: Even if all coal were displaced by natural gas, GHG levels would continue to rise. And what if the market finds a much cheaper way to extract coal (as it did in the mid-1900s with mountaintop mining). There is no market for a pollution externality, which is why the market can’t solve the problem.
- The leaders of some environmental groups who have argued that restricting refinery GHG emissions will prevent asthma in the surrounding neighborhoods, even though CO2 and methane (the major GHGs) don’t cause asthma. Evidence: Local pollutants that do cause asthma are correlated with GHG emissions. Problem: They are also correlated with the presence of freeways and heavy trucks, low-quality housing, and poor healthcare.
Such advocacy presented as “facts” raises at least three different concerns:
The first one is obvious, a simple correlate of Minchin’s rule: if you only believe in science when it supports your prior position, you don’t believe in science. If research doesn’t regularly cause you to adjust your beliefs, you either are omniscient (nope, I checked, you aren’t), or you are not really interested in evidence-based arguments.
The second concern is perhaps more subtle, or at least frequently lost in media coverage: One research study almost never establishes a fact. That’s not how knowledge advances. Rather, new research contributes to the larger body of knowledge on a subject and changes the probability of something being true. Over time, as research confirms or conflicts with existing beliefs, our understanding evolves closer to certainty, that is, facts.
Believing in science (or economics, or any empirical inquiry) means taking those probabilities seriously even when we are well short of certainty; not asserting that unlikely conclusions hold simply because they haven’t been 100% ruled out; and not asserting that likely conclusions are already indisputable “facts.”
For a few horrible years, we had no idea what caused AIDS. Then some scientists linked it to HIV, and now with more information experts view the probability that HIV causes AIDS as near certainty. But the recognition of a high probability in the 1980s made it possible to harness research and public policy to create effective AIDS treatments before near certainty had been established. Today it has even led to an early-stage vaccine. Waiting until there was certainty on the HIV-AIDS link before making policy would have permitted many more tragic deaths than have occurred.
Which leads to my third point: Every decision to take action or not is made under uncertainty. We are constantly trading off the costs and benefits of waiting for additional information. If you wait for near certainty on any important issue, you have almost surely waited too long, because the costs of waiting are seldom trivial.
If you or a loved one has faced cancer, you know this. You know that the treatment options are all about probabilities. New information on the effectiveness and the side effects of treatment options arrives every year, and preferred treatments change. But when the diagnosis arrives, you can’t wait until you are certain of how you will respond to each possible treatment before making a decision.
Which connects back to my first point. Even after we have made a decision, we have to be open to new information that suggests it’s not the right decision. It’s way too easy to become committed to the view that the cost of energy storage technology will inexorably decline, or nuclear power is (or isn’t) the cheapest way to reduce GHGs, or intermittent renewables will (or won’t) be costly to integrate into the grid. It’s hard not to want your best (or most optimistic) guesses to be fulfilled, so much so that we ignore conflicting evidence (as famously led to the explosion of the Shuttle Challenger). But as new evidence appears, we have to be willing to reconsider the best path forward.
Obviously the basic discussion of the existence and cause of climate change has been a victim of these three points. Early on, a small group of the population locked into the view that it wasn’t real, or that we weren’t certain it’s real, and ever since they have been selecting data and studies to confirm their belief. On the other side, though, there are the anti-science claims that every weather catastrophe is caused by climate change, that we must support every possible carbon reducing technology no matter how expensive or implausible, and that we know with great certainty the exact relationship between atmospheric CO2 concentrations and climate change.
These black and white views of climate change make better bumper stickers than, say, “keep the 95% confidence interval of temperature change below 2 degrees Celsius”. I get that. Making the effects of climate change salient to the average voter is critical to mobilizing support for action. But when climate activists make unsupported claims of certainty about the damage or low-cost solutions, serious researchers get queasy and start to back away.
And such claims that stretch beyond the research often end up backfiring. Assertions based on cherry-picked or misinterpreted data get discredited; overly-confident predictions fail to materialize; or advocates reject new findings simply because they don’t support their world view. Some in the do-nothing camp compile these biased analyses as evidence that all research on climate change is flawed and misleading.
So, how do we stay true to real scientific method – openness to new research results, recognition of uncertainty, and understanding that imperfect knowledge shouldn’t be an excuse for inaction – while still communicating the (high probability) seriousness of the climate change challenge and building political support? I don’t have the answer, but I think we need a more serious discussion. And we need to spend more time swinging cricket bats at our own ideas.
Categories
Severin Borenstein View All
Severin Borenstein is Professor of the Graduate School in the Economic Analysis and Policy Group at the Haas School of Business and Faculty Director of the Energy Institute at Haas. He received his A.B. from U.C. Berkeley and Ph.D. in Economics from M.I.T. His research focuses on the economics of renewable energy, economic policies for reducing greenhouse gases, and alternative models of retail electricity pricing. Borenstein is also a research associate of the National Bureau of Economic Research in Cambridge, MA. He served on the Board of Governors of the California Power Exchange from 1997 to 2003. During 1999-2000, he was a member of the California Attorney General's Gasoline Price Task Force. In 2012-13, he served on the Emissions Market Assessment Committee, which advised the California Air Resources Board on the operation of California’s Cap and Trade market for greenhouse gases. In 2014, he was appointed to the California Energy Commission’s Petroleum Market Advisory Committee, which he chaired from 2015 until the Committee was dissolved in 2017. From 2015-2020, he served on the Advisory Council of the Bay Area Air Quality Management District. Since 2019, he has been a member of the Governing Board of the California Independent System Operator.
Energy Institute Blog 
This piece could not be more timely. The paucity of credible independent climate policy research is a basic shortcoming that has to be addressed before one can expect inculcation and improved policies to take place.
The lack of independent research and evaluation of climate policies and programs leaves legislators and policy makers on thin ice. This is particularly applicable for cap and trade and regulatory programs as well. The ARB Scoping Plan process and LAO work drawing from it were revealing in their shaping and steering so all roads led to cap and trade. (Despite the LAO finding that “Different aspects of the cap-and-trade program have been the subject of much research and analysis. However, to our knowledge, a robust study of the overall statewide effects of the cap-and-trade program so far has not been conducted.” They go on to discuss the complexity of unraveling the impacts of cap and trade vs. the 10+ regulatory programs.)
The commenter in this thread claiming cap and trade is a success has little basis for this in California and even less in the decade of EU Emissions Trading System (EUETS) and New England’s Regional Greenhouse Gas Initiative (RGGI). Causality seems not to be of interest. While the vested interests around the cap and trade programs sponsor positive reports, they often focus on the “market” working (i.e., trading occurs, no abuses — see Potomac Economics RGGI market monitor reports) or the resulting revenues spent on programs. It is hard to make a sound case that chronic low allowance prices (RGGI–now under $3/ton; EU ETS well under $10/ton) and allowance surpluses forecast years out has led to significant clean energy capital investment. The price of Kyoto offsets purchased for years by EU covered industries of around 25 cents/ton CO2 makes it even harder.
Maybe it is a good time to get a serious independent assessment of the major cap and trade programs that goes beyond “cap and trade started, emissions went down, therefore cap and trade works.” The rooster didn’t make the sun rise.
I’d welcome those believing cap and trade is a success to provide the analytical basis for their conclusion of causality.
According to Amos Tversky, scientists, like other humans, do not abide by either Minchin’s Rule or its Corollary. Rather they are prone to “hindsight bias,” the tendency impute more certainty to predictions than is warranted by explanations of the past. This seems particularly germane regarding computer models of climate change. Or as von Neumann put it, “with four parameters I can fit an elephant, and with five I can make him wiggle his trunk.”
Severin, at the San Francisco Science Fair (at the terminus of the San Francisco March for Science), in the booth of Californians for Green Nuclear Power, I and other members violated Minchin’s Rule by making the point “nuclear power must…be part of a climate strategy.” We spoke with an estimated 500 attendees, including antinuclear activists, physicists, nuclear engineers, biologists, economists – scientists of every stripe – and we always began by asking questions.
There are a couple of benefits to greeting people in this manner: 1) it accords your audience implicit respect for their opinion before they’ve said a word, and 2) it forces you to consider their opinion before your own. We ended up changing minds, both theirs and ours.
What did not change was our conviction. In fact, during the course of our discussions we found it has a fundamental basis which is inarguable – one that has nothing to do with carbon footprints, with LCOE, with any of the messy, arguable details. That basis is this: a vast majority of the world’s climate experts believe nuclear power must be part of a climate strategy.
If science is the cure for bullshit, who better to trust than the best scientists in the world?
Unfortunately, climate scientists are largely unqualified to weigh in on policy responses–their expertise is limited to atmospheric and geophysical sciences. For solutions, they don’t have any more knowledge or expertise than the “man on the street.” This is why James Hansen at NOAA overstepped his bounds 30 years ago and in my opinion actually hurt the efforts to develop effective policy responses. (And I was involved at the time on that particular issue.) Raising an alarm is appropriate, but proposing solutions is not.
mcubedecon, James Hansen directs the Program on Climate Science, Awareness and Solutions at Columbia University. Either that prestigious Institution has badly mistaken a “man on the street” for someone whose expertise is internationally reknowned in climate science and potential solutions to anthropogenic climate change, or you have no idea what you’re talking about.
Here is the awards page of Dr. Hansen’s CV (through 2014):
1977 Goddard Special Achievement Award (Pioneer Venus)
1978 NASA Group Achievement Award (Voyager, Photopolarimeter)
1984 NASA Exceptional Service Medal (Radiative Transfer)
1989 National Wildlife Federation Conservation Achievement Award
1990 NASA Presidential Rank Award of Meritorious Executive
1991 University of Iowa Alumni Achievement Award
1992 American Geophysical Union Fellow
1993 NASA Group Achievement Award (Galileo, Polarimeter/Radiometer)
1996 Elected to National Academy of Sciences 1996 GSFC William Nordberg Achievement Medal
1996 Editors’ Citation for Excellence in Refereeing for Geophysical Research Letters
1997 NASA Presidential Rank Award of Meritorious Executive
2000 University of Iowa Alumni Fellow
2000 GISS Best Scientific Publication (peer vote): “Global warming – alternative scenario”
2001 John Heinz Environment Award
2001 Roger Revelle Medal, American Geophysical Union
2004 GISS Best Scientific Publication (peer vote): ‘Soot Climate Forcing’
2005 GISS Best Scientific Publication (peer vote): ‘Earth’s Energy Imbalance’
2006 Duke of Edinburgh Conservation Medal, World Wildlife Fund (WWF)
2006 GISS Best Scientific Publication (peer vote): ‘Global Temperature Change’
2006 Time Magazine designation as one of World’s 100 Most Influential People.
2007 Laureate, Dan David Prize for Outstanding Achievements & Impacts in Quest for Energy
2007 Leo Szilard Award, American Physical Society for Outstanding Promotion & Use of Physics for the Benefit of Society
2007 Haagen-Smit Clean Air Award
2008 American Association for the Advancement of Science Award for Scientific Freedom and Responsibility Nevada Medal, Desert Research Institute
2008 Common Wealth Award for Distinguished Service in Science
2008 Bownocker Medal, Ohio State University
2008 Rachel Carson Award for Integrity in Science, Center for Science in the Public Interest
2008 Carl-Gustaf Rossby Research Medal, American Meteorological Society
2009 Peter Berle Environmental Integrity Award
2009 Sophie Prize for Environmental and Sustainable Development
2010 Blue Planet Prize, Asahi Glass Foundation
2010 American Association of Physics Teachers Klopsteg Memorial Award for communicating physics to the general public
2011 Edinburgh Medal from City of Edinburgh, Edinburgh Science Festival
2011 Steve Schneider Climate Science Communications Award
2012 Foreign Policy designation as one of its Top 100 Global Thinkers
2012 Ridenhour Courage Prize
2013 NASA Distinguished Service Medal
2014 Center for International Environmental Law’s Frederick R. Anderson Award for Outstanding Contributions to Addressing Climate Change
2014 Walker Prize, Museum of Science, Boston
What are the qualifications of someone arrogant enough to assert Hansen has “overstepped his bounds”, to assess the limits of his experience? I eagerly await your response.
I understand that Dr. Hansen is well recognized, but he is a climatologist with no training or experience in policy design, and that was particularly true in the late 1980s when he first began proposing policy solutions. None of the awards that you list make him qualified for making those proposals. I haven’t checked because I don’t have the time to spend research on a blog post, but I would presume those awards are for his research into climate science, not economics or other social sciences research and analysis.
So you maintain Columbia University made an egregious oversight when Hansen was hired to direct their “Program on Climate Science, Awareness and Solutions“. And you feel there is some training lacking in his CV which would render him deficient to suggest proposals for remediating climate change. What might that be, a “Doctorate in Generic Policy Design” from a prestigious university?
That degree does not exist. And since you seem to be unfamiliar with it, here is how policy is “designed”: legislators, who are ill-equipped to make judgments on both the effects of atmospheric issues or potential solutions, summon experts to testify before Congress. Dr. Hansen has done so many times, on both.
Apparently Congress has also made an egregious oversight when they summoned Dr. Hansen instead of you. What are your qualifications?
I’m totally lost on where you’re going with your arguments, first you accuse me of being a renewable advocate, then you defend the qualifications of Dr. Hansen who has made unsupported claims about the potential effectiveness of renewables in reducing climate change impacts.
You can’t have two main objectives. If you go far enough, there will always be a trade-off. With climate, as the article identifies, it comes with accuracy and action: if we academics increase the accuracy of our communication, we will decrease the tons of carbon mitigated.
Every flood, heat wave, and massive storm should be labeled as climate change. We know they might not be. We should encourage cities to go 100% renewable. We know that’s incredibly expensive. But let’s not create an equivalence between ‘we paid too much for mitigation’ or ‘some of us look bad’ and ‘our climate is not habitable for billions of people’.
The end is what matters, not the means.
I disagree. Experts have cried wolf too many times on many other issues, which is why Trump voters distrust many experts and news sources now. They don’t distinguish on whether its about food safety, economic growth or climate change. It’s the job of the advocates to make the points you draw from the analyses. Instead, we need to change our culture so that we embrace the fact that the future is uncertain, and instead we need to mitigate against risky outcomes, instead of pretending that we have a crystal ball.
You do not have the power to change our culture. You can’t enroll everyone in a stats class, and you can’t convey the need to weigh benefits and costs of different policy options.
Conditional on these, it’s a binary choice: create change with what we have, or don’t. But please let’s not be satisfied with convincing the choir on this one.
Again I disagree. Economists changed the culture by introducing and then promoting benefit-cost analysis. Unfortunately, economists also oversold B/C analysis and what it could deliver. Now we need to change the culture again back to understanding that risk needs to be managed by everyone and that the future is uncertain. You are instead arguing that we should take on the role of the high priest and manipulate to whatever ends we desire.
You are calling for political advocacy to support whatever stance YOU believe is the appropriate one. The one power that academics and professionals have is that they at least attempt to account for different issues and perspectives and to be relatively transparent in what we do. That hasn’t always been well done to date, but it’s more evident than in any other stakeholder group in the policy process. Giving up any semblance of independence and transparency to lie to the public is not justified.
Hitler and Stalin rose to power on the basis that the ends justified the means. The fact is that the means are always important because there is always another end coming up.
First, “There is no market for a pollution externality, which is why the market can’t solve the problem.” The fact is there is a market solution–cap & trade–which requires defining and assigning property rights. Climate change skeptics object to defining those rights. That’s the first big hurdle. Once those are defined and assigned, then the market can work. What is frustrating is the belief that somehow only property rights owned by private entities can be part of a market solution.
Second, on addressing uncertainty, I’ve noted that expert analysts are the new high priests of Egypt forecasting the Nile floods. Leaders want someone else to rely on, and the public wants certainty from its high priests. And scientists (including economists) have too readily stepped into that role, instead of resisting and trying to educate leaders and the public. A psychiatrist friend points out that many people have great difficulty psychologically handling uncertainty and demand that comfort. I think the answer is stop focusing on optimizing, and instead focus on minimizing risk, and being clear about this change.
The Integrated Resource Plan (IRP) at the CPUC illustrates how the analytic process squeezes out the uncertainty. Almost all of the “scenarios” put forward by CPUC staff address only the secondary effects on overall costs, e.g., levels of flexible capacity, while ignoring the real cost drivers, e.g., ranges of gas prices and renewable energy technology costs, and demand forecasts. The IRP is evolving into yet another single-point expected value forecast that attempts to find an “optimal” outcome, instead of focusing on where are the risks and vulnerabilities, and what can utilities do to minimize those.
mcubedecon, that you regard cap-and-trade as a solution is mystifying. Neither the Kyoto Protocol, nor EU ETS, nor California’s Cap-and-Trade Program has been shown to result in a net reduction of carbon emissions. Far from it – Kyoto has been estimated to increase carbon emissions by 600 million tonnes:
https://www.theguardian.com/environment/2015/aug/24/kyoto-protocols-carbon-credit-scheme-increased-emissions-by-600m-tonnes
Do you have conclusive evidence that this elaborate exercise, plagued by corruption in every instance, is viable? If not, what’s the basis for your announcement “The fact is..[cap and trade]…is a market solution “?
California CTP has been successful. You’ll need to present evidence to the contrary. The EU ETS was set up for failure because it overallocated allowances, in large part because they were based on higher emissions from Soviet Bloc nations that saw their economies shrink so they had many allowances to sell. Kyoto didn’t have a real CTP program.
On the other hand, the Acid Rain CTP has been highly successful and is one of the elements that is now leading to the retirement of large amounts of coal-fired capacity. Also water rights markets, which are another form of cap & trade, have been successful in the Western U.S. and Australia.
mcubedecon, the statement “California CTP has been successful” is known in the science community as a bare assertion (Latin: ipse dixit). No, it is not my responsibility to prove your assertion is wrong – it’s yours to substantiate, for reasons which should be obvious.
You started this discussion with a bald assertion that it had failed. Citing to a study on Kyoto has no relevance to the California CTP. You accepted the burden of proof when you made your assertion.
Here’s the emission trends report for California that shows falling GHG emissions in absolute terms.
Click to access ghg_inventory_trends_00-15.pdf
However, note that this needs to be compared to the “business as usual” baseline forecast to see the measured emission reductions. I’ve included the link to that report as well.
Click to access scoping_plan_forecast_2014-05-22.pdf
I’m not sure what other metric of success for the Califor
mcubedecon, you are quite the catalog of logical fallacy – now you resort to the argumentum ad ignorantiam, or shifting the burden of proof: “I maintain that there are ghosts, because you cannot prove that there are no ghosts.”
Like most arguments with renewables advocates, your reference-free, fallacy-plagued opinions are quickly becoming a waste of my time. Have a nice day.
Again, you made the initial unsupported assertion that the California CATP had failed (and referring to how Kyoto performed is not evidence). I provided evidence that California emissions have been falling with the CATP. Perhaps that may be coincidence rather than causation, but on the other hand if the CA CATP had failed, then emissions would be rising, which they are not.
What do we do with those people who don’t know what ” … hit them with a cricket bat” means? They might start mating bats with crickets. And who knows what that might lead to.
Good that most researchers are open minded. But many researchers are human, they have finite lives, want to protect their domain [i.e. empires], and incomes [i.e. research grants and publishing]. And climate change happens on an almost geologic timeframe, so few of us will actually be around to conclude experiments. I do hope that someone somewhere has started recording and storing data that may be evaluated with contemporary situation ten thousand years from now.
Whether its climate change or anything else don’t overlook that basis human trait, making a buck. Much of the policy in some areas is about people getting paid, as we would say in South LA. So we only count toward GHG reduction the technologies that lobbied for the RPS rules, wind and solar. What about hydro or other ways of reducing GHG? What about issues like frequency, inertia, cost, reliability and cost shifting to name a few. If I were to ask a homeless person or for that matter over half the American public what is the most pressing problem we face, climate would be way down on the list. That doesn’t mean we shouldn’t try to address it, rather we should work on it along with all our other problems. If we are going to be serious about reducing GHG, after energy efficiency, and smaller cars we will have to make some serious lifestyle changes. It will be at that point we will see peoples commitment to change. Right now we are just making certain folks rich at the expense of others.