A recent event highlights the difficulties setting baselines for demand response programs.
It’s starting to feel like fall. In Berkeley, we are sensitive to subtle changes, but there was a bit of chill in the air last week and the sun is setting earlier.
Three months ago, the season change – from spring to an early summer heatwave – provided a poignant example of the existential problem with demand response.
Here’s what happened, as best I can reconstruct. Tuesday, June 20 was hot in Southern California. Temperatures were above 100 in Los Angeles and above 110 further inland. At 4PM that day, Southern California Edison (SCE) called on its demand response programs and achieved nearly 500 MW of reductions. The reductions are clear in the graph below – there’s a dramatic drop in load between 4:00 and 4:07 PM.
Let me be clear about terms. By “demand response,” I’m referring to programs where customers are paid, “to reduce their consumption relative to an administratively set baseline level of consumption,” following and quoting a very useful article by Jim Bushnell, Ben Hobbs and Frank Wolak. For example, SCE has an, “Agriculture and Pumping Interruptible Program,” through which customers are rewarded if they allow the utility to install a device on equipment to remotely turn it off when the electricity system is stressed, as it was in Southern California on June 20.
I’m not including “dynamic pricing” programs, such as SCE’s Summer Advantage Incentive critical peak pricing program. SCE declared June 20 a critical peak pricing event, meaning that some of their customers faced higher prices from 2 to 6PM that day.
Unfortunately, though, the dramatic 500 MW drop was not recognized by the California Independent System Operator as a real reduction. It appears that the California Independent System Operator’s (ISO) process for determining demand response payments decided that the customers’ consumption was above their baseline. The rules around demand response are super arcane, even for the electricity sector. But, I believe this means that the ISO charged SCE for these overages, rather than rewarding them for their customers’ reductions.
What’s a baseline? In a demand response program, customers are cutting their load on a specific date and time, called a demand response “event.” The challenge is to determine how much the customer would have been consuming absent the event. That’s the role of the baseline – it’s designed to reflect the “but-for” consumption levels.
Here’s where the season change comes in. The ISO calculates baselines using the average of the 10 most recent non-event business days (the “10-in-10” methodology), with an adjustment – up to a 20% adder – for conditions on the event day. Since June 20 was an early season heatwave, recent average consumption was a lot lower than the peak levels reached during the high temperatures that day. The graph below shows what Southern California Edison reported for one of its residential demand response programs. The baseline (in blue) is one third lower than the actual June 20 load (in orange), even after the apparent reduction.
Ah, you might say, why doesn’t the ISO just do what we can do with our eyes in the first graph above and compare 4:00 to 4:07? Here’s one problem with that approach: if demand response customers suspect an event will be called, maybe because the day is really hot, they might overconsume – for example, by pre-cooling their house – before 4 PM in order to inflate their baseline. One advantage of the ISO’s 10-in-10 methodology is that customers are unlikely to know that they’re in a baseline period until after the fact.
So, why am I hating on demand response? I certainly don’t object to the idea of engaging demand in electricity markets. What reasonable economist could object to working with both sides of the market – supply and demand? Plus, there are some super innovative technologies being developed to help consumers reduce demand.
The thing I object to is paying customers to reduce relative to an error-prone baseline. As we’ve emphasized in the energy efficiency context, counterfactuals are hard to develop. Economists often point to the possibility that customers will strategically increase their demand during baseline periods in order to later be paid to reduce relative to an inflated benchmark. The June 20 example highlights the opposite problem – the compensation mechanism overlooking what appear to be real reductions. If customers are taking steps to reduce and then not benefiting from it, this will dampen interest in demand response programs. And, ultimately, this will dampen company’s incentives to develop more cool technologies to help them.
It’s especially frustrating for economists to watch these baseline debates as there’s such a simple solution – use a baseline of ZERO. Don’t pay customers to reduce relative to an administratively set baseline when the electricity system is stressed, like on June 20. Instead, charge them more for consuming during these periods. Ideally, the prices, customers pay would vary by both time and location. Sound familiar? That’s exactly what dynamic pricing does.
So, why have we gone so far down the demand response path, with such limited success introducing any form of dynamic pricing? I have ideas about why, and also hope blog readers who are more steeped in the debate can chime in. One guess is that it’s been difficult to develop retail rates that would expose a lot of customers to wholesale price variation. Regulators may be locked into the status quo and have a hard time introducing new pricing paradigms. Framing a program as rewarding customers for reductions instead of charging them for consumption is probably politically more palatable.
But, the June 20 event highlights that there are costs to going this way. We’ll get things wrong and eventually get less demand-side participation. Also, Severin’s previous blog post emphasizes the costs when customers game their baselines or get paid through a demand response program for doing something they would have done otherwise, like going on vacation. Are the regulators’ needs for political acceptability strong enough to outweigh these costs? I don’t know.
Addendum: Two quick notes based on correspondence with SCE: The June 20 event discussed was in 2016, not this year. Also, the agriculture pumping program used in the example wasn’t dispatched that day. SCE attributes most of the drop to an air conditioner cycling program whereby residential and commercial customers agree to let the utility turn off their compressors during an event.
Catherine Wolfram is Associate Dean for Academic Affairs and the Cora Jane Flood Professor of Business Administration at the Haas School of Business, University of California, Berkeley. She is the Program Director of the National Bureau of Economic Research's Environment and Energy Economics Program, Faculty Director of The E2e Project, a research organization focused on energy efficiency and a research affiliate at the Energy Institute at Haas. She is also an affiliated faculty member of in the Agriculture and Resource Economics department and the Energy and Resources Group at Berkeley.
Wolfram has published extensively on the economics of energy markets. Her work has analyzed rural electrification programs in the developing world, energy efficiency programs in the US, the effects of environmental regulation on energy markets and the impact of privatization and restructuring in the US and UK. She is currently implementing several randomized controlled trials to evaluate energy programs in the U.S., Ghana, and Kenya.
She received a PhD in Economics from MIT in 1996 and an AB from Harvard in 1989. Before joining the faculty at UC Berkeley, she was an Assistant Professor of Economics at Harvard.