The Problem with Demand Response
A recent event highlights the difficulties setting baselines for demand response programs.
It’s starting to feel like fall. In Berkeley, we are sensitive to subtle changes, but there was a bit of chill in the air last week and the sun is setting earlier.
Three months ago, the season change – from spring to an early summer heatwave – provided a poignant example of the existential problem with demand response.
Here’s what happened, as best I can reconstruct. Tuesday, June 20 was hot in Southern California. Temperatures were above 100 in Los Angeles and above 110 further inland. At 4PM that day, Southern California Edison (SCE) called on its demand response programs and achieved nearly 500 MW of reductions. The reductions are clear in the graph below – there’s a dramatic drop in load between 4:00 and 4:07 PM.
Let me be clear about terms. By “demand response,” I’m referring to programs where customers are paid, “to reduce their consumption relative to an administratively set baseline level of consumption,” following and quoting a very useful article by Jim Bushnell, Ben Hobbs and Frank Wolak. For example, SCE has an, “Agriculture and Pumping Interruptible Program,” through which customers are rewarded if they allow the utility to install a device on equipment to remotely turn it off when the electricity system is stressed, as it was in Southern California on June 20.
I’m not including “dynamic pricing” programs, such as SCE’s Summer Advantage Incentive critical peak pricing program. SCE declared June 20 a critical peak pricing event, meaning that some of their customers faced higher prices from 2 to 6PM that day.
Unfortunately, though, the dramatic 500 MW drop was not recognized by the California Independent System Operator as a real reduction. It appears that the California Independent System Operator’s (ISO) process for determining demand response payments decided that the customers’ consumption was above their baseline. The rules around demand response are super arcane, even for the electricity sector. But, I believe this means that the ISO charged SCE for these overages, rather than rewarding them for their customers’ reductions.
What’s a baseline? In a demand response program, customers are cutting their load on a specific date and time, called a demand response “event.” The challenge is to determine how much the customer would have been consuming absent the event. That’s the role of the baseline – it’s designed to reflect the “but-for” consumption levels.
Here’s where the season change comes in. The ISO calculates baselines using the average of the 10 most recent non-event business days (the “10-in-10” methodology), with an adjustment – up to a 20% adder – for conditions on the event day. Since June 20 was an early season heatwave, recent average consumption was a lot lower than the peak levels reached during the high temperatures that day. The graph below shows what Southern California Edison reported for one of its residential demand response programs. The baseline (in blue) is one third lower than the actual June 20 load (in orange), even after the apparent reduction.
Ah, you might say, why doesn’t the ISO just do what we can do with our eyes in the first graph above and compare 4:00 to 4:07? Here’s one problem with that approach: if demand response customers suspect an event will be called, maybe because the day is really hot, they might overconsume – for example, by pre-cooling their house – before 4 PM in order to inflate their baseline. One advantage of the ISO’s 10-in-10 methodology is that customers are unlikely to know that they’re in a baseline period until after the fact.
So, why am I hating on demand response? I certainly don’t object to the idea of engaging demand in electricity markets. What reasonable economist could object to working with both sides of the market – supply and demand? Plus, there are some super innovative technologies being developed to help consumers reduce demand.
The thing I object to is paying customers to reduce relative to an error-prone baseline. As we’ve emphasized in the energy efficiency context, counterfactuals are hard to develop. Economists often point to the possibility that customers will strategically increase their demand during baseline periods in order to later be paid to reduce relative to an inflated benchmark. The June 20 example highlights the opposite problem – the compensation mechanism overlooking what appear to be real reductions. If customers are taking steps to reduce and then not benefiting from it, this will dampen interest in demand response programs. And, ultimately, this will dampen company’s incentives to develop more cool technologies to help them.
It’s especially frustrating for economists to watch these baseline debates as there’s such a simple solution – use a baseline of ZERO. Don’t pay customers to reduce relative to an administratively set baseline when the electricity system is stressed, like on June 20. Instead, charge them more for consuming during these periods. Ideally, the prices, customers pay would vary by both time and location. Sound familiar? That’s exactly what dynamic pricing does.
So, why have we gone so far down the demand response path, with such limited success introducing any form of dynamic pricing? I have ideas about why, and also hope blog readers who are more steeped in the debate can chime in. One guess is that it’s been difficult to develop retail rates that would expose a lot of customers to wholesale price variation. Regulators may be locked into the status quo and have a hard time introducing new pricing paradigms. Framing a program as rewarding customers for reductions instead of charging them for consumption is probably politically more palatable.
But, the June 20 event highlights that there are costs to going this way. We’ll get things wrong and eventually get less demand-side participation. Also, Severin’s previous blog post emphasizes the costs when customers game their baselines or get paid through a demand response program for doing something they would have done otherwise, like going on vacation. Are the regulators’ needs for political acceptability strong enough to outweigh these costs? I don’t know.
Addendum: Two quick notes based on correspondence with SCE: The June 20 event discussed was in 2016, not this year. Also, the agriculture pumping program used in the example wasn’t dispatched that day. SCE attributes most of the drop to an air conditioner cycling program whereby residential and commercial customers agree to let the utility turn off their compressors during an event.
Categories
Catherine Wolfram View All
Catherine Wolfram is the William F. Pounds Professor of Energy Economics at the MIT Sloan School of Management. She previously served as the Cora Jane Flood Professor of Business Administration at the Haas School of Business at UC Berkeley. From March 2021 to October 2022, she served as the Deputy Assistant Secretary for Climate and Energy Economics at the U.S. Treasury, while on leave from UC Berkeley. Before leaving for government service, she was the Program Director of the National Bureau of Economic Research’s Environment and Energy Economics Program, Faculty Affiliate of the Energy Institute at Haas from 2000 to 2023, as well as Faculty Director of the Energy Institute from 2009 to 2018. Before joining the faculty at UC Berkeley, she was an Assistant Professor of Economics at Harvard. Wolfram has published extensively on the economics of energy markets. Her work has analyzed rural electrification programs in the developing world, energy efficiency programs in the US, the effects of environmental regulation on energy markets and the impact of privatization and restructuring in the US and UK. She is currently working on several projects at the intersection of climate and trade. She received a PhD in Economics from MIT in 1996 and an AB from Harvard in 1989.
Energy Institute Blog 
Picking up on Robert Borlick, demand response is not merely counterintuitive, it’s a repudiation of basic economics. Goods are good. The basis for consumer theory. Demand Response should be a dead giveaway that electricity is not an orthodox, normal private good. It’s more like a public good. Unfortunately, too many people are now invested in the idea of electricity markets that those close to policy will not face up to this. Hence, two decades of pointless “debate”.
In the “bad old days’ when restructuring electricity markets was initially contemplated, two of the primary metrics of the restructured markets’ success were better capital investments for electricity supply and providing retail customers prices that better reflected the actual economics of the markets, the latter envisioned as a check on suppliers’ market power. Whether the former has happened is open to debate. On the other hand, providing the right price signals in the right way to retail electric customers, particularly the residential and small commercial ones, has turned out to be just hard. To start with, the price signals they can be given are, in a sense, incomplete. Locational marginal prices (LMPs) are not inclusive of capacity costs. They might be in a market with just energy prices, but it is has always been clear that RTOs and regulators are uncomfortable letting LMPs fully reflect not just transmission congestion costs, but generation congestion costs, too. Secondly, giving retail customers price information is hard. If you cruise the aisles of a grocery store or drive into a service station, you know the prices of the goods or services you purchase. Even with the advent of smart meters, that information remains at arm’s length from the customer for the most part. You need to provide the proper “tools” for dealing with the electricity market’s (volatile) prices. For example, customers need the capability of observing the prices they are being charged through, say, an easily readable interface (presumably accompanied by a market information stream to their devices – computers, mobile devices, etc.). In the best of all worlds, since energy markets are such specialized kinds of institutions, you might even provide to such customers price prediction tools they could base their short and long behavior on, both how intensely they use their energy using devices in the immediate future and in their appliance investment decision making. Third, in the long run there will be the marvels of the Internet of Things (IoT) and which will automate some of that short-run decision making by customers on their appliance usage, etc. If the control technology doesn’t make plain how to accomplish the trade-off between operating costs and comfort/convenience/needs, then the IoT will fail to fulfill its potential for energy markets.
This all to say that efficient retail market rationing, whether on a quantity basis through demand response or on a price basis through dynamic pricing, is hard.
Do demand response programs, with a fixed payment, allow utilities to capture more of the surplus created by customer flexibility? Or at least, to think that they will? Theory says they won’t, but when combined with the peculiarities of rate regulation and consumer marketing, perhaps they could have that impression.
There are other fundamental problems with almost all demand response programs, in addition to arbitrary baselines. First, it is a “bang-bang” solution to a continuously varying situation. The utility either does nothing, or it “flips the switch” on many customer devices, almost simultaneously. In your example, load dropped by 500 MW in 7 minutes. The utility does not have the option, for example, of asking for a 50% response at 3pm and 100% at 4pm.
The second problem is that it replaces marginal and customer decision making, with decisions made by the utility with no knowledge of customer circumstances. All but the large customers will probably choose a cutoff price as their default response, but it is valuable to them to know that they have an opportunity to “buy out” of their agreement if something special is happening. And the larger/more sophisticated customers will have many opportunities to make marginal and partial adjustments based on their current and projected needs.
There are many other disadvantages compared with dynamic pricing. Now that customers have 15 minute meters as the default, any demand response program can be Pareto-improved by dynamic pricing plus a default price decision rule. William Vickrey laid out the basic principles in the 1970s.
I used to claim, half seriously, that the emphasis on demand response in the (academic and applied) electricity pricing literature was due to econometricians’ preference for large data sets with clear signals in them. But statistical tools have improved by orders of magnitude, and more complex data is probably preferable now. Prof. Fred Schweppe at MIT used to say that “power system engineers like to be able to push a big red button” when they want something to happen, i.e. that they liked the feeling that they had instant control. Could the continuing bias be due to current utility executives and regulators having been trained 20+ years ago, and preferring what they were taught? I look forward to other answers.
The whole idea of paying someone not to consume is counter intuitive – and for good reasons. Establishing a counterfactual through the use of baselines is flawed and is error-prone even when the consumers do not try to game the system (which the will – particularly large C&I who are sophisticated and can hire people to design schemes to “fiddle” the system).
The obvious answer is to require everyone to pay retail prices that closely reflect their suppliers’ marginal costs of serving them. This is simple economics 101.
The consumer advocates oppose cost-reflective retail pricing, presumably to protect the consumer from unexpectedly getting large bills. However, there are ways for consumers to shield some or all of their consumption from market prices by buying hedges. In fact, the fixed price tariffs in common use are nothing more than 100 percent hedges. What gets lost in the discussion is that hedges cost the consumer a lot of money because the shift financial risk to the suppler and no rational suppler will assume that risk without collecting a risk premium, often a large one, which is built into the fixed price.
What the consumer advocates seem to miss is that consumers have to pay the suppliers’ full costs of serving them, one way or another. Like the mechanic (in a commercial for some motor oil) said, “you can pay me now or you can pay me later.” Not allowing consumers the choice to be served under tariffs that apply hourly retail prices indexed to the day-ahead or reali-time market prices produces economically inefficient consumption that drives up the cost for all consumers. Consumer advocates need to understand that their efforts to “protect” consumers from volatile retail prices are actually harming them.
A more appropriate role for consumer advocates is to educate consumers on how they can benefit from dynamic pricing by being able to shift their usage and also on how they can shield their inflexible loads through well designed hedge products. For demand response to be effective only a small portion of total demand (e.g., 20 percent) needs to be exposed to dynamic prices. That would be enough to eliminate most of the need for expensive peaking generators.
I extensively addressed these issues in filings before the FERC in the two dockets that led up to the infamous Order 745. That order overcompensates customers for their demand response – particularly large customers that already pay prices tied to the hourly wholesale market prices.
How many times does this issue have to be debated before clarity is achieved?
“How many times does this issue have to be debated before clarity is achieved?”
Apparently ad nauseum.
Some of us have heard these same arguments about demand response for nearly two decades. The fact is, existing demand response programs do not work all that well and the abuses have been well documented. You won’t find another industry on earth (including the airlines) that pays consumers to avoid using a good or service they never purchased in the first place.
Another non-sensical example of paying to not do something – farm subsidies. They may have made some sense as a social welfare initiative when there were small/ family-owned farms. No more.
Either we let the ‘market’ decide – or we are a centrally managed economy; cant have a mix without picking winners-losers.
Let me try to contribute my views to market economics which pays you not to consume electricity. During DR events the utility knows that it will be struggling or not be able at all to provide additional services its customers will request. Hence it can not make profits. However, due to its contractual obligations and nature of its business has to deal somehow with those additional services including several options: (1) hire other service providers / ramp up generation / capacity market and/or invest in building additional grid capacity when its limits are reached, (2) persuade its customers not to request more services / demand response; (3) pay penalties in the form of hampered reputation, future business prospects, political pressure, legislative requirements, possibly system component breakdown / repair costs etc. when the customers are forcefully cut-off from the grid as their demand is not met. For which of these the utility may opt should simply depend on the cost of each option. Often the cost of demand response option is far lower than of the other two options. However, as the demand continues to rise, we will be seeing that cost for DR and other options will also increase to the point that some new ways could emerge through innovation / R&D, wireless energy transfer, peer to peer trading…
And indeed the airline industry sometimes faces similar problems and options, choosing / offering to pay the passengers not to be on a flight they booked. And rarely, as we have seen not so long ago, even chooses the option (3) to forcefully remove passengers from airplanes (given the aftermath costs of such an option, I think you would agree this turned to be really costly).
It’s impossible to come up with rules that pay customers for what they don’t consume because there is no way to measure it. Estimates are about as good as we can do and if we aren’t willing to settle wholesale transactions based on estimates because it invites gaming, we should be equally unwilling to settle retail transactions on the same basis. Paying one group of customers for demand reductions that may or ,may not be real is unfair to the customers that can’t or don’t participate, and to the market participants that must settle based solely on meter data.
The evidence I’ve seen suggests customers can and will respond to prices as is the case in Illinois. Charging customers for what they use is fair and objective. There are ways to “protect” customers who cannot or will not respond to variable prices, either by charging them a small premium or having them buy a subscription that includes a known quantity and price for each hour.
The issues with the 10-in-10 baseline are well-known in the industry. CAISO has already approved a proposal to revamp those rules, based on a proposal that Nexant developed as part of a statewide working group. See here: http://www.caiso.com/Documents/2017BaselineAccuracyWorkGroupFinalProposalNexant.pdf. The problem is that those rules haven’t gone into effect yet because CAISO is waiting on FERC approval (it could be a long wait).
I think the reason we don’t see more dynamic pricing in retail rates has to do with poor communications strategy and the political economy of PUCs. Too often, the message that Commissioners hear is that time-varying rates of any sort will expose low and fixed income ratepayers to price spikes they can’t control or avoid. This is a largely unsubstantiated claim made by rate payer advocates using hypotheticals rather than data. If we want more dynamic pricing, we need to (1) produce the evidence that shows that not allowing for dynamic pricing imposes a penalty on the most price-sensitive customers (ie low and fixed income ratepayers) and (2) that flat rate volumetric electricity pricing is akin to gasoline subsidies in developing countries – a program that purports to help low-income consumers while conferring the vast majority of benefits on wealthy users of the power system. The Brattle Group has gone some way in producing evidence on this but far more needs to be done.
It appears to me that the “problem” here is the ISO rules that fail to take into account significant and “real” reductions in usage when customers are asked to do so. Your response would require all customers to see unpredictable price signals and face the potential of unaffordable bills (speaking about residentials primarily). Let’s make the current programs work by linking retail programs to proper wholesale market rules and it seems that in this case the wholesale market rules need to change. The suggestion that customers will “game” the system if they think a critical peak event is coming is without any justification when dealing with hundreds, thousands, or tens of thousands of demand response installations. What is wrong with pre cooling anyway? Sounds like a rational response to me as long as usage is reduced during the critical hours.