Professional Documents
Culture Documents
Lee
Sibley School of Mechanical and
Aerospace Engineering,
Cornell University,
Ithaca, NY 14853
e-mail: zel3@cornell.edu
Qingxuan Sun
School of Energy and Power Engineering,
Xi’an Jiaotong University, Providing Grid Services With Heat
Zhao Ma The integration of variable and intermittent renewable energy generation into the power
School of Energy and Power Engineering,
system is a grand challenge to our efforts to achieve a sustainable future. Flexible
Xi’an Jiaotong University,
demand is one solution to this challenge, where the demand can be controlled to follow
Xi’an 710049, China
energy supply, rather than the conventional way of controlling energy supply to follow
e-mail: zm64@cornell.edu
demand. Recent research has shown that electric building climate control systems like
heat pumps can provide this demand flexibility by effectively storing energy as heat in
Jiangfeng Wang the thermal mass of the building. While some forms of heat pump demand flexibility have
School of Energy and Power Engineering,
been implemented in the form of peak pricing and utility demand response programs, con-
Xi’an Jiaotong University,
trolling heat pumps to provide ancillary services like frequency regulation, load following,
Xi’an 710049, China
and reserve have yet to be widely implemented. In this paper, we review the recent advances
e-mail: jfwang@xjtu.edu.cn
and remaining challenges in controlling heat pumps to provide these grid services. This
analysis includes heat pump and building modeling, control methods both for isolated
Jason S. MacDonald heat pumps and heat pumps in aggregate, and the potential implications that this
Ernest Orlando Lawrence Berkeley concept has on the power system. [DOI: 10.1115/1.4045819]
National Laboratory,
1 Cyclotron Road, Keywords: air conditioning, building, control systems, energy, grid, optimal controls,
Berkeley, CA 94720 optimization
e-mail: JSMacDonald@lbl.gov
K. Max Zhang1
Sibley School of Mechanical and
Aerospace Engineering,
Cornell University,
Ithaca, NY 14853
e-mail: kz33@cornell.edu
Journal of Engineering for Sustainable Buildings and Cities FEBRUARY 2020, Vol. 1 / 011007-1
Copyright © 2020 by ASME
harnessed by utilities in the form of thermostat-based demand which are broadly defined based on their time scale, presented in
response programs. These programs generally consist of utilities Table 1. In deregulated markets, grid operators procure these services
turning off heat pumps during extreme peak load hours, either through ancillary service markets. In contrast to energy markets,
through setpoint modification or direct load control. In addition, where generators are only paid for the energy they produce, ancillary
the use of thermal energy storage (TES) has grown in popularity par- service markets are primarily capacity markets, where a grid operator
ticularly in Europe and allows for load shifting to accommodate high also pays for the capacity of a generator to alter its production. A tech-
levels of renewable energy [6]. However, new research shows the nical review of ancillary services is given in Ref. [14], while Ref. [15]
potential for heat pumps to provide more complex grid services by gives a review of various US ancillary service market structures.
operating in ancillary service markets. Ancillary services, which While ancillary services are often provided by generators, they can
are often provided by controllable thermal generators, are essential also be provided through demand response. Demand response is
for power system stability and maintain the instantaneous balance the process of controlling the demand to respond to grid signals. Ref-
of electricity supply and demand on the grid. Providing these services erence [16] describes the role of demand response in ancillary service
if accepted, must follow the power signal sent by the system oper- 2.2 Load Following. Load following consists of generators
ator. Depending on the system operator, the regulation market either following the slower, more predictable fluctuations in electricity
has separate markets for regulation-up and regulation-down or demand on a time scale of several minutes to several hours. This
requires symmetric regulation capacity (equal up and down regula- is often procured through economic dispatch, where generators
tion capacity). Currently, California Independent System Operator are dispatched according to their generation cost [25]. However,
(CAISO) and Electric Reliability Council of Texas (ERCOT) are as wind and solar supply an increasing percentage of electricity
the only US system operators that operate separate up and down on the grid, this service could become much more important, partic-
regulation markets. For generators, these two methods are equiva- ularly for ramping in the mornings and evenings [26]. This could be
lent [14]. However, for energy storage and load control, significant a potential service provided by heat pumps, either as a reserve
differences in revenue can occur based on the market type. capacity similar to CAISO’s flexible ramping product [27] or
Another challenge for demand response and energy storage through responsiveness to a real-time price disseminated by the
systems is that the frequency regulation signal is not necessarily system operator. For example, when solar and wind energy are
zero-mean. For heat pumps providing frequency regulation, a readily available, electricity prices can drop significantly due to a
signal bias can cause the heat pump to run consistently below or surplus in supply, encouraging loads to operate during these
above its baseline consumption, potentially violating comfort con- times. In grids with high solar penetration, such as in the CAISO,
straints. To resolve this, some system operators have introduced there is a growing frequency of negative wholesale electricity
fast regulation signals that are designed to be zero-mean [19]. For prices [28], where generators must pay to produce electricity.
example, Pennsylvania, Jersey, Maryland Regional Transmission This poses a unique opportunity for heat pumps to potentially be
Organization (PJM) has filtered its signal into two, called RegA paid for operation.
and RegD, which are shown in Fig. 1. The RegA signal has a Since heat pumps operate in the retail electricity market, they are
slower time constant and was designed to accommodate steam gen- often charged a static electricity price, giving no incentive to shift
erators with relatively low ramping capability. RegD consists of operation toward times of high energy supply. Time-of-use rates,
higher frequency fluctuations and often converges to zero-mean which have predefined price tiers for peak and off-peak hours,
within 15 min [20]. For this reason, many studies on the technical have had some success in providing consumers’ indirect access
capability of providing frequency regulation with heat pumps for providing a load following service by shifting load away from
follow the RegD signal [11,12,21,22]. peak hours [29]. Connected thermostat demand response programs
In addition, many US system operators have implemented a such as Austin Energy’s Power Partnersm program5 have also been
pay-for-performance pricing structure in response to FERC Order widely deployed. These programs allow the utility to turn off heat
755 [23]. In addition to paying for capacity, this pricing structure pumps for short periods of time to reduce the peak load.
also pays for mileage and performance. Mileage, or movement, is However, these methods are simplified and therefore do not
calculated as the sum of the absolute values of the regulation capture the full potential of heat pumps to provide a load following
control signal movements and given in ΔMW/MW. Given capacity service.
Pmax and power outputs {P1 … Pn}, the mileage for n time steps is A second challenge to provide this service is the relatively low
calculated as [19] follows: frequency of a load following signal. If the frequency of the load
following signal is on the same order of the building’s thermal
n
response, comfort constraints can be violated [30]. This severely
M= |Pi − Pi−1 |/Pmax (1) limits the capacity that heat pumps can offer for load following
i=1
compared with a higher frequency signal like frequency regulation.
Performance is given as a score between 0 and 1 and represents how However, this time constant has the added benefit of reducing the
well the participant follows a regulation signal. A frequency regu- need for fast response controllers.
lator must achieve a minimum performance score to qualify, and
depending on the market structure, higher performance scores can 2.3 Reserve. Power systems are required to maintain a certain
lead to higher payments. PJM’s performance score is often used amount of reserve margin to ensure reliability in case of contingen-
as a benchmark for frequency regulation control algorithms and is cies. For example, if a large generator unexpectedly trips, the
calculated using a combination of three subscores involving system might need to dispatch reserves. To provide this service, a
delay, correlation, and precision [24]. More information on fre- heat pump or heat pump aggregation bids a reserve capacity into
quency regulation policies for specific Independent System Opera- the reserve capacity market. This contract requires the system to
tors can be found in Ref. [15]. curtail its full capacity offering for a certain amount of time
4 5
http://bit.ly/2yieHWA http://bit.ly/2YvJvlu
Journal of Engineering for Sustainable Buildings and Cities FEBRUARY 2020, Vol. 1 / 011007-3
determined by the reserve dispatch signal. After the signal ends, the regression. The dynamic VSHP model is expressed as the transfer
heat pump system can recover back to its baseline energy function:
consumption.
nω1 s + nω0
The reserve market can be split into two main categories: spin- ΔP(s) = Δω(s) (3)
ning and nonspinning reserve. While different system operators s2 + dω1 s + dω0
can sometimes have different definitions [15], spinning (or syn- The coefficients nω1, nω0, dω1, and dω0 can similarly be fit from
chronous) reserves primarily consist of online generators syn- the experimental data. Another simplified model for the fast
chronized to the grid and capable of dispatching to full dynamics of a water-based VSHP is given in Ref. [34]. While
capacity within 10 min. Nonspinning reserves must respond this control model assumes a steady-state response, the nonlinear
within 10–30 min but are not necessarily connected to the grid. transient dynamics is accounted for with an estimated “lost
Providing spinning reserve is preferred over nonspinning reserves thermal energy.” These simplified models allow for manipulation
for two main reasons. First, heat pumps are already connected to
Journal of Engineering for Sustainable Buildings and Cities FEBRUARY 2020, Vol. 1 / 011007-5
control method was used to directly control TCL status. This time such that the contracted ancillary service capacity can be main-
method prioritized turning on or off the TCLs that were closest to tained. This baseline definition is slightly different from a traditional
automatically turning on or off, respectively. Finally, Ref. [66] demand response counterfactual baseline, which uses the historical
explored the stability of TCLs as a result of significant perturbations data to estimate what the unmodified energy consumption would
during control for demand response. have been to measure the amount of demand response provided.
However, the majority of these TCL controllers use simplified, In contrast, an ancillary service baseline can be decided by the ancil-
simulated models that neglect many important differences lary service provider based on market and weather conditions to
between heat pumps and other TCLs like electric heaters. For optimize a user-defined objective. Model-predictive control
example, to avoid damaging the compressor and reducing effi- (MPC) is among the most widely used methods to determine an
ciency, heat pumps have minimum on and off times, which can ancillary service baseline. MPC is an iterative control scheme that
be the most financially and physically limiting factor for ancillary optimizes a model-based objective function over a given time
service provision [68]. Moreover, heat pump COP can vary drasti- horizon. The optimal control for the first time step is then imple-
Journal of Engineering for Sustainable Buildings and Cities FEBRUARY 2020, Vol. 1 / 011007-7
energy storage devices include redox flow batteries (65–85%), attributed the low RTE values reported from experiments [92] to
lithium-ion batteries (85–95%), flywheels (93–95%), and pumped the fact that the experiment ran only one cycle. Therefore, more
hydro storage (70–82%) [96]. For a heat pump providing a sym- experimental results are required to accurately define the RTE for
metrical ancillary service request, the RTE can be defined simi- a heat pump.
larly [92], In addition to RTE, there are efficiency losses associated with
providing flexibility capacity. To provide flexibility, the heat
Eout pump might need to deviate from the energy optimal control sched-
RTE =
Ein ule. The amount of energy increase compared with an energy
optimal controller in Refs. [9,10] was 68 % for the fan and 11%
where Eout is the energy reduction with respect to the baseline due to for the chiller. However, by including payments for providing ancil-
the ancillary service and Ein is the increase with respect to the base- lary services, this controller provided the cost optimal solution
line. In calculating RTE, the baseline is set to be the counterfactual despite increases in energy. Moreover, Ref. [34] found that the
Table 2 Revenue summary of ancillary service provision by heat pumps and other TCLs
Reference [12] PJM—RegA and Offsets 46% of the electricity cost for RegA 2–4.75 kW VSHP power
RegD
Offsets 56% of the electricity cost for RegD
Reference [105] PJM—RegA and Offsets 20–48% of the electricity cost 44.0-kWth variable-speed rooftop unit
RegD
35.2-kWth split heat pump
Reference [101] CAISO— AC: $0.31–9.36/kW/year AC with electrical capacity of 4–7.2 kW
regulation market
HP: $2.04–8.31/kW/year HP with electrical capacity of 4–7.2 kW
Water heater: $33.72/kW/year Water heater with electrical capacity of 4–5 kW
Refrigerator: $36/kW/year Refrigerator with electrical capacity of 0.1–0.5 kW
Reference [94] CAISO— AC: $0–5.71/kW/year AC with electrical capacity of 4–7.2 kW
regulation market
HP: $3.93–10/kW/year HP with electrical capacity of 4–7.2 kW
Electrical heater: $5.33/kW/year Electrical water heater with electrical capacity of 4.5 kW
Refrigerator: $31.43/kW/year Refrigerator with electrical capacity of 0.2–0.5 kW
Reference [68] Germany— Not financially viable Electrical storage system of 5 kWh
residential
frequency reserve
3.7 kW heat pump
Water heat storage of 400 L
Reference [106] Netherlands— $26.56/kW/year in “always available” scenario Heat pump with electrical capacity of 0.5 kW
frequency
containment
reserve
$115.44/kW/year in “always reliable” scenario
Note: AC, a heat pump providing air conditioning; HP, a heat pump in heating mode.
Journal of Engineering for Sustainable Buildings and Cities FEBRUARY 2020, Vol. 1 / 011007-9
[14] Kirby, B., 2016, Ancillary Services: Technical and Commercial Insights, [45] Eisenhower, B., O’Neill, Z., Narayanan, S., Fonoberov, V. A., and Mezić , I.,
Wärtsilä. 2012, “A Methodology for Meta-Model Based Optimization in Building
[15] Zhou, Z., Levin, T., and Conzelmann, G., 2016, “Survey of U.S. Ancillary Energy Models,” Energy Buildings, 47, pp. 292–301.
Services Markets,” Argone National Laboratory, Lemont, IL. [46] Blum, D. H., Arendt, K., Rivalin, L., Piette, M. A., Wetter, M., and Veje, C. T.,
[16] MacDonald, J., Cappers, P., Callaway, D., and Kiliccote, S., 2012, “Demand 2019, “Practical Factors of Envelope Model Setup and Their Effects on the
Response Providing Ancillary Services,” Proceedings of the Grid-Interop Performance of Model Predictive Control for Building Heating, Ventilating,
Forum., Irving, TX, Dec. 3–6. and Air Conditioning Systems,” Appl. Energy, 236, pp. 410–425.
[17] Le Dréau, J., and Heiselberg, P., 2016, “Energy Flexibility of Residential [47] Wang, D., Parkinson, S., Miao, W., Jia, H., Crawford, C., and Djilali, N., 2013,
Buildings Using Short Term Heat Storage in the Thermal Mass,” Energy, 111, “Hierarchical Market Integration of Responsive Loads as Spinning Reserve,”
pp. 991–1002. Appl. Energy, 104, pp. 229–238.
[18] Makarov, Y. V., Reshetov, V. I., and Stroev, V. A., 2005, “Blackout [48] Finck, C., Li, R., Kramer, R., and Zeiler, W., 2018, “Quantifying Demand
Prevention in the United States, Europe, and Russia,” Proc. IEEE, 93(11), Flexibility of Power-to-Heat and Thermal Energy Storage in the Control of
pp. 1942–1955. Building Heating Systems,” Appl. Energy, 209, pp. 409–425.
[19] Xu, B., Dvorkin, Y., Kirschen, D. S., Silva-Monroy, C. A., and Watson, J., 2016, [49] Alimohammadisagvand, B., Jokisalo, J., Kilpeläinen, S., Ali, M., and Sirén, K.,
“A Comparison of Policies on the Participation of Storage in U.S. Frequency 2016, “Cost-Optimal Thermal Energy Storage System for a Residential Building
Journal of Engineering for Sustainable Buildings and Cities FEBRUARY 2020, Vol. 1 / 011007-11