SAND2016-11254C
Application of a Uniform Testing Protocol for
Energy Storage Systems
David Rosewater Paul Scott
Sandia National Laboratories TransPower USA
Albuquerque, New Mexico Poway, California
Email: dmrose@sandia.gov Email: paul@transpowerusa.com
Abstract—Methods for bench-marking and comparison can
either limit or accelerate the adoption of emerging energy storage
technologies on the grid. This paper assess the efficacy of the
methods in the US DOE Protocol for Uniformly Measuring and
Expressing the Performance of Energy Storage to in order to
remove barriers to the technology’s acceptance. The protocol
was developed through a multi-year collaborative, government-
industry process and enables standardized data collection to
fairly compare different technologies for energy storage applica-
tions. We apply the protocol to a 1 MW rated lithium-ion battery
system to provide this critical assessment. Field experience and
data will be invaluable to standards development organizations
as they begin to consider these methods for codification.
Index Terms—Energy Storage, Testing, Batteries, Standards
activities
N OMENCLATURE
• BMS - Battery Management System
• CAN - Controller Area Network
• CEC - California Energy Commission
• DOE - US Department of Energy
• DOE Protocol - US DOE Protocol for Uniformly
Measuring and Expressing the Performance of
Energy Storage
• ESTP - Energy Storage Test Pad
• EUT - Equipment Under Test
• HMI - Human Machine Interface
• Hz - Hertz (or kHz - Kilohertz)
• MW - Megawatts (or kW - Kilowatts)
• ppm - parts-per-million
• p.u. - Per Unit
• RMS - Root Mean Squared
• RTE - Round Trip Efficiency
• RTO - Regional Transmission Operator
• SDO - Standards Development Organization
• SOC - State of Charge
• σ - Standard Deviation
Surya Santoso
School of Electrical and
Computer Engineering
University of Texas at Austin
Austin, Texas
Email: ssantoso@mail.utexas.edu
I. I NTRODUCTION
With the increasing role of renewables and aging grid
infrastructure, the need to enhance the stability, reliability, and
efficiency of the electric grid has become critical. While the
value of deploying energy storage systems for this purpose in
the grid is widely recognized, so far energy storage integration
has been limited [1]. Projections indicate a growing role for
energy storage in grid [2] and hence there is a pressing
need to better understand how energy storage can provide
grid services. Consistent metrics for system performance are
useful for comparing different energy storage technologies and
for matching each technology to the appropriate set of grid
services that it can provide.
To obtain consistent performance metrics, consistent meth-
ods are required. IEEE Standard 1679 contained guidance
on many of the methods of energy storage testing [3]. The
Protocol for Uniformly Measuring and Expressing the Per-
formance of Energy Storage [4] was developed to continue to
build agreement over industry best practices for testing energy
storage technologies. The Protocol contains procedures for
administering reference performance tests on energy storage
systems to derive capacity, efficiency, responsiveness, stand-
by losses, and self discharge rate. Additionally, application
specific duty-cycle performance tests are provided for a num-
ber of grid services including frequency regulation. Frequency
regulation is a grid service that helps a utility or Regional
Transmission Operator (RTO) manage the moment-to-moment
differences between load and supply in their areas [5]. Each
application specific test in the Protocol includes a duty-cycle
which has been assessed to represent of the demands of that
service. The derivation of the frequency regulation duty cycle
is explained in depth in [6] and includes a mix of nominal and
extreme operation.
We apply the frequency regulation section of the DOE
Protocol to a 1-MW lithium-ion battery system. A preliminary
report on this system’s performance was made in [7]. Here
we present a detailed investigation of how the DOE Protocol
was applied in order to aid standards developers. As with
any test procedure, adaptations and accommodations must
be made to the procedure to collect the data required to
measure performance. Similarly, the system itself must be
designed and configured in such a way to enable accurate,
time-synchronized data to be collected in a uniform manner.
This paper presents a deep dive on these challenges in order Step
to inform the discussion around performance testing standard
1.0
development. Our data and observations will be useful to Stan-
1.1
dards Development Organizations (SDOs) seeking to codify
1.2
methods for energy storage performance testing.
The rest of this paper is organized as follows: Section II 1.3
explains the relevant parts of the DOE Protocol and what
modifications are done to its procedure to apply it to the 1.4
equipment under test. Section III describes the laboratory 2.0
setup used to perform the experimental regimen including: the 2.1
Energy Storage Test Pad (ESTP), the Equipment Under Test
(EUT), and the control/instrumentation configuration. Section 2.2
IV provides the data collected from the EUT during each test 2.3
and the resulting calculated performance values. Section V
2.4
discusses the qualitative and quantitative experience developed
2.5
through application of the DOE Protocol. Lastly, Section VI
2.6
summarizes this research and explores what conclusions can
3.0
be drawn.
3.1
II. M ETHODOLOGY 3.2
3.3
This section explains how the test procedures and calcula-
tions outlined in the DOE Protocol were adapted to measure 3.4
and express the performance of the equipment under test.
TABLE I
T EST P ROCEDURES
Parameter Measurement Procedure
Stored Energy Capacity Test (adapted from [4])
Charge the EUT to full state of charge
Discharge the EUT to its minimum state of charge level at full
rated power, measure and record the energy delivered by the EUT
Charge the system to full state of charge, measure and record the
energy absorbed by the EUT
Repeat Steps 1.2 and 1.3 at 1/2 rated power
Response Time and Ramp Rate Test (adapted from [4])
Precondition the EUT into a state of active standby (power set to
0kW)
Charge or discharge the EUT to near 50% state of charge
Discharge the EUT at 25% rated power, measure and record the
command time-stamp and AC current
Return the system to active standby
Repeat steps 2.3 and 2.4 for 100% rated charge power
Repeat steps 2.3 through 2.5 twice
Reference Signal Tracking Test (adapted from [4])
Charge the EUT to full state of charge
Discharge the EUT at 1/2 rated power to 50% state of charge
Apply the time-series of commands for the frequency regulation
duty-cycle
Once the duty-cycle is complete charge the EUT at 1/2 rated power
to full state of charge

A. Test Procedures
The adapted procedures used here are shown in Table I.
Changes to the Stored Energy Capacity Test from how it is
described in the Protocol include: one capacity cycle (instead
of five), and two power levels, 100% and 50% of rated power
(instead of four, 100%, 75%, 50%, and 25%). This subset was
selected to fit the limitations of the scope of the study. The
Response Time and Ramp Rate Test was adjusted by repeating
the experiment three times and averaging the results. The
Reference Signal Tracking Test procedure was adapted to use
only one representative ‘average’ 2-hour duty-cycle (instead of
a 24-hour duty-cycle). This shortened duty cycle was selected
because the full duty cycle is energy-neutral (equal charge and
discharge energy) whereas the system requires more charge
than discharge to maintain its SOC. The two hour cycle
was short enough to allow the system to stay in a normal
SOC range throughout. Each of these procedures were to
be applied in accordance with the equipment manufacturer’s
specifications and operating instructions.
B. Performance Calculation
Many performance metrics are important when bench-
marking an energy storage system. Two of the most funda-
mental are capacity and efficiency. Capacity is defined as the
energy which can be supplied by the system, at a given rate,
before it must be recharged [4]. This calculation is shown in
Equation 1. The charge capacity, shown in Equation 2, is then
defined as the maximum energy which can be absorbed before
the system must be discharged [4]. Round Trip Efficiency
(RTE), Equation 3, is then defined as the ratio of these two
capacities [4].
1X
n
W hd = W hdi (1)
n i=1
n
1X
W hc = W hci (2)
n i=1
Pn
W hdi
RTE = Pi=1
n (3)
i=1 W hci
Communication Latency is defined in the DOE Protocol as
the time between receiving the power command and starting to
change its power output [4]. Ramp Rate is defined in the DOE
Protocol as the settling time divided by rated power (settling
time is from when the system starts to respond to when it
settles within 2% of its rated power) [4]. As the exact time
that the system starts to respond can be difficult to measure
we modify this defamation of Communication Latency to be
the time from sending the command to when system power
exceeds 2% of rated power. Similarly, Ramp Rate is defined
here as the average rate of change in system power between
10% and 90% of the set point [8]. These changes make
measurements more robust to noise and hence more consistent.
For grid service specific performance the protocol uses
Duty-Cycle Round-Trip Efficiency, signal tracking Squared Er-
ror, Absolute Error, Energy Error, and Percent Time the Signal
is Tracked. Duty-Cycle Round-Trip Efficiency is calculated
the same as round trip efficiency, using Equations 1, 2, and TABLE II
3, except the duty cycle is used as the control signal. In the S TORED E NERGY C APACITY T EST R ESULTS
protocol the error metrics are not normalized to either the Metric Full Rated Power 1/2 Rated Power
size of the system or the sample rate of the data acquisition Performance Performance
making the results difficult to compare to other systems tested Max Power Discharge 990 kW 502 kW
by other laboratories. For this reason, this paper uses the Duration Max Discharge 219 seconds 2305 seconds
following additional service specific duty-cycle performance Discharged Energy 383 kWh 409 kWh
metrics: Tracking error RMS, Tracking error RMS %, and Alt. Max Power Charge 780 kW 543 kW
% of Time Signal is Tracked. The formulas for these metrics Charged Energy 433 kWh 468 kWh
are shown in Equations 4, 5, and 6. Where PSignal is the Auxiliary load 13.24 kW 15.04 kW
commanded power, PESS is the instantaneous energy storage Round Trip Efficiency* 82.9 % 81.6 %
power output, N is the number of points in the time record,
Measurement accuracy ±5.1kW, *Includes auxiliary load
and Prated is EUT rated discharge power.

control equipment and software allowed operators to automate

some of the EUT’s functions. This equipment sent open loop
Tracking Error RMS =
rP commands to the EUT over the CAN bus and was used during
(PSignal (k) − PESS (k))2 the Response Time and Ramp Rate Test and the Reference
(4) Signal Tracking Test which required time synchronized data
N
for commands and system response.
AC Voltage data were collected from both the 3-phase,
Delta, 480V grid power connection bus, as well as the 3-
Tracking Error RMS % = phase, Y, 208V house power used to supply the lights, battery
qP
(PSignal (k)−PESS (k))2
management, battery rack fans, and inverter coolant pump. AC
N current data were collected from each phase of all four of the
(5)
Prated system’s power inverters, each phase of the air conditioner,
and each phase of the house power (house power and air
conditioner are considered auxiliary loads). All voltage mea-
surements were calibrated to within 0.1% of nominal voltage
Alt. % of time signal is tracked = while all current measurements were calibrated to within 0.5%
PN (PSignal (k)−PESS (k))
< 2% of full-scale current. For the Stored Energy Capacity and
k=1 Prated
(6) Reference Signal Tracking Tests data were collected at 24
N kHz and averaged for recording to 1 sample per second (1Hz).
III. E XPERIMENTAL S ETUP For the Response Time and Ramp Rate Time Test data were
Experimental data were collected at the Energy Storage collected and recorded at 12.5 kHz with a clock accuracy
Test Pad (ESTP) located at the Sandia National Laboratories within 50 ppm. Assuming that errors in current, voltage, and
in Albuquerque New Mexico US. The ESTP is a laboratory time measurements are independent, it can be calculated that
designed for experimentation on large grid connected energy power measurement accuracy was within ± 0.51% of full scale
storage systems. More information on the ESTP and its (5.1 kW) for the results described in Section IV.
capabilities can be found at www.sandia.gov/batterytesting/.
IV. R ESULTS
A. Equipment Under Test (EUT) A. Stored Energy Capacity Performance
The TransPower GridSaver was commissioned by the Cal- Figure 1 shows the data collected from the EUT during the
ifornia Energy Commission (CEC) as part of their public Stored Energy Capacity Test. Both cycles started at 100% SOC
interest energy research program. A full description of the and self-limited each battery string’s power output according
system can be found in the final project report to the CEC to their lowest voltage cell and highest cell temperature. Once
[9]. the end-of-discharge conditions were reached, (all DC string
currents had reduced below 10A at minimum cell voltage),
B. Control and Instrumentation
the system was set to recharge until reaching end of charge
The Equipment Under Test (EUT) was primarily controlled conditions. These data where then processed according to
through its Human Machine Interface (HMI). This HMI al- Equations 1, 2, and 3. The performance metrics calculated
lows system operators to send real power commands, view for these conditions are shown in Table II.
system data including warnings and errors, and change some
automated functions, such as voltage and temperature limits. B. Response Time and Ramp Rate Performance
In addition to the HMI, commands and data were available The data collected during the Response Time and Ramp
over a Controller Area Network (CAN) bus. Custom-designed Rate Test on one of four battery strings in the EUT is
 Frequency Regulation Performace Test (2 hour Average Duty Cycle)
Capacity Cycle Test Data 1000
System Total Power
1000 Command Signal
Half Rated Power 800
Total Aux Load
800 Full Rated Power Error
600

600 400

Power (kW)
400 200

0
200
Power (kW)

-200
0
-400
-200
-600

-400
-800
0 0.5 1 1.5 2 2.5 3 3.5
-600 Time (hours)

-800
Fig. 3. Frequency Regulation Duty-Cycle Test Data
-1000
0 0.5 1 1.5 2
Time (hours)
100
SOC Average
Fig. 1. Stored Energy Capacity Test Data 80

State of Charge (%)

60
SOC max = 48.7%
1.5
40
Per unit power out (single inverter)

Discharge Response
1 Charge Response SOC min = 31.2%
20
0.5
0
0 0 0.5 1 1.5 2 2.5 3
Time (hr)
-0.5
Fig. 4. State of Charge During Duty-Cycle
-1

-1.5
0 0.2
Fig. 2. Response Time and Ramp Rate Test Data (Normalized and Filtered)
shown in Figure 2. Charge and discharge tests were performed
separately and potted together. Instantaneous power was cal-
culated from three-phase voltage and current. To clean up the
instantaneous power signal for analysis, a 5th order digital
Butterworth low-pass filter with natural frequency ωn = 0.01
for sample rate = 12.5 kHz was applied. Charge power and
discharge power were normalized separately to 1.0 and -1.0
p.u. respectively. Figure 2 shows fully processed response
characteristics for 100% charge and discharge power for one
of the three tests. In this plot, t = 0 represents the time when
the EUT revives a command. The result of the performance
metric calculations are shown in Table III.
R ESPONSE T IME AND R AMP R ATE T EST R ESULTS
Metric
Mean Communication Latency (Tr )
σ of Communication Latency
Mean Ramp Rate (Prr )
σ of Ramp Rate
0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
Time (s) C. Reference Signal Tracking Test Performance
The data recorded during the Reference Signal Tracking
Test is shown in Figure 3. The calculated error metrics are
shown in Table 4. Additionally, SOC was recorded by the
system’s BMS during the experiment and is shown in Figure
4. The maximum and minimum SOC during the duty-cycle and
the change in SOC over the duty cycle were then determined
from these data. The change in state of charge (∆ SOC) was
found to be -11.22%. Note that the BMS did not record the
SOC returning to 100%. The wide difference between the
highest cell voltage and the lowest cell caused all four strings
highest cell voltages to reach their maximum charge voltage
when the systems average SOC was only 70.96%.
V. D ISCUSSION
Applying the protocol to a 1-MW energy storage system
developed a better understanding of the requirements the
protocol. Three observations are discussed here along with
recommendations for improvements to the Protocol.
TABLE III First, the EUT must be designed and configured to respond
to an external command signal. Control schema for energy
storage systems vary widely and depend on their indented
Performance
applications. For example a system designed for peak shaving
1.079 s
could respond to a clock while a system designed for voltage
0.027 s
regulation could respond to voltage. As the protocol is written
2.12 MW/s
now, it does not fully account for these alternate control
0.08 MW/s mechanisms when testing systems. A full description of how
 TABLE IV ACKNOWLEDGMENTS
R EFERENCE S IGNAL T RACKING T EST R ESULTS
This work was funded by the US DOE OE’s energy storage
Metric Performance program. The authors would like to thank Dr. Imre Gyuk at
Discharge Energy 567 kWh the DOE for his support of research advancing the industry
Charge Energy 638 kWh acceptance of grid energy storage, Mike Gravely at the CEC
Auxiliary load 12.11 kW for his support of this work, and the whole team at TransPower
Duty cycle RTE 83.6% for the tremendous effort that went into their prototype system.
Sum of squared error 3,646,416 kW2 Sandia National Laboratories is a multi-program laboratory
Sum of absolute error 60,491 kW
managed and operated by Sandia Corporation, a wholly owned
Sum of energy error 381,561,167 kWh
subsidiary of Lockheed Martin Corporation, for the U.S.
% of time signal is tracked 24.5%
Department of Energy’s National Nuclear Security Adminis-
*Tracking Error RMS 22.5 kW
tration under contract DE-AC04-94AL85000.
*Tracking Error RMS % 2.3 % R EFERENCES
*Alt. % of time signal is tracked 73.5%
[1] A. Akhil, “DOE/EPRI 2013 Electricity Storage Handbook in Collabora-
*Additional metrics not in the DOE Protocol tion with NRECA,” Sandia National Laboratories, Tech. Rep., 2013.
[2] M. Hand, S. Baldwin, E. DeMeo, J. Reilly, T. Mai, D. Arent,
G. Porro, M. Meshek, and D. Sandor, “Renewable Electricity Futures
Study, Volume 2: Renewable Electricity Generation and Storage
to implement a control signal for each test would increase the Technologies,” National Renewable Energy Laboratory, Tech. Rep.,
versatility of the protocol for wider adoption. 2012, NREL/TP-6A20-52409. [Online]. Available: http://www.nrel.gov/
Second, the system must have a accurate estimate of its analysis/re futures/
[3] “Ieee recommended practice for the characterization and evaluation of
SOC. SOC estimations provided by battery managements emerging energy storage technologies in stationary applications,” IEEE
systems have may not designed or configured to account for Std 1679-2010, pp. 1–38, Oct 2010.
the specific system or use conditions of the test. This can cause [4] D. Conover, S. Ferreira, A. Crawford, D. Schoenwald, J. Fuller, D. Rose-
water, S. Gourisetti, and V. Viswanathan, “Protocol for Uniformly Mea-
difficulty in applying the test routines to a device as setting a suring and Expressing the Performance of Energy Storage Systems,” San-
system to 50% SOC may not guarantee adequate operational dia National Laboratories and Pacific Northwest National Laboratories,
margin to complete a test. Tech. Rep., 2016.
[5] Member Services, “Definitions and acronyms,” PJM, Tech. Rep., 2014.
Last, reported metrics should be normalized to account for [Online]. Available: http://www.pjm.com/∼/media/documents/manuals/
different laboratories. Normalized versions of sum of squared m35.ashx
[6] D. Rosewater and S. Ferreira, “Development of a frequency regulation
error and sum of absolute error, including RMS error and duty-cycle for standardized energy storage performance testing,” Journal
RMS% error, are easier to interpret and compare across energy of Energy Storage, vol. 7, pp. 286 – 294, 2016. [Online]. Available:
storage technologies. Similarly, the requirement for the % http://www.sciencedirect.com/science/article/pii/S2352152X16300536
[7] P. Scott and D. Rosewater, “Energy storage used for frequency regula-
Time the signal is tracked metric, as it is currently normalized tion and grid firming,” in Energy, Utility and Environment Conference
to the instantaneous requested power, is nearly impossible to (EUEC), February 2016.
satisfy at low power levels. This causes a misleading result [8] N. S. Nise, Control Systems Engineering 4th Edison. John Wiley and
Sons, 2004.
for how closely the system follows the duty-cycle which can [9] P. Scott and M. Simon, “Energy Research and Development Division,
be corrected for by normalize to the rated power instead. FINAL PROJECT REPORT, UTILITY SCALE ENERGY STORAGE,
Grid-Saver Fast Energy Storage System,” Prepared for the California
Energy Commission, Tech. Rep., 2015.
VI. S UMMARY /C ONCLUSION

The Protocol for Uniformly Measuring and Expressing the

Performance of Energy Storage enables fair bench-marking
and comparison of different storage technologies. We applied
the protocol to a 1 MW rated energy storage system in
order to collect application experience and data that will help
standards development organizations to adapt and adopt the
protocol with confidence and clarity. The equipment under
test had to be adapted to the protocol’s requirements and the
Protocol had to be adapted to the system’s constraints. This
lead to the development of four concrete recommendations
for improvements to the Protocol concerning: addition of
guidance for configuration of the command signal, addition
of explicit state-of-charge accuracy requirements, and three
new proposed metrics for reference signal tracking error. As
standards developers consider the provisions in the Protocol
this information will help guide their assessment.