Review of Methods of US DOE 2021

THE NATIONAL ACADEMIES PRESS
This PDF is available at http://nap.edu/25992 SHARE

   
Review of Methods Used by the U.S. Department of Energy in

Setting Appliance and Equipment Standards (2021)
DETAILS
142 pages | 8.5 x 11 | PAPERBACK

ISBN 978-0-309-68545-0 | DOI 10.17226/25992
CONTRIBUTORS
GET THIS BOOK Committee on Review of Methods for Setting Building and Equipment Performance
Standards; Board on Infrastructure and the Constructed Environment; Division on
Engineering and Physical Sciences; National Academies of Sciences, Engineering,
and Medicine
FIND RELATED TITLES
SUGGESTED CITATION
National Academies of Sciences, Engineering, and Medicine 2021. Review of

Methods Used by the U.S. Department of Energy in Setting Appliance and
Equipment Standards. Washington, DC: The National Academies Press.
https://doi.org/10.17226/25992.

Visit the National Academies Press at NAP.edu and login or register to get:
– Access to free PDF downloads of thousands of scientiﬁc reports

– 10% off the price of print titles
– Email or social media notiﬁcations of new titles related to your interests
– Special offers and discounts

Distribution, posting, or copying of this PDF is strictly prohibited without written permission of the National Academies Press.
(Request Permission) Unless otherwise indicated, all materials in this PDF are copyrighted by the National Academy of Sciences.
Copyright © National Academy of Sciences. All rights reserved.

Review of Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards
REVIEW OF METHODS USED BY THE U.S.

DEPARTMENT OF ENERGY IN SETTING APPLIANCE
AND EQUIPMENT STANDARDS
Committee on Review of Methods for Setting Building and Equipment Performance Standards
Board on Infrastructure and the Constructed Environment
Division on Engineering and Physical Sciences
A Consensus Study Report of
Copyright National Academy of Sciences. All rights reserved.

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This activity was supported by task order 89243419F-EE400089 of contract DE-EP0000026 with the U.S.
Department of Energy. Any opinions, findings, conclusions, or recommendations expressed in this
publication do not necessarily reflect the views of any organization or agency that provided support for the
project.
International Standard Book Number-13: 978-0-309-68545-0

International Standard Book Number-10: 0-309-68545-1
Digital Object Identifier: https://doi.org/10.17226/25992
Additional copies of this publication are available from the National Academies Press, 500 Fifth Street,
NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2021 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2021. Review of
Methods Used by the U.S. Department of Energy in Setting Appliance and Equipment Standards.
Washington, DC: The National Academies Press. https://doi.org/10.17226/25992.

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by
President Lincoln, as a private, nongovernmental institution to advise the nation on issues
related to science and technology. Members are elected by their peers for outstanding
contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the
National Academy of Sciences to bring the practices of engineering to advising the nation.
Members are elected by their peers for extraordinary contributions to engineering. Dr. John L.
Anderson is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970
under the charter of the National Academy of Sciences to advise the nation on medical and
health issues. Members are elected by their peers for distinguished contributions to medicine
and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and
Medicine to provide independent, objective analysis and advice to the nation and conduct other
activities to solve complex problems and inform public policy decisions. The National Academies
also encourage education and research, recognize outstanding contributions to knowledge, and
increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at
www.nationalacademies.org.

Consensus Study Reports published by the National Academies of Sciences, Engineering, and
Medicine document the evidence-based consensus on the study’s statement of task by an
authoring committee of experts. Reports typically include findings, conclusions, and
recommendations based on information gathered by the committee and the committee’s
deliberations. Each report has been subjected to a rigorous and independent peer-review
process and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and

Medicine chronicle the presentations and discussions at a workshop, symposium, or
other event convened by the National Academies. The statements and opinions contained in
proceedings are those of the participants and are not endorsed by other participants, the
planning committee, or the National Academies.
For information about other products and activities of the National Academies, please
visit www.nationalacademies.org/about/whatwedo.

COMMITTEE ON REVIEW OF METHODS FOR SETTING BUILDING AND EQUIPMENT

PERFORMANCE STANDARDS
LINDA COHEN, University of California, Irvine, Chair

CHARLES CULP, Texas A&M University
SUSAN DUDLEY, George Washington University
CLARK GELLINGS, NAE,1 Electric Power Research Institute
W. MICHAEL HANEMANN, NAS,2 Arizona State University
DALIA PATINO-ECHEVERRI, Duke University
ANAND PATWARDHAN, University of Maryland
JAMES SALLEE, University of California, Berkeley
Staff
PEYTON GIBSON, Associate Program Officer, Board on Infrastructure and the Constructed
Environment (BICE)
MARTIN C. OFFUTT, Senior Program Officer, BICE, Study Director
JOSEPH L. PALMER, Senior Project Assistant, BICE
CAMERON OSKVIG, Director, BICE
1
Member, National Academy of Engineering.
2
Member, National Academy of Sciences.

BOARD ON INFRASTRUCTURE AND THE CONSTRUCTED ENVIRONMENT
THOMAS P. BOSTICK, NAE,1 Bostick Global Strategies, Chair

BURCA AKINCI, Carnegie Mellon University
STEPHEN T. AYERS, The Ayers Group, LLC
BURCIN BERCERIK-GERBER, University of Southern California
LEAH BROOKS, George Washington University
JACK DEMPSEY, Definitive Logic
LEONARDO AUGUST DUEÑAS-OSORIO, Rice University
SANJIV GOKHALE, Vanderbilt University
DAVID GOODYEAR, NAE, Independent Consultant
DAVID J. HAUN, Haun Consulting, Inc.
CHRISTOPHER MOSSEY, Fermi National Accelerator Laboratory
ANDREW PERSILY, National Institute of Standards and Technology
CHRIS D. POLAND, NAE, Chris D Poland Consulting Engineer
JAMES RISPOLI, North Carolina State University
DOROTHY ROBYN, Boston Institute for Sustainable Energy
SHARON L. WOOD, NAE, The University of Texas at Austin
Staff
CAMERON OSKVIG, Director

MARTIN OFFUTT, Senior Program Officer
JOSEPH PALMER, Senior Program Assistant
PEYTON GIBSON, Associate Program Officer
1
vi

Preface
The consumption of energy and water in buildings has been the subject of programs, policies, and
financing by government and civil society since the 1970s. Buildings use more than one-third of the final
energy consumed in the United States, the balance taken up by transportation and industry. At the federal
level, the U.S. Department of Energy (DOE) and its predecessor agencies have taken responsibility, along
with other agencies such as the U.S. Environmental Protection Agency (EPA), for designing and
executing programs to support national goals aimed at lowering energy consumption and reducing the
environmental impact of energy use.
This study report of the Committee on Review of Methods for Setting Building and Equipment
Performance Standards is focused on DOE’s program of mandatory standards for some commercial and
residential appliances that dictate minimum energy performance and maximum water consumption.
DOE’s Building Technologies Office (BTO) manages the standards program within a portfolio that
includes research and development, technology validation, field and laboratory testing, decision tools,
market integration, and codes and standards. There are also appliance-labeling programs such as those of
the Federal Trade Commission and of the Energy Star program, which DOE runs jointly with the EPA.
In addition to saving significant energy and/or water, DOE standards are required to be
technologically feasible and economically justified and to not unduly burden either small domestic
manufacturers or low income consumers. This report assesses the analyses BTO conducts when setting
standards for a category of appliance or equipment. These analyses estimate the effectiveness of the
standards at achieving the stated goal, their net impact compared to the baseline of current policies, and
the cost effectiveness with which that goal has been achieved. This report discusses the methods used in
these analyses and considers potentially useful modifications.
The rest of the programs of the BTO portfolio, listed earlier, are known to the committee, and while
the report mentions these non-standards programs to provide context, it does not take a position on them.
Furthermore, in writing the report, the committee has accepted the goals of the standards program as
given by DOE and refrains from making any judgments about them. The report instead answers the
narrower question of how best to conduct the analyses that support the development of minimum energy
and water efficiency standards.
Many individuals volunteered significant time and effort to address and educate the committee during
its public information sessions. John Cymbalski, Manager of the Appliance and Equipment Standards
Program for DOE’s Building Technologies Office, provided several substantive briefings and coordinated
responses to the committee’s written queries. Michael Kido, an attorney with DOE’s Office of the
General Counsel, provided important explanations of the legal underpinning of the program. Numerous
individuals whose work is supported by the BTO provided descriptions of the analytical methods
supporting the standards. They engaged with the committee in valuable discussions. The committee is
grateful to these individuals and to others not named here.
vii

The committee has been pleased to provide DOE with this review of its methods and believes such
periodic reviews to be essential to a robust standards program. The committee hopes the observations,
findings, and recommendations of the report will be embraced by DOE.
Linda Cohen, Chair

Committee on Review of Methods for Setting Building and
Equipment Performance Standards
viii

Acknowledgment of Reviewers
This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse
perspectives and technical expertise. The purpose of this independent review is to provide candid and
critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in
making each published report as sound as possible and to ensure that it meets the institutional standards
for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft
manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following individuals for their review of this report:
Jennifer Baxter, Industrial Economics,

Michelle (Shelby) Bensi, University of Maryland, College Park,
Laura Cozzi, International Energy Agency,
Ahmad Faruqui, Brattle Group,
Kenneth Gillingham, Yale University,
Jason Glazer, GARD Analytics,
Heidi King, Heidi R. King Analytic Consulting,
Cathy Kling, NAS,1 Cornell University,
Phillip Krein, NAE,2 University of Illinois at Urbana-Champaign,
David McCollum, Electric Power Research Institute.

Although the reviewers listed above provided many constructive comments and suggestions, they
were not asked to endorse the conclusions or recommendations of this report nor did they see the final
draft before its release. The review of this report was overseen by Marilyn A. Brown, NAS/NAE, Georgia
Institute of Technology, and Charles D. Kolstad, Stanford Institute for Economic Policy Research. They
were responsible for making certain that an independent examination of this report was carried out in
accordance with the standards of the National Academies and that all review comments were carefully
considered. Responsibility for the final content rests entirely with the authoring committee and the
National Academies.
1
Member, National Academy of Sciences.
2
ix


Contents
SUMMARY 1
1 INTRODUCTION 9
Context: Energy End-Use, 9
Role of the Standards Program, 10
Origin of Study, 10
Charge to the Committee, 11
Committee’s Approach to the Statement of Task, 11
Structure of the Report, 11
References, 12
2 FRAMEWORK FOR DEVELOPING REGULATIONS 13

Introduction, 13
Context—Statutory Requirements, 13
Evolution of Standards, 18
Requirements for Regulatory Analysis, 19
Program Impacts, 26
Criteria and Principles Guiding This Review, 29
References, 36
3 ASSESSING CURRENT MODELS: ENGINEERING AND TECHNOLOGY

COST AND TECHNOLOGY PERFORMANCE 38
Introduction, 38
Screening and Analysis of Design Options, 38
Screening Analysis, 39
Engineering Analysis, 42
Electric Power System, 49
Final Observations, 50
References, 51
4 THE ECONOMIC ANALYSIS OF STANDARDS 52

Introduction, 52
The Effect of Standards on Producers, 53
The Effect of Standards on Consumers, 58
Energy Conserved, 68
Pollutant Emissions from Energy Use, 71
Market Failures That Rationalize Standards, 75
Final Observations, 80
References, 81
ANNEX 4A: Detailed Comments on the Life-Cycle Cost Analysis for Residential Dishwashers 84
xi

5 CONSIDERATION OF ALTERNATIVES TO A NATIONAL STANDARDS PROGRAM 98

Introduction, 98
Energy Efficiency Standards and the Built Environment, 99
Energy Efficiency and Consumer Behavior, 100
Appliance and Equipment Standards and Electric Power Generation, 105
Exogenous Factors That Complement the National Standards Program, 106
Conclusion, 109
References, 109
6 ADDRESSING THE NEEDS OF A CHANGING WORLD 111

Introduction, 111
The Internet of Things, 111
The Integrated Grid, 113
Fuel Switching, 114
Changes in Electricty Regulation and Business Models, 115
Pattern and Amount of Consumption, 116
Further Sources of Information, 118
References, 121
APPENDIXES
A Committee Biographies 125

B Committee Activities 128
xii

Summary
The U.S. Department of Energy (DOE) issues “standards regulations” for energy conservation
pursuant to the Energy Policy and Conservation Act of 1975 (EPCA), as amended, and other authorities.
These standards regulations apply to certain consumer products and commercial and industrial equipment.
These can include air conditioning and heating systems, washing machines, and commercial refrigeration,
among numerous other examples. DOE issues standards regulations by rulemaking and includes
quantitative maximum water and energy use or minimum energy conservation standards. There are
currently standards regulations for more than 70 product classes (i.e., a specific type of consumer product
or commercial or industrial equipment).
This study of the Committee on Review of Methods for Setting Building and Equipment Performance
Standards is a review of the assumptions, models, and methodologies (“analytical methods”) that DOE
uses in setting the quantitative portion of the standards regulations following the Office of Management
and Budget’s (OMB’s) guidance1 on the use of scientific information. The EPCA includes a list of
statutory criteria DOE must consider in setting these standards. In conformance with these statutory
criteria, DOE has evolved a set of analytical methods it currently uses when setting a standard for a
specific product class.
The analyses under review construct cost-efficiency relationships for products in a class to arrive at
quantitative trial standard levels (TSLs) for the product class under consideration. The analyses also
consider downstream effects of different quantitative standards on consumer life-cycle cost, manufacturer
impact, employment impact, and impact on small entities. The committee’s task was to consider the
methods employed in each analysis.
DOE provided the committee with the full suite of analyses for standards regulations for each of the
following three product classes in the Appliance and Equipment Standards Program: (1) residential
dishwashers, (2) commercial refrigeration equipment, and (3) residential furnaces. DOE had developed
the analyses for these product classes to support specific rulemaking efforts (i.e., the administrative
process of setting standards regulations). The three rulemakings provide examples of product categories
in wide deployment. The three product categories are diverse in the type of energy service delivered, the
novel issues they instantiated, and the consumer behaviors of relevance to each. Nonetheless, the
committee found commonalities among them, and many of the committee’s findings and
recommendations proved robust across all three illustrative product classes. DOE also discussed other
standards from its Appliance and Equipment Standards Program—there are more than 70—in response to
the committee’s queries.
1
Office of Management and Budget, 2005, “Final Information Quality Bulletin for Peer Review,” Federal
Register 70: 2664, January 14.

FIGURE S.1 Sequencing and dependencies of analyses. SOURCE: John Cymbalski, DOE, “Appliance
and Equipment Standards Program Buildings Technology Office,” presentation to the committee,
November 19, 2019.
DOE also briefed the committee on its current practices for conducting and sequencing the analyses.
DOE presenters explained the dependencies among the analyses and where the output of one becomes the
input of another (see Figure S.1). The sections below provide a summary of the committee’s observations
on the analyses that concern the hardware (i.e., the product or equipment itself) and the impact on
producers and consumers.
POLICY CONTEXT
DOE establishes its standards regulations in the context of statutory authority and executive orders.
EPCA, as amended, authorizes DOE to issue standards to “achieve the maximum improvement in energy
[or water] efficiency . . . which the Secretary determines is technologically feasible and economically
justified,” which involves evaluating “whether the benefits of the standard exceed its burdens.” In
addition, the Administrative Procedure Act, the Regulatory Flexibility Act, and other laws concerning
administrative process guide DOE’s issuance of regulations. The executive branch exercises its discretion
as well, and the Regulatory Impact Analysis (RIA) framework, created by presidential executive order
and subject also to OMB Circular A-4, provides a structure for analyzing and developing new and
amended standards and for reviewing existing standards. To develop standards consistent with its
statutory and executive authority, DOE has designed the series of analyses depicted in Figure S.1. While
DOE does prepare RIAs for its energy efficiency standards, it does not integrate them into the standard-
setting framework but conducts them as a separate analysis after key decisions have been made. The RIA
framework is used across the federal government, including at agencies with related missions (such as the
EPA, which also seeks to address environmental externalities), so its use would improve transparency and
effectiveness across federal policy. Furthermore, because the RIA framework is consistent with DOE’s
statutory mandate to ensure its standards are technologically feasible and economically justified,
organizing its analysis according to the RIA framework—for example, (1) identifying material failure of

private markets or other compelling public need, (2) analyzing the benefits and costs of alternative
approaches to addressing that need, and (3) evaluating the distribution of those benefits and costs—could
improve the consumer impacts and societal outcomes of DOE’s standards.
To better understand the social impacts of its standards, DOE should organize its analysis
following the regulatory analysis framework laid out in OMB Circular A-4. For example, rather
than presenting the RIA as the last stage in the process, the analyses shown in Figure S.1 could be
technical appendices to the RIA, which would integrate this information in presenting estimates of the
net benefits and distributional impacts of the proposed efficiency standard and reasonable
alternatives, referencing information from relevant appendices as it describes a baseline scenario and
cost and benefit models. (Recommendation 2-1)
DOE should pay greater attention to the justification for the standards, as required by
executive orders and the EPCA requirement that standards be economically justified. DOE
should attempt to find significant failures of private markets or irrational behavior by
consumers in the no-standards case and should consider such a finding as being necessary to
conclude that standards are economically justified. (Recommendation 2-2)
DOE’s regulatory analyses that support tighter standards are often premised upon an energy
efficiency gap—that is, DOE’s engineering analysis demonstrates that readily available technologies save
far more in energy costs than they require in upfront costs. However, the RIA does relatively little to
interpret this fact. Not all interpretations of the energy efficiency gap may imply a market failure. Other
explanations include biases in the cost estimates, a failure to properly account for performance trade-offs,
a failure to account for irreversibility and option value, a failure to account for borrowing constraints, and
mismeasurement of actual energy efficiency.
DOE should place greater emphasis on providing an argument for the plausibility and
magnitude of any market failure related to the energy efficiency gap in its analyses. For some
commercial goods in particular, there should be a presumption that the market actors behave
rationally, unless DOE can provide evidence or argument to the contrary. (Recommendation 4-
13)
DOE should give greater attention to a broader set of potential market failures on the supply
side, including not just how standards might reduce the number of competing firms, but also
how they might impact price discrimination, technological diffusion, and collusion.
(Recommendation 4-14)
SCREENING AND ENGINEERING ANALYSES
The objective of the Screening Analysis and Engineering Analysis is to identify TSLs and develop
cost-efficiency relationships by estimating the manufacturer’s costs of achieving increased efficiency
levels and determining the maximum technologically feasible efficiency level. Overall, the committee
found that the approach DOE used was sound, but it could be improved.
The cost-efficiency relationship developed and used by DOE currently assumes that the only two
determinants to be considered in appliance standards are cost and efficiency. However, changes in
wholesale markets for electricity now necessitate that assessments of standards include an evaluation of
demand response readiness. It has also become important to include a thorough evaluation of the impact
that demand-side management (DSM) programs and activities can have on leveraging a new or revised
standard’s effectiveness.

DOE should consider demand response readiness as a factor in cost-efficiency calculations. This
necessitates the inclusion of power system benefits not currently considered. (Recommendation
3-6)
These additional elements necessitate that the engineering analysis include a more detailed
understanding of the projected energy consumption patterns and amounts for appliances. Elements for
DOE to consider include ranges of costs, patterns of consumption by seasonal time of use (day, week, and
month) and (heterogeneous) pattern of consumption by user, diversity factors, energy peak demand, and
variance regarding environmental factors. DOE often uses engineering estimates to develop these data.
The energy consumption patterns and amounts that are projected for an appliance will need to include
results by seasonal time-of-use (day, week, and month); these projections will be important for the RIA as
well. The use of actual test data, including experience from previously implemented standards in place of
computer simulation or to validate computer simulations, will improve the analytical outcomes.
DOE should expand the cost analysis segment of the engineering analysis to include ranges of
costs, patterns of consumption, diversity factors, energy peak demand, and variance regarding
environmental factors. (Recommendation 3-5)
DOE screens out options during its analysis supporting the Notice of Proposed Rulemaking that are
technologically infeasible or impractical. It also screens out options that will adversely impact society or
the product utility and availability. The current screening analysis is conservative and may force the
exclusion of some otherwise desirable technology options. The most effective appliance efficiency
standards would encourage the development and eventual adoption of innovative technology. Other
agencies use alternative methods in their own technology evaluations for identifying a technology’s
readiness, such as the technological readiness level (TRL) taxonomy developed originally by the National
Aeronautics and Space Administration (NASA).
DOE should consider technologies that are at early, pre-competitive technological readiness
levels and have promise for use in consumer products and commercial/industrial equipment as
part of product population analyzed, even if it seems plausible that they will be screened out in
later stages of the analysis, such as in the screening analysis made during the Notice of
Proposed Rulemaking. DOE should continue to use the tools at its disposal, such as
reconsidering a previously excluded technology, to avoid prematurely screening-out innovative
technologies. (Recommendation 3-1)
DOE often uses engineering analysis as a substitute for monitoring actual performance of new or
improved appliances. Field tests of a cohort of buildings to test alternative technologies would be one way
to ensure optimal evaluation.
DOE uses point estimates of efficiency levels, but there is an opportunity to characterize the
uncertainty of the efficiency ratings as experience in deployment for a given appliance. Performance can
depend on characteristics of the environment within which the appliance or equipment is installed and on
how homeowners or commercial businesses use the appliance. Thus, rather than providing a “point”
estimate of efficiency, DOE could provide a range that reflects the variability in energy consumption
under different uses. In the future, sensor data may provide great insight into use patterns that allow DOE
to understand impacts on low-frequency users of the technology and therefore improve distributional
impact data.
The ongoing explosion in the use of sensors, computational ability, and communications creates an
ever-increasing library of consumption data at the appliance, building, and power system level. A large
percentage of U.S. residential electricity consumers now have smart meters that can record and
communicate consumption data. Many new appliances are web-connected and incorporate added
functionality to their products that permit the manufacturer to monitor and manage consumption. This

data influx is an opportunity. However, some of these data may generate privacy concerns that will need
to be addressed.
WELFARE ANALYSIS OF CONSUMERS AND PRODUCERS
The economic analysis translates changes in the costs and attributes of appliances and equipment into
changes in seller profit, consumer well-being, and environmental quality. The committee’s charge
focused on the methods employed in this portion of DOE’s analysis.
The committee’s report notes various ways that DOE’s analysis deviates from an ideal economic
analysis and offers numerous concrete steps for incremental improvement over a range of topics. But,
even more, the report concludes that there is a great deal of uncertainty that cannot be resolved because
economic models are ambiguous on key points, because research on other key points is unsettled, and
because DOE lacks necessary data. Thus, the committee’s broadest recommendations pertain to (1)
quantifying and presenting uncertainty and (2) collecting additional data.
Regarding uncertainty, DOE’s economic analysis treats uncertainty around key parameters
inconsistently and in segmented ways. The committee advocates a more systematic and integrated
representation of uncertainty that propagates uncertainty throughout the analysis. It also focuses on
parameters most likely to be pivotal to decisions and makes sharper distinctions between variability and
uncertainty. Finally, the committee provides a recommendation for DOE to present a complete picture of
the consequences of alternative standards to the final decision-makers.
In order to evaluate the economic costs and benefits of a standard, DOE should present the
distribution of costs and benefits estimated in its models when (1) uncertain parameters are
represented by probability distributions and (2) parameters that vary across geographic and
other relevant dimensions are disaggregated. The uncertainty or variability the parameters
represent should be compounded or propagated—properly accounting for any correlations—
throughout the calculation. This methodology is necessary for the markup analysis and
manufacturer impact analysis, the shipments analysis, and all components of the life-cycle cost
analysis. Where multiple sources of uncertainty must be combined for the final benefits result,
as with net benefits depending on both the shipments analysis and the appliance unit cost and
performance, the subcomponents should be reported as well. (Recommendation 4-15)
Regarding additional data, the challenge is in providing detailed, forward-looking cost-benefit

analysis in markets that deviate in significant ways from the benchmark model of rational, competitive
markets, and about which there is frequently a lack of hard data. True improvements are likely to come
from gathering and using new data, rather than from grafting more sophisticated economic models into
the core of the analysis.
Several new data types could prove invaluable, including better information on product prices and
markups (to ground models of seller behavior); more information about consumer preferences (to
quantify attribute-energy trade-offs) and usage (to better model variability); as well as measures of in situ
product performance (to more accurately measure energy consumption of products as deployed and used).
Perhaps most important, given the extent of uncertainty in how these markets operate, how new
generations of appliances operate and how technological change will impact both supply and demand, is a
greater store of ex post analysis (i.e., after the standards have had time to take effect) that validates
assumptions made in prior standards and evaluates the implications of prior forecasts’ inaccuracies and
mistakes.
DOE should obtain better data for improving the economic analyses of appliance and
equipment performance standards. Empirical data are needed on markups, consumer choices
in appliance markets, and in situ performance. Some of this information can come from

relatively simple changes to current surveys and studies, including engineering analyses of the
Appliance and Equipment Standards Program and the Residential Energy Consumption
Survey. (Recommendation 4-16)
Conducting an evaluation, ex post, of the performance of appliances and equipment can permit
evaluation of the validity of the ex ante assumptions relied upon in setting standards in the first instance
and will allow learning and iterative improvement of the analysis. This can be significant when economic
models require unverifiable assumptions, critical inputs are highly uncertain, or relevant data are
unavailable.
DOE should commit to collecting data necessary to conduct more rigorous ex post analysis of
the effects of standards on consumers, producers, energy consumption, and environmental
impacts. (Recommendation 2-3) Data are in particular needed on prices and markups
(Recommendation 4-1), consumer choices (Recommendation 4-3), and in situ performance
(Recommendation 4-8).
Ex post analyses can validate assumptions made in prior standards and evaluate the
implications of prior forecasts’ inaccuracies and mistakes. DOE should use such ex post
analyses routinely to improve forward-looking standards iteratively. (Recommendation 4-17)
An integral part of the welfare analysis for energy efficiency programs is the quantification of
emissions reductions. While DOE does well to monetize the changes in emissions quantities using the
most recent outside studies and government reports on the economic and health damages associated with
emissions, it also appears not to monetize some pollutants, nor does it represent or communicate all of the
uncertainty and variability in these economic numbers. The problem is complex as emissions reductions
critically depend on other public policies. For example, the extent of penetration of renewable generation
in the electric grid, whether pollution caps are binding for emissions subject to cap-and-trade programs or
the implementation of demand-side management or demand response programs. However, because of the
importance of emissions reductions to evaluating prospective standards, the committee recommends that
DOE expand its valuations of emissions reductions.
DOE should monetize all emissions impacts for which meaningful estimates of social impacts
are available, considering policy interactions when those interactions are deemed significant.
(Recommendation 4-12)
CONTEXT OF ENERGY CONSERVATION STANDARDS
DOE considers alternatives to any new or revised appliance and equipment standards when drafting
the RIA that accompanies the rulemakings by which it issues new or revised standards. These alternatives
are not standards regulations but rather involve other types of government interventions related to, for
example, taxes or rebates that affect consumer choice; voluntary targets perhaps combined with
information campaigns; government purchases; and the no-action alternative. The RIA presents an
opportunity to estimate the effectiveness of other means of achieving the goals of the Appliance and
Equipment Standards Program. For example, there are a number of attributes sought by the consumer
beyond low energy costs, including maintaining comfort, safety, convenience, environmental impact,
security, control, appearance, and high-tech attributes.
The RIA process for the three rulemakings assessed by the committee evaluated DSM elements
related to consumer rebates and consumer and manufacturer tax credits as alternatives to standards.
Analyzing these elements as an integrated program, rather than individually, would more adequately

assesses the DSM program’s effectiveness as a complementary effort to the Appliance and Equipment
Standards Program.
A further consideration is the impact of a standards program on the load shape (purchase pattern of
demand). To understand such impact, an estimate can be made through a capacity expansion and
production costing model to understand the impact on the need for capacity and the cost of electricity.
RECOMMENDATION 5-1: The RIA should be grounded in an appreciation of consumer

needs beyond reduced energy costs. The RIA should analyze not only the impact of appliance
and equipment standards but also the additional impact of measures within a framework that
includes the following:
 The portfolio of voluntary elements of demand-side management; and
 Impact of standards on the overall power system external and internal to the building
and of the natural gas supply and transmission and distribution.
FUTURE STANDARDS REGULATIONS
There are many trends in technology, regulation, and business models that will influence the structure
and effectiveness of appliance efficiency standards, including information and communications
technology (ICT) enabling the Internet of Things (IoT); the evolution of the integrated grid; the move
toward electrification; and regulatory changes involving electricity markets that can transform the role of
appliance standards in achieving energy efficiency. This transformation is an opportunity for DOE to
develop a standards program that encompasses the entire scope of energy utilization—appliances, their
installation, controls, and the built environment.
The IoT connects smart meters, smart homes, and smart buildings, providing increased visibility into
real-time electricity demand. This connection may directly impact efficiency standards. Utilities,
consumers, and electricity suppliers can use the information to match generation resources accordingly.
Vice versa, smart meters and mobile apps will provide consumers with real-time pricing of electricity.
Consumers can match their use of electricity to reduce their cost, which reduces peak demand and levels
out the consumption profile, providing both more efficient generation and use of electricity. These
features can dramatically affect the impact of energy efficiency standards.
In the future, DOE may need to modify appliance standards’ applicability because of emerging
integrated grids, regardless of how widely distributed or centralized they are. In the integrated grid
concept of operations, appliances must be flexible enough to optimize the value of a combination of local
generation and energy storage, along with energy efficiency and new uses of electricity integrated with
central generation and storage. In this new integrated grid, energy efficiency includes both appliances and
the buildings or systems that house them as distributed resources.
Appliance and equipment standards are ideally never a factor constraining or delaying technologies
that add flexibility to electrical loads. Information must be available regarding estimates of energy
savings at different times of day and seasons, and the value of those savings under different assumptions
on future penetration of renewable energy.
There are many changes in the U.S. electricity marketplace that will affect DOE’s decisions when
creating prudent standards. One change is “fuel switching,” or programs created to encourage consumers
to electrify. For example, California enables municipalities to adopt building codes that prohibit the use of
natural gas in buildings.
The IoT provides the framework that may allow DOE to shift its focus from the efficiency of an
individual appliance in ideal conditions to a broader focus that includes installing and operating the
appliance and its actual demand and pattern of usage governed by consumer behavior.
Preparing for the future will require collecting a broader array of data and information on overall
energy consumption. Disaggregated data allows energy system planners, regulators, and policymakers to

best judge equitable policies, adjudicate tariffs for regulated entities, and plan critical community
infrastructure.
The energy industry is increasingly exploring the use of analytics to mine value from customer data.
These data may lead to the ability to modify methods for setting and evaluating appliance and equipment
standards, resulting in a standards program that could look very different than it does today.
Restructured electricity markets seem inevitable, evidenced by the numerous states that have adopted
some pieces of restructured markets. When setting or revising the appliance and equipment standards,
DOE can ensure that standards do not slow or preclude the development of these markets. However,
while such markets exist today, certain market barriers still exist. These barriers—including a lack of
information, the absence of qualified installation contractors, and limited financing options—inhibit
energy consumers from fully comprehending all cost-effective energy-efficient products and services.
CLOSING
The current Appliance and Equipment Standards Program includes a sequence of analyses good at
finding a standards level that can be met by current technology. However, DOE can continue to meet its
statutory obligations under EPCA, as amended, and improve the analytical methods applicable to its 70-
plus categories of appliances and equipment subject to the energy and water conservation standards.
The lack of empirical data make improving the analysis a challenge, and changes to existing surveys
could improve this, as would moving away from the use of point estimates. Ex-post analysis of the costs
and benefits of the standards would both provide a check on the assumptions made when the rulemaking
was promulgated and offer a guide on how to improve future standards. The extent of uncertainty in
estimating costs and benefits of proposed standards underscores the value of analyzing the underlying
market conditions that suggest standards regulation will be beneficial. To that end, framing the analyses
using the RIA principles is a useful refinement. Finally, a more forward-leaning analysis might capture
more of the potential for the leading edge of technology to enter the market and thereby increase the
energy and water conservation possible.

Introduction
CONTEXT: ENERGY END-USE
Modern forms of energy can be consumed at the point of end-use to provide numerous energy
services such as space conditioning, illumination, hygienic services, cooking, and food preservation.
Occupants of architectural spaces—residential and commercial—with some exceptions rely on one or
more grid fuels to provide these energy services. The principle grid fuels are natural gas and electricity.
(An example of a less common grid “fuel” is district heating.) The electrification movement in the 20th
century proved to be near-universal, while natural gas has been less thorough in its penetration. In
circumstances where the build-out of grids has not occurred, various other fuels provide the chemical
energy needed for energy services.
Among grid-based options, electricity is a premium fuel noted for the ease with which it can be
transmitted long distances and distributed; its suitability for devices having flexible operating
characteristics such as turn-down, variable speed, and portability; its grid topology and network
characteristics that allow it to be distributed to terminal points of end-use that can in turn feed-in power
back to the grid; and the variety of prime movers from which it can be generated using diverse fuels.
Electricity emits no air pollutants at the point of use, and its generation can be accomplished free of air
emissions using nuclear fuel1 or via non-chemical, non-extractive resources such as solar, hydroelectric,
wind, and geothermal.2
Natural gas is transported via 300,000 miles of pipelines to most areas of the United States. Its
principal component, methane, burns relatively cleanly and completely at a variety of scales. In
commercial and residential buildings, natural gas is used (i.e., combusted) mainly for cooking and to
supply heat for space conditioning and hygienic services (hot water, drying clothes). Bunker fuels,
propane and petroleum residuals (i.e., the heavier hydrocarbon molecules or “fractions”) and distillates,
are distributed via surface transportation chiefly in areas not served by natural gas pipelines. These fuels
nonetheless provide some of the same energy services as pipeline (grid) natural gas. Combusted at the
point of end-use, these provide fuels for industrial processes, provision of hot water for hygienic services,
furnaces for space conditioning, and fuels for electric generators. In the case of both grid natural gas and
bunker fuels and other non-grid fuels, the resultant gases are typically not scrubbed, or only minimally so,
before being diluted in the troposphere. Emissions include greenhouse gases, acidifying gases, and ozone
precursors.
1
Minute quantities of radioactive gases and liquids are emitted under controlled circumstances. See Nuclear
Regulatory Commission, “Radiation Monitoring at Nuclear Power Plants,” updated March 20, 2020,
https://www.nrc.gov/about-nrc/radiation/protects-you/radiation-monitoring.html.
2
Fuel cells can also generate electricity without emissions although some of the fuels will have undergone
upstream processing and/or conversion that is not emissions-free (NRC, 2004).

Appliances and equipment that provide energy services harness technology as simple as Ohmic
heating (cooking, space conditioning), ranging to the familiar electric motor, such as for fans in air
handlers (ventilation), to the more complex thermodynamic cycles of air conditioners and refrigerators
(space conditioning, food preservation). Equipment in the building’s infrastructure uses physical
principles to adapt the voltage and current of electricity to make it suitable for end-use devices. Natural
gas appliances use combustion technology.
Some of these appliances consume water at the point of use. For example, residential dishwashers
consume water while providing hygienic services.
ROLE OF THE STANDARDS PROGRAM
Devices with improved thermodynamic, mechanical, or electrical performance can deliver the same
quantity of an energy service while consuming less fuel and water. This formula is the basis for many
programs at the U.S. Department of Energy (DOE), which since 1977 has administered programs of
research and development, standards (e.g., for appliances and vehicles), rebates, and other measures
affecting all portions of the energy value chain (NRC, 2001) as well as water consumption. These
programs deliver economic, environmental, and security benefits (NRC, 2001). Consuming less means
lowered demand and a proportional reduction in the impacts on human health and the environment (NRC,
2010) owing to the decreased level of extraction of primary energy resources and their processing,
conversion, delivery, and end-use. It also can mitigate the risk of disruption of energy supply resulting
from resource nationalism or armed conflict. For households or landlords, improved appliances and
equipment can offer lowered energy and water costs, which can lead to positive net benefits when
purchase costs and other factors are considered.
DOE issues “standards regulations” for energy and water conservation pursuant to the Energy Policy
and Conservation Act of 1975 as amended and other authorities. These standards regulations apply to the
appliances and equipment that provide energy services. The regulations also apply to the building
infrastructure that converts or distributes energy. All of these are issued by rulemaking and typically
include maximum water and energy use or minimum energy conservation standards.
ORIGIN OF STUDY
DOE undertakes peer reviews of its assumptions, models, and methodologies (“analytical methods”)
in the Appliance and Equipment Standards Program. These periodic reviews conform to the Office of
Management and Budget’s guidance3 on the use of scientific information, and the first one was performed
in 2007 (DOE, 2007). In 2017, DOE noted its intention to conduct a peer review of the analytical methods
used in setting the standards regulations.4
Following a request from the Office of Energy Efficiency and Renewable Energy, the National
Academies of Sciences, Engineering, and Medicine entered into a contract on July 15, 2019, with DOE to
conduct a peer review of the analytical methods in the buildings standards regulations. The National
Academies established the Committee on Review of Methods for Setting Building and Equipment
Performance Standards. Committee member biographical information is provided in Appendix A.
3
Office of Management and Budget, 2005, “Final Information Quality Bulletin for Peer Review,” Federal
Register 70: 2664, January 14.
4
See Office of Energy Efficiency and Renewable Energy, 2017, “Energy Conservation Program for Appliance
Standards: Proposed Procedures for Use in New or Revised Energy Conservation Standards and Test Procedures for
Consumer Products and Commercial/Industrial Equipment,” Federal Register 84: 3910-3953, February 13, p. 3936.
10

CHARGE TO THE COMMITTEE
Per the contract, the committee is charged with the following:
The National Academies of Sciences, Engineering, and Medicine will appoint a committee to peer
review the analytical methods employed by DOE in setting standards regulations. DOE sets these
standards following procedures and methods designated in the 1975 Energy Policy and
Conservation Act. In conformance with the requirements of EPCA and Executive Order 12866,
DOE implements a set of analyses that accompany its regulations. The methods used in
conducting these analyses will be the subject of the study, and in reviewing these methods the
committee will give due consideration to analytical focus areas of interest in setting standards
regulations. DOE may also provide specific methodological questions for the committee to
address. At the conclusion of the study the committee will issue a report that makes findings and
recommendations on how DOE can improve its analyses and align its regulatory analyses with
best practices for cost- benefit analysis.
COMMITTEE’S APPROACH TO THE STATEMENT OF TASK
While conducting this study, committee members relied on their own expertise, information from
publications they judged to be of high quality, and many interactions with officials from Building
Technologies Office of DOE and its contractors. A list of the committee activities is included as
Appendix B.
DOE provided the committee with the full suite of analyses for standards regulations for each of the
following three product classes in the Appliance and Equipment Standards Program: (1) residential
dishwashers, (2) commercial refrigeration equipment, and (3) residential furnaces. DOE had developed
the analyses for these product classes to support specific rulemaking efforts (i.e., the administrative
process of setting standards regulations). The three rulemakings provide examples of product categories
in wide deployment. The three product categories are diverse in the type of energy service delivered, the
novel issues they instantiated, and the consumer behaviors of relevance to each. Nonetheless, the
committee found commonalities among them, and many of the committee’s findings and
recommendations proved robust across all three illustrative product classes. DOE also discussed other
standards from its Appliance and Equipment Standards Program—there are more than 70—in response to
the committee’s queries.
STRUCTURE OF THE REPORT
Chapter 2 reviews the statutory background for the DOE Appliance and Equipment Standards
Program going back to 1975. It also describes the Executive Orders and laws that affect the analyses to
support standards setting. Lastly, it provides a conceptual overview of how the committee has undertaken
its work to evaluate the analyses DOE has chosen to conduct based on statutory and other considerations.
Chapter 3 evaluates those analyses chiefly concerned with engineering that support the Standards
Program. Chapter 4 considers the economic analyses DOE conducts. Chapter 5 situates the DOE
Appliance, Equipment Standards Program within the broader suite of interventions DOE makes on energy
consuming appliances and equipment; the chapter in particular considers how these interventions may be
related to or affect the Regulatory Impact Analysis. Chapter 6 anticipates future changes that might occur
in the use of fuels to provide energy services by the appliances and equipment considered in the Standards
Program and the implications these might have for the analyses DOE will conduct in the future in support
of the program.
Some readers may first want context about the changes under way in the electric grid. The section
“The Integrated Grid” in Chapter 6 provides such a description. Furthermore, readers wanting to gain
11

additional background on the marketplace for appliances and equipment and on consumer choice could
read the sections, “Error! Reference source not found.” and “Conceptual Model of Consumer Choice,”
in Chapter 4.
REFERENCES
DOE (U.S. Department of Energy). 2007. Energy Conservation Standards Rulemaking Peer Review
Report: Prepared Pursuant to the Office of Management and Budget’s “Final Information
Quality Bulletin for Peer Review.” Washington, DC. February.
NRC (National Research Council). 2001. Energy Research at DOE: Was It Worth It? Energy Efficiency
and Fossil Energy Research 1978 to 2000. Washington, DC: National Academy Press.
NRC. 2004. The Hydrogen Economy: Opportunities, Costs, Barriers, and R&D Needs. Washington, DC:
The National Academies Press.
NRC. 2010. Hidden Costs of Energy: Unpriced Consequences of Energy Production and Use.
Washington, DC: The National Academies Press.
12

Framework for Developing Regulations
INTRODUCTION
This chapter reviews the U.S. Department of Energy’s (DOE’s) practices for setting energy efficiency
standards in the context of the statutory and executive mandates that guide them. It discusses the
regulatory impact analysis (RIA) framework embodied in long-standing executive orders and the manner
in which this framework provides a useful structure for analyzing and developing new and amended
standards and for reviewing existing standards.
The committee reviews the history of energy efficiency standards from Congress’s passage of the
Energy Policy and Conservation Act of 1975 (hereafter “EPCA”) to today. It revisits some of the
controversies surrounding the Act and associated regulations and examines the impacts of subsequent
amendments and the evolution of standard-setting practices over the last 45 years. It summarizes the ex
ante analytical requirements with which DOE must comply in setting standards and then reviews the
program’s impacts. These impacts include the number of appliances covered and standards established,
claims of energy use reductions and consumer cost savings, concerns about impacts on appliance
functionality, and effects on competition. This review suggests that framing DOE’s analysis according to
RIA principles and steps could provide more information on the societal impacts of proposed standards
and better achieve statutory goals. The chapter concludes with a description of the factors the committee
considered in its peer evaluation.
CONTEXT—STATUTORY REQUIREMENTS
DOE’s Legislative Authority
DOE’s authority to set energy conservation standards is codified at 42 U.S.C. § 6295.1 In 1975, 2
years before DOE was created by the Department of Energy Organization Act of 1977, Congress passed
EPCA, directing the National Institute of Standards and Technology (then known as the National Bureau
of Standards) to develop test procedures for measuring the energy efficiency of appliances. Then, in 1978
the National Energy Conservation Policy Act (NECPA) first gave the newly created DOE the authority to
set mandatory minimum energy performance standards, which would preempt State standards. As Table
2.1 and Figure 2.1 show, Congress has amended the laws governing energy conservation standards
numerous times since then.
1
Throughout the chapter, U.S.C. is used to refer to the United States Code, a collection of certain statutes
arranged by titles according to subject matter. The format is [TITLE NUMBER] U.S.C. § [SECTION NUMBER].
13

TABLE 2.1 Significant Legislative Action for the Appliance and Equipment Standards Program
Public Law
Action (P.L.) Date
Energy Policy and Conservation Act (EPCA) P.L. 94-163 December 22, 1975
National Energy Conservation Policy Act P.L. 95-619 November 9, 1978
National Appliance Energy Conservation Act of 1987 (NAECA) P.L. 100-12 March 17, 1987
National Appliance Energy Conservation Amendments of 1988 P.L. 100-357 June 28, 1988
Energy Policy Act of 1992 P.L. 102-486 October 24, 1992
Energy Policy Act of 2005 (EPACT 2005) P.L. 109-58 August 8, 2005
Energy Independence and Security Act of 2007 (EISA 2007) P.L. 110-140 December 19, 2007
American Energy Manufacturing Technical Corrections Act (AEMTCA) P.L. 112-210 December 18, 2012
FIGURE 2.1 Legislative history. SOURCE: John Cymbalski, DOE, “Appliance and Equipment
Standards Program Buildings Technology Office,” presentation to the committee, November 19, 2019.
14

EPCA specifies a list of consumer products2 and industrial equipment3 covered by the statute and
authorizes the Secretary of Energy to set standards for additional consumer products if the average annual
per-household energy use by such product is likely to exceed 100 kilowatt-hours per year (42 U.S.C. §
6292(b)).
Under the Act, DOE must review standards every 6 years, and either determine that the standard does
not need to be amended or propose a new standard. If the Secretary determines that a revision is
necessary, DOE then has 2 years within which to seek and evaluate public comment and issue a final
standard.
In setting new or amended standards, EPCA directs DOE to design standards to “achieve the
maximum improvement in energy [or water] efficiency . . . which the Secretary determines is
technologically feasible and economically justified” (42 U.S.C. § 6295(o) and (42 U.S.C. §
6313(a)(6)(B)(iii)). It further guides the Secretary to determine whether a standard is “economically
justified” using a benefit-cost standard, based on “whether the benefits of the standard exceed its
burdens,” taking into consideration the following:
(I) the economic impact of the standard on the manufacturers and on the consumers of the
products subject to such standard;
(II) the savings in operating costs throughout the estimated average life of the covered product in
the type (or class) compared to any increase in the price of, or in the initial charges for, or
maintenance expenses of, the covered products which are likely to result from the imposition of
the standard;
(III) the total projected amount of energy, or as applicable, water, savings likely to result directly
from the imposition of the standard;
(IV) any lessening of the utility or the performance of the covered products likely to result from
the imposition of the standard;
(V) the impact of any lessening of competition, as determined in writing by the Attorney General,
that is likely to result from the imposition of the standard;
(VI) the need for national energy and water conservation; and
(VII) other factors the Secretary considers relevant. (42 U.S.C. § 6295(o)(2)(B)(i))
In addition to these criteria, the EPCA provides a rebuttable presumption that a standard is
economically justified “if the Secretary finds that the additional cost to the consumer of purchasing a
product complying with an energy conservation standard level will be less than three times the value of
the energy, and as applicable, water, savings during the first year that the consumer will receive as a result
of the standard” 42 U.S.C. § 6295(o)(2)(B)(iii)). This implies that if DOE finds that the energy savings of
a product will allow consumers to recoup the upfront costs within 3 years, it need not conduct further
analysis to demonstrate the product is economically justified. In addition to being technologically feasible
and economically justified, the standard must “result in significant conservation of energy” or water (42
U.S.C. § 6295(o)(3)).
Furthermore, “the Secretary may not prescribe an amended or new standard . . . [that] is likely to
result in the unavailability in the United States in any covered product type (or class) of performance
characteristics (including reliability), features, sizes, capacities, and volumes that are substantially the
2
These are listed at 42 U.S.C. § 6292(a) and include refrigerators, kitchen ranges and ovens, water heaters,
dishwashers, clothes washers and dryers, television sets, general service incandescent lamps, and showerheads.
3
Listed at 42 U.S.C. § 6311(1), covered equipment includes electric motors and pumps, commercial
refrigerators, automatic commercial ice makers, walk-in freezers, and commercial clothes washers.
15

same as those generally available in the United States at the time of the Secretary’s finding” (42 U.S.C. §
6295(o)(4)).
Other Requirements Governing Standards Development
In addition to its organic statutes, DOE is subject to other legislative and executive requirements
when developing and issuing new standards. This section summarizes the Administrative Procedure Act
and key executive orders that guide DOE’s regulatory practices.
The Administrative Procedure Act
The Administrative Procedure Act of 1946 (APA) established procedures an agency must follow in
issuing binding rules and regulations. It constrains executive rulemaking in three main ways. First,
agencies can only act within limits set by their organic statutes. Second, agencies must notify the public
of the proposed action and consider public comments before issuing a final rule. Third, final actions must
be supported by the factual record developed during rulemaking, including substantive comments
received.
DOE relies on the APA’s “informal” or “notice and comment” rulemaking procedures to issue energy
efficiency standards (5 U.S.C. § 553). Usually, it proposes a rule or standard and invites public comment
through a Notice of Proposed Rulemaking (NOPR) published in the Federal Register. After reviewing
public comments on the NOPR, it will issue a final rule.
DOE occasionally engages in negotiated rulemaking to bring different stakeholders to the table to
jointly draft a proposed regulation. (An example is the rulemaking for distribution transformers.4,5) The
resulting draft usually must comply with the APA’s informal rulemaking requirements and go through
public notice and comment. However, EPCA gives DOE authority to publish a direct final rule that
establishes energy conservation standards based on a recommendation “that is submitted jointly by
interested persons that are fairly representative of relevant points of view (including representatives of
manufacturers of covered products, States, and efficiency advocates), as determined by the Secretary” (42
U.S.C. § 6295(p)(4)). When issuing a direct final rule, the Secretary must simultaneously publish an
identical proposed rule. If DOE receives an adverse comment or alternative recommendation that
provides a reasonable basis for withdrawal, it must withdraw the direct final rule (42 U.S.C. §
6295(p)(4)(C)).
Executive Order 12866
Executive Order (E.O.) 12866 (Clinton, 1993) guides both regulatory practice and analysis. Pursuant
to E.O. 12866, before publishing significant proposed and final rules in the Federal Register, DOE must
send them to the Office of Information and Regulatory Affairs (OIRA) within the Office of Management
and Budget (OMB) for interagency review. This review serves two primary purposes: to coordinate
regulatory policies across agencies and to ensure regulations follow good regulatory practices outlined in
4
U.S. Department of Energy (DOE), 2012, “Notice of Proposed Rulemaking and Public Meeting: Energy
Conservation Program: Energy Conservation Standards for Distribution Transformers,” Federal Register 77: 7282-
7381, February 10.
5
DOE, 2012, “Final Rule: Energy Conservation Program: Energy Conservation Standards for Distribution
Transformers,” Federal Register 78: 23336-23436, April 18.
16

E.O. 12866 (discussed further below).6 The default length of this review is 90 days (E.O. 12866 Sec.
6(b)(2)), although either OMB or DOE can extend that.
Other Legislative and Executive Requirements
Energy conservation standards are subject to other procedural requirements; the most relevant are
summarized in Table 2.2.
TABLE 2.2 Procedural and Regulatory Review Requirements

Requirement Summary
E.O. 12866 (Clinton, 1993) Requires regulatory analysis and OMB/OIRA review of significant
regulatory actions.
E.O. 13563 (Obama, 2011) Reinforced E.O. 12866 and emphasized integration and innovation, flexible
approaches, science, and retrospective review.
Regulatory Flexibility Act (1980; 5 “Requires preparation of an initial regulatory flexibility analysis for any rule
U.S.C. 604) that by law must be proposed for public comment, unless the agency certifies
that the rule, if promulgated, will not have a significant economic impact on
a substantial number of small entities” (DOE EERE, 2021, p. 18911).
Paperwork Reduction Act (1995) The collection-of-information requirement for certification and
recordkeeping is subject to review and approval by OMB/OIRA.
Unfunded Mandates Reform Act Requires agencies “to assess the effects of Federal regulatory actions on
(1995) State, local, and tribal governments and the private sector,” to provide a
statement of benefits and costs on national economy, and to seek input from
state, local and tribal government on a “proposed significant
intergovernmental mandate” (DOE EERE, 2021, p. 18912).
E.O. 13132 “Federalism” (1999) Policies and regulations that affect state law are required to “examine the
constitutional and statutory authority supporting any action that would limit
the policy making discretion of the States and to carefully assess the
necessity for such actions” (DOE EERE, 2021, p. 18912). Must give states
the opportunity to provide input.
Review Under the Information DOE conducted peer review of “energy conservation standards development
Quality Bulletin for Peer Review process and analyses and has prepared a Peer Review Report pertaining to
(2005) the energy conservation standards rulemaking analysis” (DOE EERE, 2021,
p. 18913).
NOTE: E.O. = executive order; OIRA = Office of Information and Regulatory Affairs; OMB = Office of
Management and Budget.
SOURCE: U.S. Department of Energy Office of Energy Efficiency and Renewable Energy (DOE EERE), 2021,
“Energy Conservation Program for Appliance Standards: Procedures, Interpretations, and Policies for Consideration
in New or Revised Energy Conservation Standards and Test Procedures for Consumer Products and
Commercial/Industrial Equipment,” Federal Register 86(68): 18901-18921.
6
E.O. 12866 Sec. 2(b) states as follows: “Coordinated review of agency rulemaking is necessary to ensure that
regulations are consistent with applicable law, the President’s priorities, and the principles set forth in this Executive
order, and that decisions made by one agency do not conflict with the policies or actions taken or planned by another
agency.”
17

EVOLUTION OF STANDARDS
Standards Versus Labeling
In 1975, EPCA established the first federal program aimed at energy conservation for consumer
products. It “consist[ed] of test procedures, labeling, and energy targets for consumer products.”7 It
authorized federal appliance efficiency standards, but only if efficiency labels—providing consumers with
relevant information on which to make their own choices—were unsuccessful.
In 1978, EPCA was amended to remove the labeling prerequisite. In response, on June 20, 1980,
DOE proposed energy efficiency standards for eight types of home appliances: refrigerators and
refrigerator-freezers, freezers, clothes dryers, water heaters, room air conditioners, kitchen ranges and
ovens, central air conditioners, and furnaces. Its supporting analysis relied heavily on private benefits, in
the form of cost savings to consumers that modeling predicted would be obtained with these standards.
DOE argued that standards were necessary because myopia prevented consumers from appreciating that
their cost savings from reduced energy usage over the life of a more efficient appliance would exceed the
higher up-front purchase price (Mannix and Dudley, 2015).
DOE’s explanation of consumer short-sightedness did not persuade President Carter’s Regulatory
Analysis Review Group (RARG).8 The RARG argued that “[w]hile consumers still might be ‘myopic’ in
considering future energy savings, the case is not nearly so clear-cut as it once might have seemed”
(RARG, 1980, p. 5). Instead, RARG “identified several assumptions and methodologies that appear
unrealistic, unduly pessimistic about the workings of the market or of labelling, unduly optimistic about
the effect of mandatory standards, or simply undocumented or unclear” (RARG, 1980, p. 7).
Rejecting DOE’s “assumptions . . . of extremely irrational behavior on the part of consumers,” RARG
recommended that DOE not issue standards.
DOE’s analysis of the net benefits of the standards appears to have exaggerated them, particularly
in comparison to the benefits of labels. We suggest that this analysis be redone with more realistic
and cautious assumptions and with lower standards, as suggested above. If, as a result, a particular
standard does not appear to offer significant net benefits beyond those available from labelling, we
recommend that DOE find, as it has the power to do, that such a standard is not justified. (RARG,
1980, p. 7)
Notwithstanding the Carter White House’s criticisms, DOE attempted to issue final appliance
efficiency standards in January 1981, but the new Reagan administration agreed with its predecessor and
instructed DOE to issue a “no-standard standard” (Mannix and Dudley, 2015). In 1982 and 1983, DOE
issued regulations finding that appliance efficiency standards were not economically justified.9,10 The
passage of the 1987 National Appliance Energy Conservation Act led to new standards for a number of
additional product categories (see Figure 2.1).
A few years later, the Energy Star program was initiated as a voluntary measure and has attracted
high engagement from manufacturers seeking to use the label as a marketing tool. However, its analysis
and techniques do not need to employ the same rigor or transparency as DOE does in the appliance
7
Further information is available at DOE, “History and Impacts: Buildings,”
https://www.energy.gov/eere/buildings/history-and-impacts.
8
RARG membership included most executive branch regulatory agencies. Its executive committee consisted of
the Council of Economic Advisors, the Office of Management and Budget, the U.S. Department of Labor, and the
U.S. Environmental Protection Agency. See OMB (1997).
9
DOE, 1982, “Energy Conservation Program for Consumer Products; Final Rule for Clothes Dryers and
Kitchens Ranges and Ovens,” Federal Register 47: 57198-57219, December 22.
10
DOE, 1983, “Energy Conservation Program for Consumer Products; Final Rule for Refrigerators and
Refrigerator-Freezers, Freezers, Water Heaters, Room Air Conditioners, Furnaces and Central Air Conditioners,”
Federal Register 48: 39376-39409.
18

standards development in the methods used to categorize products into like groups or “levels” based on
energy efficiency performance.
REQUIREMENTS FOR REGULATORY ANALYSIS
In parallel with legislative authority for issuing energy efficiency standards, presidents have required
agencies to analyze the likely impact of regulations before issuing them. Both legislation and executive
order require DOE to qualitatively and quantitatively assess the impacts of its standards. This section
describes executive and statutory RIA requirements and draws parallels between them and those
embodied in EPCA.
Regulatory Impact Analysis (RIA)
Since the 1970s, presidents of both parties have directed regulatory agencies to consider the costs and
benefits of regulatory actions. Executive Order 12866, issued by President Clinton in 1993, continues to
guide the development of rulemaking today. It requires agencies to conduct an RIA, which (1) identifies
the compelling public need for regulation; (2) evaluates alternative approaches to address that need; and
(3) estimates the benefits and costs of those alternatives (OMB, 2010, pp. 2-3). In writing regulations, it
directs agencies to “assess all costs and benefits of available regulatory alternatives, including the
alternative of not regulating,” and to select alternatives that maximize net benefits, to the extent permitted
by law.11 (See also Box 2.1.)
For regulations deemed significant under E.O. 12866 § 3(f)(1),12 agencies must provide as part of
their decision-making process (unless prohibited by law):
(i) An assessment, including the underlying analysis, of benefits anticipated from the regulatory
action (such as, but not limited to, the promotion of the efficient functioning of the economy and
private markets, the enhancement of health and safety, the protection of the natural environment,
and the elimination or reduction of discrimination or bias) together with, to the extent feasible, a
quantification of those benefits;
(ii) An assessment, including the underlying analysis, of costs anticipated from the regulatory
action (such as, but not limited to, the direct cost both to the government in administering the
regulation and to businesses and others in complying with the regulation, and any adverse effects
on the efficient functioning of the economy, private markets (including productivity, employment,
and competitiveness), health, safety, and the natural environment), together with, to the extent
feasible, a quantification of those costs; and
(iii) An assessment, including the underlying analysis, of costs and benefits of potentially effective
and reasonably feasible alternatives to the planned regulation, identified by the agencies or the
public (including improving the current regulation and reasonably viable nonregulatory actions),
and an explanation why the planned regulatory action is preferable to the identified potential
alternatives. (Clinton, 1993, § 6(a)(3)(C))
11
E.O. 12866, Section 1.a. states as follows: “in choosing among alternative regulatory approaches, agencies
should select those approaches that maximize net benefits (including potential economic, environmental, public
health and safety, and other advantages; distributive impacts; and equity), unless a statute requires another
regulatory approach.”
12
OMB refers to this category of significant regulations as “economically significant.”
19

BOX 2.1 E.O. 12866 § 1.b. The Principles of Regulation (Clinton, 1993)
To ensure that the agencies’ regulatory programs are consistent with the philosophy set forth
above, agencies should adhere to the following principles, to the extent permitted by law and where
applicable:
(1) Each agency shall identify the problem that it intends to address (including, where applicable, the
failures of private markets or public institutions that warrant new agency action) as well as assess the
significance of that problem.
(2) Each agency shall examine whether existing regulations (or other law) have created, or contributed
to, the problem that a new regulation is intended to correct and whether those regulations (or other law)
should be modified to achieve the intended goal of regulation more effectively.
(3) Each agency shall identify and assess available alternatives to direct regulation, including providing
economic incentives to encourage the desired behavior, such as user fees or marketable permits, or
providing information upon which choices can be made by the public.
(4) In setting regulatory priorities, each agency shall consider, to the extent reasonable, the degree and
nature of the risks posed by various substances or activities within its jurisdiction.
(5) When an agency determines that a regulation is the best available method of achieving the
regulatory objective, it shall design its regulations in the most cost-effective manner to achieve the
regulatory objective. In doing so, each agency shall consider incentives for innovation, consistency,
predictability, the costs of enforcement and compliance (to the government, regulated entities, and the
public), flexibility, distributive impacts, and equity.
(6) Each agency shall assess both the costs and the benefits of the intended regulation and, recognizing
that some costs and benefits are difficult to quantify, propose or adopt a regulation only upon a
reasoned determination that the benefits of the intended regulation justify its costs.
(7) Each agency shall base its decisions on the best reasonably obtainable scientific, technical,
economic, and other information concerning the need for, and consequences of, the intended
regulation.
(8) Each agency shall identify and assess alternative forms of regulation and shall, to the extent
feasible, specify performance objectives, rather than specifying the behavior or manner of compliance
that regulated entities must adopt.
(9) Wherever feasible, agencies shall seek views of appropriate State, local, and tribal officials before
imposing regulatory requirements that might significantly or uniquely affect those governmental
entities. Each agency shall assess the effects of federal regulations on State, local, and tribal
governments, including specifically the availability of resources to carry out those mandates, and seek
to minimize those burdens that uniquely or significantly affect such governmental entities, consistent
with achieving regulatory objectives. In addition, as appropriate, agencies shall seek to harmonize
federal regulatory actions with related State, local, and tribal regulatory and other governmental
functions.
(10) Each agency shall avoid regulations that are inconsistent, incompatible, or duplicative with its
other regulations or those of other federal agencies.
(11) Each agency shall tailor its regulations to impose the least burden on society, including
individuals, businesses of differing sizes, and other entities (including small communities and
governmental entities), consistent with obtaining the regulatory objectives, taking into account, among
other things, and to the extent practicable, the costs of cumulative regulations.
20

(12) Each agency shall draft its regulations to be simple and easy to understand, with the goal of
minimizing the potential for uncertainty and litigation arising from such uncertainty.
The RIAs DOE prepares for its energy efficiency standards are not integrated into the standard-setting
framework and mainly discuss alternatives that DOE concludes are not permitted by statute though
conceivably might be authorized by legislators in the future. It is presented separately from, and after,13
other analyses, which appears inconsistent with the purpose of the RIA “to inform agency decisions in
advance of regulatory actions and to ensure that regulatory choices are made after appropriate
consideration of the likely consequences” (OMB, 2011b). According to OIRA, “[r]egulatory analysis also
has an important democratic function; it promotes accountability and transparency and is a central part of
open government” (OMB, 2011b). As is discussed below and in subsequent chapters, more attention to
the RIA’s analysis of societal benefits could improve DOE’s analyses and potentially the consumer
impacts and societal outcomes of DOE’s standards.
Because the RIA requirements are consistent with DOE’s statutory mandate to ensure its standards
are technologically feasible and economically justified, using the RIA framework to support new or
amended standards offers advantages. As OMB’s Circular A-4 explains:
Regulatory analysis is a tool regulatory agencies use to anticipate and evaluate the likely
consequences of rules. It provides a formal way of organizing the evidence on the key effects—
good and bad—of the various alternatives that should be considered in developing regulations.
The motivation is to (1) learn if the benefits of an action are likely to justify the costs or (2)
discover which of various possible alternatives would be the most cost-effective. (OMB, 2003)
Need for Regulation
The first step in the RIA framework is identifying the problem to be solved. The “Regulatory
Philosophy” stated in E.O. 12866 is that “Federal agencies should promulgate only such regulations as are
required by law, are necessary to interpret the law, or are made necessary by compelling public need, such
as material failures of private markets to protect or improve the health and safety of the public, the
environment, or the well-being of the American people” (Clinton, 1993, § 1(a)). The first two principles
reproduced above reinforce that step (Clinton, 1993, § 2(b)(1) and § 2(b)(2)).
Circular A-4 provides direction to agencies:
Before recommending Federal regulatory action, an agency must demonstrate that the proposed
action is necessary. If the regulatory intervention results from a statutory or judicial directive, you
should describe the specific authority for your action, the extent of discretion available to you, and
the regulatory instruments you might use.
The major types of market failure include externality, market power, and inadequate or
asymmetric information. Correcting market failures is a reason for regulation, but it is not the only
reason. Other possible justifications include improving the functioning of government, removing
distributional unfairness, or promoting privacy and personal freedom.
The concept of “market failure” is important. Market economies rely on competition and price signals
to allocate scarce resources to their most valued uses, to encourage innovation, and to satisfy consumer
13
For example, the RIA is the last of 17 chapters in DOE’s 2014 Technical Support Document for Residential
Dishwashers (DOE, 2014a).
21

needs. When markets are operating properly—which, as Circular A-4 makes clear, is not always the
case—government regulation can disrupt those signals, reducing the efficiency of market forces and
harming social welfare. Because of these effects, longstanding good regulatory practice begins with an
explanation of why market outcomes are less efficient than what government regulations could be
expected to accomplish (Dudley et al., 2017). Appliance markets are subject to a wide range of market
imperfections, on both the producer and consumer side, including issues related to oligopolistic markets,
consumer information, renters and other housing market issues, and the environmental and security
consequences of energy use. As is developed in Chapter 4, these failures are specific to particular
consumers, manufacturers and appliances; interact in different ways with the minimum efficiency
standards contemplated by this program; and their importance in particular cases may rest on empirical
verification. Because of this heterogeneity, careful consideration of market outcomes and market failures
can contribute meaningfully to DOE analyses. We consider some of these possibilities in this section in
the context of the RIA structure, and return to the relationship between market failures and standards in
Chapter 4.
DOE discusses several possible market failure justifications for energy efficiency standards. For
example, the 2012 residential dishwasher rule published by DOE identifies three problems the standards
address.14 Two relate to what Allcott and Greenstone (2012) refer to as “investment inefficiencies,” where
consumers’ lack of information or the ability to process information on energy efficiency such that they
do not take profitable investments, leading to an “energy efficiency gap” (Jaffe and Stavins, 1994). The
other recognizes “externalities related to environmental protection and energy security that are not
reflected in energy prices, such as reduced emissions of greenhouse gases.”15,16
Alternatives
E.O. 12866 directs agencies to assess alternatives to regulation (including economic incentives or
information provision) (Clinton, 1993, § 1(b)(3)), as well as alternative forms of regulation (Clinton,
1993, § 1(b)(5)). For efficiency standards, alternatives to regulation might include labeling, “consumer-
initiated litigation in the product liability system,” or “regulation at the State or local level” (OMB,
2011a). For example, household appliances, particularly those involved in heating or cooling, might be
better addressed locally (OMB, 2011b), given the variation in temperature conditions across the United
States. E.O. 13563 directs agencies to “consider regulatory approaches that reduce burdens and maintain
flexibility and freedom of choice for the public” (Obama, 2011). These include “appropriate default rules,
and disclosure requirements” (OMB, 2011a).
Two of DOE’s three justifications for action on a dishwasher standard relate to product purchasers
having inadequate information on potential cost savings from more efficient appliances. Given this, E.O.
12866 and OMB’s Circular A-4 call for more consideration than DOE currently provides of the
alternatives to standards regulations that address information asymmetry. Tailoring regulatory design to
the problem to be solved is particularly important given OMB’s warning that regulations can impede
market efficiency. Its guidelines include a “presumption against certain types of regulatory action”
14
DOE lists and describes these failures on DOE, 2012, “Direct Final Rule: Energy Conservation Program:
Energy Conservation Standards for Residential Dishwashers,” Federal Register 77: 31917-31963, May 30, p.
31959.
15
For commercial refrigeration equipment, where consumers are profit-motivated commercial entities and not
individuals, DOE suggests that “[f]or certain segments of the companies that purchase commercial refrigeration
equipment, such as small grocers, there may be a lack of consumer information and/or information processing
capability about energy efficiency opportunities in the commercial refrigeration equipment market” (DOE, 2014,
“Final Rule: Energy Conservation Program: Energy Conservation Standards for Commercial Refrigeration
Equipment,” Federal Register 79(60): 17726-17818, March 28, p. 17812).
16
DOE, 2012, “Direct Final Rule: Energy Conservation Program: Energy Conservation Standards for
Residential Dishwashers,” Federal Register 77: 31917-31963, May 30.
22

including “mandatory uniform quality standards for goods or services if the potential problem can be
adequately dealt with through voluntary standards or by disclosing information of the hazard to buyers or
users” (OMB, 2003). “In light of both economic theory and actual experience,” OMB calls for “a
particularly demanding burden of proof . . . to demonstrate the need for” such regulations (OMB, 2003).
Benefits and Costs
E.O. 12866 directs agencies, “in choosing among alternative regulatory approaches, [to] select those
approaches that maximize net benefits (including potential economic, environmental, public health and
safety, and other advantages; distributive impacts; and equity), unless a statute requires another regulatory
approach.” EPCA does not require another approach. It directs DOE only to set standards that are
“economically justified,” and explains:
In determining whether a standard is economically justified, the Secretary shall, after

receiving views and comments furnished with respect to the proposed standard, determine
whether the benefits of the standard exceed its burdens by, to the greatest extent practicable.
(42 U.S.C. § 6295(o)(2)(B))
FINDING: Given the similarities between DOE’s statutory requirements and the regulatory
impact analysis practices embodied in presidential executive orders and OMB guidelines, using
the RIA framework to organize the development of the standards could lead to greater attention to
their social welfare impacts.

DOE generally provides its estimates of benefits and costs in sections of the Technical Support
Document (TSD) other than the RIA. For example, for the three rulemakings considered as part of this
review (commercial refrigeration equipment (CRE), residential dishwashers, and residential furnaces), the
TSD is organized such that Chapter 8 estimates consumer costs (in terms of life-cycle costs and payback
periods). Chapter 10 estimates national energy savings, Chapter 12 looks at manufacturer impacts, and
Chapters 13 and 14 quantify and monetize the benefits of emissions reductions at different levels. Chapter
17, the final chapter of the TSD, is labeled RIA and compares several alternatives to regulation.
The dishwasher rule provides a good example of the value of integrating the RIA analysis into the
detailed and extensive economic analyses of costs and benefits. In this case, DOE’s analyses conclude
that the changes mandated by the proposed standard provide consumers with significant economic
benefits—in effect, that the investment in a standard-complying dishwasher saves consumers money
owing to the value of energy savings to the consumer being larger than the cost of compliance (i.e., the
increased price of the appliance). If this is the case, why don’t consumers buy the more efficient
appliance? One possibility, subscribed to by the DOE analysis is a consumer market failure that results in
an energy efficiency gap. However, as is discussed in Chapter 4, the relationship between an observed
energy efficiency gap and the proposed market failures is not a given: the observed gap may be related to
systematic market failures that are properly addressed by standards, but in some cases evidence suggests
that other non-market factors could be present. Such other factors could include misrepresenting the
following: the actual costs of ownership (Jaccard et al., 2003; NRC, 2010, p. 47); the pattern of use of the
consumer (Jaffe and Stavins, 1994); or the performance of the appliance (NRC, 2013, p. 28). (See also
Allcott and Greenstone, 2012; Gerarden et al., 2017; Hassett and Metcalf, 1992; Stern et al., 2016.) An
analysis focused on market failures can consider not only whether a standard is likely to be justified, but
also whether the calculations provided by the analysis are likely to be correct.
As with the dishwasher rule, for many energy conservation regulations, most of the quantified
benefits derive from private savings to purchasers of more efficient products (Gayer and Viscusi, 2013).
The National Highway Traffic Safety Administration’s 2011 regulation for fuel efficiency of medium and
heavy-duty engines and vehicles, for example, attributed $42 billion of the net benefits of the
23

regulation—the vast majority—to private savings on fuel costs that would accrue to the would-be
purchasers of the more efficient engines and vehicles.17 The conclusion that DOE standards are needed
for consumers to realize these savings require a fuller justification than the brief statements currently
provided in TSDs and rule preambles. These preambles typically suggest that consumers lack information
or information processing ability to realize those benefits themselves. DOE’s RIAs do not present
evidence that individuals behave irrationally when making purchasing decisions on covered products. Nor
do they provide evidence that indicates DOE is better able to judge other people’s preferences or be more
faithful agents of their interests than the people themselves (Mannix and Dudley, 2015), consistent with
the admonition of E.O. 12866 to identify the failures of private markets (Clinton, 1993, sec. 1.b.1).
Distributional Effects
E.O. 12866 requires agencies to consider distributive impacts and equity (Clinton, 1993, Sec. 1.a and
Sec. 1.b.5). Chapter 11 of DOE’s TSDs18 provide a subgroup analysis, which “evaluates impacts on
groups or customers who may be disproportionately affected by any national energy conservation
standard . . . by analyzing the life cycle cost (LCC) impacts and payback period (PBP) for those
consumers from the considered energy efficiency levels” (DOE, 2016). This is consistent with the OMB
Circular A-4 requirement to “provide a separate description of distributional effects (i.e., how both
benefits and costs are distributed among sub-populations of particular concern) so that decision makers
can properly consider them along with the effects on economic efficiency.”
As Jaffe and Stavins (1994) observe, “If the relevant population is heterogeneous with respect to the
amount of energy it uses . . . even a technology that looks very good for the average user will be
unattractive for a portion of the population.” President Biden has emphasized the importance of analyzing
the distributional impacts of regulations (Biden, 2021). Consistent with E.O. 12866 and OMB Circular A-
4, DOE conducts subgroup analyses for its standards, where relevant groups vary with the appliance
considered, considering such features as regional and socioeconomic variation. But here too a greater
focus on market failure could be useful. For example, the existence of unemployed but apparently cost
effective energy efficiency technology can arise in rental units where landlords purchase appliances,
tenants pay energy costs, and landlords are unable to recoup through higher rent the utility savings
associated with more expensive, energy efficient appliance. If such a market failure is expected, a
subgroup analysis focused on rental properties may be useful. The landlord/tenant problem (sometimes
referred to as split incentives) is discussed in more detail in Chapters 4 and 5.
Retrospective Review
E.O. 12866 directs each agency to “periodically review its existing significant regulations to
determine whether any such regulations should be modified or eliminated so as to make the agency’s
regulatory program more effective in achieving the regulatory objectives, less burdensome, or in greater
alignment with the President’s priorities and the principles set forth in this Executive order” (Clinton,
1993, Sec. 5.a). E.O.13563 (Obama, 2010, Sec. 6.a.) further tells agencies to “consider how best to
promote retrospective analysis of rules that may be outmoded, ineffective, insufficient, or excessively
burdensome, and to modify, streamline, expand, or repeal them in accordance with what has been
learned.”
17
See U.S. Environmental Protection Agency and National Highway Traffic Safety Administration, 2011,
“Greenhouse Gas Emissions Standards and Fuel Efficiency Standards for Medium- and Heavy-Duty Engines and
Vehicles” Federal Register 76(179): 57106-57513, September 15, p. 57106b.
18
See, for example, DOE (2014a,b, 2016).
24

Further discussion of ex post analysis, including how this could apply to price, is included in Chapter
4 in the section “The Determination of Markups.” See also Recommendation 4-1.
FINDING: Conducting an evaluation, ex post (i.e., after promulgation of standards), of the

performance of appliances and equipment will permit DOE to evaluate the validity of the ex ante
assumptions it relied on in setting standards and will allow learning and iterative improvement of its
analysis and its standards.

RECOMMENDATION 2-1: To better understand the social impacts of its standards, DOE
should organize its analysis following the regulatory analysis framework laid out in OMB
Circular A-4. For example, rather than presenting the RIA as the last stage in the process, the
analyses shown in Figure S.1 could be technical appendices to the RIA, which would integrate
this information in presenting estimates of the net benefits and distributional impacts of the
proposed efficiency standard and reasonable alternatives, referencing information from
relevant appendices as it describes a baseline scenario and cost and benefit models.

RECOMMENDATION 2-2: DOE should pay greater attention to the justification for the
standards, as required by executive orders and the EPCA requirement that standards be
economically justified. DOE should attempt to find significant failures of private markets or
irrational behavior by consumers in the no-standards case and should consider such a finding
as being necessary to conclude that standards are economically justified.
RECOMMENDATION 2-3: DOE should commit to collecting data necessary to conduct more
rigorous ex post analysis of the effects of standards on consumers, producers, energy
consumption, and environmental impacts.
Regulatory Flexibility Analysis
The Regulatory Flexibility Act (5 U.S.C. 601 et seq.) requires agencies to consider the impact of their
regulatory actions on small entities (e.g. businesses, organizations, or government jurisdictions), analyze
effective alternatives that minimize small entity impacts, and make their analyses available for public
comment. Unless an agency certifies that its proposed regulation would not have a “significant economic
impact on a substantial number of small entities,” it must prepare a regulatory flexibility analysis. DOE
established procedures in 2003 to consider impacts on small entities.19
Of the three appliance rules reviewed in this report, DOE certified that the residential dishwasher
standard would not have a significant economic impact on small entities. It conducted the required
regulatory flexibility analysis for both the CRE and residential furnace standards.
Paperwork Reduction Act
The Paperwork Reduction Act of 1980, as amended in 1995, aims to among other things, “minimize
the paperwork burden . . . resulting from the collection of information by or for the Federal Government,”
“ensure the greatest possible public benefit from and maximize the utility of information created,
collected, maintained, used, shared and disseminated by or for the Federal Government” and “improve the
quality and use of Federal information to strengthen decision making, accountability, and openness in
Government and society” (44 U.S.C. § 3501).
19
DOE Office of General Counsel, 2003, “Notice of Procedures and Policies: Executive Order 13272;
Consideration of Small Entities in Agency Rulemaking,” Federal Register 68: 7990-7994, February 19.
25

It requires federal agencies to seek public comment on proposed collections from 10 or more persons,
and to submit those proposed collections for OIRA/OMB review and approval. When OMB approves an
information collection, it assigns an OMB control number that the agency must display on the
information collection. (For more information, see Sunstein, 2010.) The public is not required to respond
to any collection that does not display an OMB control number.
OMB defines information as “any statement or estimate of fact or opinion, regardless of form or
format, whether in numerical, graphic, or narrative form, and whether oral or maintained on paper,
electronic or other media” (5 C.F.R. 1320.3(h) [i.e., Code of Federal Regulations, Title 5, Section
1320.3(h)]). This includes government forms, surveys, recordkeeping requirements and third-party or
public disclosures (5 C.F.R. 1320.3(c)).
DOE regulations require manufacturers to certify that all covered consumer products and commercial
equipment comply with applicable energy conservation standards. In certifying compliance,
manufacturers must test their products according to DOE test procedures.20,21 For each standard, DOE
estimates the number of hours needed to certify compliance (reporting burden). For example, for the CRE
standards, DOE states “the reporting burden for the certification is estimated to average 20 hours per
response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information.”22
PROGRAM IMPACTS
List of Regulations Issued
DOE provides the table of standards and estimated energy savings for its regulations, replicated here
as Table 2.3.
20
There is no requirement for third-party testing. In the case of motors, however, the laboratory itself must be
certified.
21
DOE, 2016, “Energy Conservation Program: Energy Conservation Standards for Residential Boilers,”
Federal Register 80(61): 17222-17305.
22
DOE, 2014, “Energy Conservation Program: Energy Conservation Standards for Commercial Refrigeration
Equipment,” Federal Register 79: 17726-17818, p. 17814.
26

TABLE 2.3 Energy Savings Data for U.S. Department of Energy (DOE) Energy Conservation Standards, 1989-2019
Date of Final Analysis Site Energy Site Energy
Rule Period Savings Savings
Product/Equipment Citation Publication (years) (quads) (%)
Small gas furnaces 54 FR 47916 11/17/1989 24 0.300 0.2%
Refrigerators, refrigerator-freezers, freezers 54 FR 47916 11/17/1989 23 1.900 13.0%
Residential clothes washers 56 FR 22250 5/14/1991 23 0.290 5.0%
Clothes dryers 56 FR 22250 5/14/1991 23 0.320 6.0%
Dishwashers 56 FR 22250 5/14/1991 23 0.450 12.0%
Room air conditioners 62 FR 50122 9/24/1997 30 0.170 3.0%
Fluorescent lamp ballasts 65 FR 56740 9/19/2000 26 0.43-0.83 Unavailable
Water heaters 66 FR 4474 1/17/2001 27 2.730 4.0%
Central air conditioners and heat pumps 66 FR 7170 1/22/2001 30 2.400 10.0%
Distribution transformers 72 FR 58190 10/12/2007 30 1.260 21.0%
Furnaces and boilers 72 FR 65136 11/19/2007 24 0.240 0.2%
27
Packaged terminal air conditioners (PTACs) and 73 FR 58772 10/7/2008 30 0.009 4.0%
packaged terminal heat pumps (PTHPs)
Commercial refrigeration equipment 74 FR 1092 1/9/2009 30 0.360 16.0%
Electric and gas kitchen ranges and ovens (actual savings for gas 74 FR 16040 4/8/2009 30 0.130 4.0%
cooktops and ovens)
Incandescent reflector lamps 74 FR 34080 7/14/2009 30 0.35-0.88 20-25%
General service fluorescent lamps 74 FR 34080 7/14/2009 30 1.4-3.7 4-12%
Beverage vending machines 74 FR 44914 8/31/2009 30 0.068 20.0%
Commercial clothes washers 75 FR 1122 1/8/2010 30 0.078 12.0%
Small electric motors (SEMs) 75 FR 10874 3/9/2010 30 1.050 52.0%
Pool heaters 75 FR 20112 4/16/2010 30 0.018 1.0%
Direct heating equipment 75 FR 20112 4/16/2010 30 0.190 3.0%
Water heaters 75 FR 20112 4/16/2010 30 1.690 3.0%
Room air conditioners 76 FR 22454 4/21/2011 30 0.150 6.0%
Clothes dryers 76 FR 22454 4/21/2011 30 0.190 3.0%
Central air conditioners and heat pumps and furnaces 76 FR 37408 6/27/2011 30 2.62-3.58 1-2%

Date of Final Analysis Site Energy Site Energy

Rule Period Savings Savings
Product/Equipment Citation Publication (years) (quads) (%)
Fluorescent lamp ballasts 76 FR 70548 11/14/2011 30 0.9-1.8 2-4%
Dishwashers 77 FR 31918 5/30/2012 30 0.070 2.0%
Distribution transformers 78 FR 23336 4/18/2013 30 1.900 18.0%
Microwave ovens (standby/off mode) 78 FR 36316 6/17/2013 30 0.250 57.0%
Metal halide lamp fixtures 79 FR 7746 2/10/2014 30 0.16-0.20 3.0%
External power supplies 79 FR 7846 2/10/2014 30 0.320 56.0%
Commercial refrigeration equipment 79 FR 17726 3/28/2014 30 1.140 13.0%
Electric motors 79 FR 30934 5/29/2014 30 2.800 0.7%
Walk-in coolers and freezers 79 FR 32050 6/3/2014 30 1.250 28.0%
Furnace fans 79 FR 38130 7/3/2014 30 1.080 10.0%
Commercial clothes washers 79 FR 74492 12/15/2014 30 0.044 7.0%
General service fluorescent lamps 80 FR 4042 1/26/2015 30 0.900 7.0%
Automatic commercial ice makers 80 FR 4646 1/28/2015 30 0.063 8.0%
28
Single package vertical air conditioners and heat pumps 80 FR 57438 9/23/2015 30 0.050 4.0%
Ceiling fan light kits 81 FR 580 1/6/2016 30 0.016 3.0%
Beverage vending machines 81 FR 1028 1/8/2016 30 0.044 16.0%
Residential boilers 81 FR 2320 1/15/2016 30 0.137 0.6%
Commercial warm air furnaces 81 FR 2420 1/15/2016 30 0.200 0.8%
Small, large, and very large commercial package air conditioning 81 FR 2420 1/15/2016 30 5.500 24.0%
and heating equipment
Commercial, industrial pumps 81 FR 4368 1/26/2016 30 0.105 0.9%
Commercial pre-rinse spray valves 81 FR 4748 1/27/2016 30 0.065 8.0%
Battery chargers 81 FR 38266 6/13/2016 30 0.061 11.0%
Dehumidifiers 81 FR 38338 6/13/2016 30 0.100 7.0%
Miscellaneous refrigeration products 81 FR 75194 10/28/2016 30 0.540 58.0%
Central air conditioners and heat pumps 82 FR 1786 1/6/2017 30 1.200 3.0%
Dedicated-purpose pool pumps 82 FR 5650 1/18/2017 30 1.300 39.0%
Ceiling fans 82 FR 6826 1/19/2017 30 0.740 26.0%
Walk-in coolers and freezers 82 FR 31808 7/10/2017 30 0.290 8.0%
NOTE: FR = Federal Register, e.g., “82 FR 31802” means Federal Register Volume 82, page 31808; Quads = quadrillion British thermal units.
SOURCE: DOE, 2019, “Energy Savings Data for DOE Energy Conservation Standards, 1989-2019,” EERE-2017-BT-STD-0062-0144, https://www.regulations.gov.

Concerns About Effects on Appliance Function, Impact on Features Consumers Value
DOE’s statute directs it to set standards for which the benefits exceed the burdens “to the greatest
extent possible” (42 U.S.C. 6295 § (o)(2)(B)(i)). In weighing benefits and burdens, DOE is to compare
estimated energy cost savings to possible increases in maintenance expenses as well as initial appliance
prices (42 U.S.C. 6295 § (o)(2)(B)(i)(II)). The statute also requires DOE to consider “any lessening of the
utility or the performance of the covered products likely to result from the imposition of the standard” (42
U.S.C. 6295 § (o)(2)(B)(i)(IV)).
It is often difficult to predict maintenance expenses or impacts on the product’s utility at the time
DOE conducts ex ante analysis. Ex post evaluation of actual experiences is essential to know whether
DOE is meeting its statutory obligations, and to inform subsequent ex ante regulatory impact analysis.
Fraas and Miller’s retrospective evaluation of clothes washer and refrigerator standards identifies
three issues that led actual cost savings to be less than projected in DOE’s ex ante analyses:
1. Product life and reliability;

2. Greater energy usage than anticipated; and
3. Additional operation and maintenance costs (Fraas and Miller, 2020).
Their two case studies suggest between 2001 and 2011, compliance with the energy efficiency
standards brought design challenges that increased repair rates and significantly decreased the product life
of clothes washers and refrigerators. Their case studies “illustrate the need to consider the potentially
substantial costs of operation and repair in conducting retrospective analyses of DOE energy efficiency
standards” (Fraas and Miller, 2020), as well as more attention to these unintended consequences in setting
initial standards.
Impact on Competition
EPCA directs DOE to consider the impact of “any lessening of competition, as determined in writing
by the Attorney General, that is likely to result from the imposition of the standard” (§ (o)(2)(B)(i)(V)).
With respect to the markets for dishwasher and refrigerators, Fraas and Miller (2012) find:
The markets for these appliances have experienced an intense oligopolistic competition for market share
with the consolidation of U.S. manufacturers and aggressive entry strategies by Samsung and LG
Electronics. Any assessment of the pricing behavior and product differentiation in these appliance
markets needs to be evaluated within this context.
CRITERIA AND PRINCIPLES GUIDING THIS REVIEW
This section describes the criteria the committee used to assess DOE’s methods in setting the
minimum energy standards for products and equipment. It also describes the principles and worldviews
that guided the committee’s deliberations. These criteria and principles are applied in Chapters 3 and 4
where the committee reviews the various components of the analyses of the TSDs that support the NOPR
for standards for Residential Furnaces (DOE, 2016), Residential Dishwashers (DOE, 2014a), and
Commercial Refrigeration Equipment (DOE, 2014b). The sequence of analyses in the TSDs is show in
Figure 2.2.
29

FIGURE 2.2 Sequence of analyses in DOE’s technical support documents. SOURCE: John Cymbalski,
DOE, “Appliance and Equipment Standards Program Buildings Technology Office,” presentation to the
committee, November 19, 2019.
In the following chapters the methods used to set standards are examined in terms of five attributes:
 Adequacy and consistency with program’s goal, consistent with relevant law and executive order;
 Transparency and replicability;
 Adherence to state-of-the-art theory and methods of analysis commensurate with data availability
and DOE’s ability to collect and analyze new data;
 Rigor in the treatment of uncertainty and variability; and
 Effectiveness in setting a path of continuous improvement of the process to set standards.
Adequacy of Methods and Consistency with Program’s Goal
The first criterion the committee applies in assessing the methods used is whether they allow DOE to
determine appropriate appliance standards that are adequate to meet overall goals of the statute in terms of
promoting energy efficiency and energy conservation—that is, the report asks if the methods are “fit for
purpose.” To the extent that the goal of a program to set minimum energy efficiency standards for
appliances and equipment is broader than just reducing annual energy consumption in the compliance
year and the future, minimum energy efficiency standards for appliances and equipment are a component
of a larger strategy to fulfill DOE’s mission to “ensure America’s security and prosperity by addressing
its energy, environmental and nuclear challenges through transformative science and technology
solutions.”1 Hence, in the committee’s view, the main goal of the standards is reducing the costs and
environmental externalities of the energy consumed by the regulated equipment and products. This view
contrasts with a narrower interpretation that sees reducing the energy consumption of products and
equipment as the main goal of the standards.
Because the products and equipment regulated by the standards program have an economic life time
that spans years and because both the costs and externalities of the energy consumed in the future are
1
U.S. Department of Energy (DOE), “Mission,” https://www.energy.gov/mission.
30

highly uncertain, the committee agrees that the goals of each standard must be expressed in terms of a
planning horizon and a probabilistic measure. For example, the goal of a particular standard is to reduce
the expected value of costs and environmental externalities of the energy that will be consumed during the
next 3 decades. Currently DOE uses a planning horizon of 30 years.
In looking at whether DOE’s methods are suitable for this larger purpose of reducing costs and
externalities of the energy consumed, the committee considers it necessary to ask whether DOE’s
approach is comprehensive, forward-looking, and mindful of path dependencies (Arthur, 1989).
It is important to note that in its assessment of the methods and recommendations, the committee does
not make any assumptions about the availability or limitations on DOE’s analytical or material resources
or its priorities in the use of such resources. The committee offers its recommendations in light of what is
considered a best practice and feasible given DOE’s existing programs but does not make any statements
or assumption about the financial or personnel requirements to implement changes. Nonetheless, in
selecting recommendations for inclusion in the Summary of this report, the committee has prioritized
those recommendations that clearly communicate uncertainty, are feasible in scope, and which would be
cost effective for DOE to implement in future TSDs.
Comprehensiveness of the Technological and Regulatory Alternatives Considered
In applying this first criterion on adequacy of methods, the committee asks whether, in its analysis,
DOE is comprehensive in its consideration of technologies and regulatory mechanisms likely to deliver
similar energy and environmental outcomes of the standard examined. In terms of technological
comprehensiveness, the committee discusses the need to examine a large set of technologies and assess
their energy consumption and environmental effects before any potential effects on industry
competitiveness are brought into consideration.
In terms of regulatory comprehensiveness, the committee asks if DOE looks at its standards setting
process as a component of its long-term strategy on energy and the environment. In this vein, the
committee asks if DOE is properly considering tradeoffs between the stringency of a standard and the
timing and flexibility of compliance to achieve better outcomes. DOE’s method to set standards already
examines whether non-regulatory policies (e.g., voluntary targets, tax credits) can deliver better outcomes
than the standard proposed. However, the committee’s question is not whether non-regulatory approaches
are better than the standard but instead, whether DOE is being comprehensive in its consideration of a
standard that varies in its stringency, flexibility of compliance, and timing of compliance. The
committee’s view is that comprehensiveness in the consideration of the bundle of attributes of a standard
(i.e., its stringency, timing of compliance, and flexibility) allows DOE to consider path dependencies.
Consideration of the Outcomes of Energy Efficiency Standards in a Changing Context
A further question the committee asks on this first criterion is whether DOE is forward-looking in its
consideration of the possible future circumstances that affect the outcomes achieved by standards. A
standard that passes the cost-benefit analysis under a business-as-usual scenario may fail when plausible
future changes are considered. Advances in the technology of the equipment subject to the standards,
changes in the U.S. grid energy mix, adoption of a carbon tax, new ways to price electricity (e.g., by time
of use), and shifts in electricity and energy demand caused by a changing climate, may all significantly
affect the value of different attributes of the regulated products and equipment. For example, the
flexibility of operations and responsiveness of an appliance may be more valuable (in terms of reducing
electricity costs and externalities) than its energy consumption efficiency under a future scenario with
large shares of renewable energy in the grid. The committee’s view is that accounting for a wide range of
plausible technological, socio-demographic, policy, and climate scenarios is necessary for a proper
consideration of the span of costs and benefits brought by a regulation.
31

Path-Dependencies in DOE’s Analysis of the Societal Benefits and Costs of Standards
The standards setting process entails one of many choices that DOE must make as part of a
continuous decision-making process that spans decades into the future. As such, a further question on
adequacy of methods is whether DOE considers the path dependencies (Arthur, 1989) that may arise with
particular standards. Path dependencies can lead to sub-optimal efficiency as has been observed in the
United Kingdom’s interventions favoring adoption of a new technology for domestic heating (Gross and
Hanna, 2019). Each new standard puts DOE on a different path for achieving its long-term energy and
environmental goals. Each new standard makes some alternative courses of action infeasible and opens up
other ones. In particular, standards that require substantial changes in the technology and/or supply chains
used by manufacturers may set technical path dependencies that must be considered in the context of a
long-time horizon and DOE’s future actions. For example, it may be that a moderately stringent energy
efficiency standard set today precludes the evolution of more energy-efficient technologies in the future
because the factory upgrades or development of supply chains for complying with today’s standard, are
incompatible with those required alternative future technologies. The Global Energy Assessment (Ürge-
Vorsatz, 2012, pp. 709-710) considered energy efficiency in the buildings sector and the cost benefits of
two paths: one a suboptimal step followed by a top-up step; the other the path efficient method of
reaching the same efficiency level in one step. The study found that the former path risked lock-in
because the top-up step would not be cost-effective or perhaps even highly uneconomic. DOE already
performs an analysis of Cumulative Regulatory Burden as part of its assessment of Other Impacts of a
standard, but this analysis looks at the regulations 3 years before and after the compliance date of the
strategy and does not consider potential technological developments or standards further in the future (6,
12, and 18 years into the future).
Transparency, Clarity, and Replicability of Analysis
The second criterion that the committee applies to assess the standards-setting process is whether
DOE describes methods and assumptions in sufficient detail such that the analysis can be replicated, and
whether DOE communicates the final results clearly, properly characterizing the uncertainty that
surrounds them and the variability of outcomes across geographies, income-groups, types of households
etc.
Greater transparency in the TSDs DOE produces for each standard would make them more accessible
to those interested in understanding and commenting on DOE’s proposals. Facilitating input from diverse
parties early in the standard-development process could improve DOE’s analysis, the information on
which it depends, and ultimately the standards themselves. It may be possible that improving the
presentation of the TSDs to include summaries of data inputs and intermediate and final outputs of the
different analyses would lead to standards that reduce the costs of participation for different societal
groups.
The committee has also noted that although some information obtained by DOE are covered by
nondisclosure agreements (NDAs), transparency can still be achieved by describing the methods used to
gather that information and how that process leads to the inputs in the models. The committee comments
on the transparency in the assumptions, data gathering methods, and model structures because it thinks
that this allows DOE to evaluate the outcomes of the approaches used in past analyses, to learn from past
experience, and constantly improve their methods (as evaluated with the fifth criteria).
32

Adherence to State-of-the-Art Theory and Methods of Analysis Commensurate with Data

Availability and DOE’s Ability to Collect and Analyze New Data
The third criterion is whether the models DOE uses to characterize the products’ supply chains and
consumers are consistent with the most current knowledge regarding industrial organization, practices of
manufacturers and distributors, and consumers’ choices and behavior. The committee also looks at
whether the analysis makes full use of new sources of relevant data, new methods of data collection, and
of DOE’s own programs for data collection and analysis.
In particular, the committee looks at whether assumptions about consumers’ preferences regarding
products’ attributes, mode, and time of use, which have profound impact on estimates of energy costs and
emissions, are informed by data collection and analysis from DOE’s programs. The committee asks
whether DOE’s standards office capitalizes on all of DOE’s resources to enhance is data and information.
For example, does the Building Technologies Office maintain regular interaction with the Energy
Information Administration2 on the Residential Energy Consumption Survey (RECS) and Commercial
Building Energy Consumption Survey (CBECS) to ensure that the scope of the questions and the methods
of data collection gather key information on consumers’ valuation of a product’s attributes as well as
timing, intensity and mode of use? In addition, the committee asks whether DOE takes advantage of new
technologies for the collection of real-time energy use (with smart meters and in-home energy use
monitors) to gain a deeper understanding of any discrepancies between laboratory and real-setting results
and of the relationship between climatic conditions and the amount and timing of energy consumption of
different equipment.
Rigor in the Treatment of Variability and Uncertainty
The fourth criterion the committee considers is whether the methods used include an explicit
characterization of the uncertainty and variability in all of the factors that affect the welfare impacts of a
standard, and whether DOE clearly presents the variability in the impacts of the standards across different
segments and the uncertainty surrounding key assumptions.
Rigor in the Representation and Communication of Variability
The characteristics of the residential, commercial, or industrial buildings where the regulated products
and equipment are installed and used and the hydroclimatic and economic conditions of the place where
they are purchased determine the effects of energy efficiency standards. Geographical, socioeconomic,
and behavioral factors affect not only the costs of the equipment or amount of energy consumed, but also
the cost of such energy and its associated emissions. In addition to variability in the effects of a standard
for different types of consumers, there is widespread variability in the temporal effects; for the same
customer, standards may reduce the costs of energy and/or reduce associated emissions during some times
of the year and may increase it under other times. Thus, in applying this fourth criterion on treatment of
variability and uncertainty, the committee asks whether DOE’s methods consider the variability in the
characteristics of the economic, climatic, energy, and water systems where the customers of the regulated
products and equipment reside, and whether DOE communicates results acknowledging such variability.
Because energy efficiency standards are set for the entire United States after calculating the benefits at a
national level and for a horizon of three decades, DOE’s analysis presents the aggregated results for the
2
The Energy Information Administration (EIA) is a federal statistical agency created in 1977 by the U.S.
Department of Energy Reorganization Act of 1977. According to its website, the EIA “collects, analyzes, and
disseminates independent and impartial energy information to promote sound policymaking, efficient markets, and
public understanding of energy and its interaction with the economy and the environment.” Further information is
available at http://www.eia.doe.gov.
33

entire Nation and for the entire planning horizon. Nevertheless, executive orders necessitate having an
understanding of the effects on different populations and circumstances, the latter including consumers’
socioeconomic status, intensity, frequency and mode of use of the product, climatic or geographic region,
time of day, time of year. Accounting for variability across all of the dimensions that affect the impacts of
standards can both improve the usefulness of the final results and facilitate communicating the findings to
decision makers and the public. Once DOE has segmented manufacturers and users by attributes known
to influence the impacts of a standard, the uncertainty on standards impacts within each group will be
narrower.

Communicating the Variability on the Results
While DOE’s statutory authority envisions setting one standard for the entire nation, presenting
results at more granular level will be informative and likely improve regulatory outcomes. By displaying
the variability in intermediate and final outputs of its analysis at different levels of resolution (e.g., by
climate zone, socio economic status, type of consumer, state), DOE’s analysis will better contribute to
achieving DOE’s ultimate goal of reducing energy costs and emissions in this country. Compounding the
variability across multiple dimensions (e.g., climate zones, socioeconomic status, type of user, state and
local taxes, electricity prices) will result in a large number of scenarios to analyze. For example,
presenting results for each state (e.g., 50), income decile (i.e., 10), and frequency of use of the regulated
product (assume four categories) will result in 2000 subcases, though this number could be reduced by
substituting climate zones for states. It is the committee’s view that decision makers and the public will
benefit from obtaining detailed information on the factors that determine the impacts of a standard. It is
also the committee’s view that new tools for sharing and visualizing data sets enhance the value of
providing such information. For the TSDs, it may be enough that DOE presents summary statistics for
different dimensions. For example, DOE can show the estimated net benefits for each state, both as an
absolute metric and per-capita, or can rank the states by net benefits and discuss in detail the benefits for
the states in the 5th, 25th, 50th, 75th and 95th percentiles. It may also be useful for DOE to present
detailed results for the extremes of the ranges obtained when all dimensions of variability are
compounded. For example, DOE can describe the characteristics of the commercial establishments that
would see the highest reduction in energy costs when replacing a baseline refrigerator with one meeting
the proposed standard, or the characteristics of the households that benefit the least from purchasing a
new dishwasher that meets the standards vs. a dishwasher that met the old standards.
Rigor in the Representation and Communication of Uncertainty
A further question in applying the committee’s criterion on variability and uncertainty asks whether
uncertainty on all of the factors affecting the costs and benefits of a standard (on consumers, utilities,
manufacturers, and society) must be propagated throughout the analysis to obtain an estimate of the
probability distribution of all of the relevant quantities calculated. Following the RIA framework
described in OMB Circular A-4, including that circular’s guidance for understanding and presenting
uncertainty, would facilitate greater clarity that could be reflected in DOE’s Life Cycle Cost and Pay
Back Period Analysis, Energy and Water Use Analysis and Emissions Analyses (see Figure 2.1) and the
calculations of net present value of savings and so forth contained within.
In general, the uncertainty surrounding an estimate is the result of the compounded effects of model
and parametric uncertainty (Manning et al., 2004). Model uncertainty refers to the degree of confidence
on the representation of the relationship between different variables and parameters of the system
modeled. In contrast, parametric uncertainty refers to the degree of confidence on the values of the
parameters. For example, in the estimation of an unknown variable Y with the model Y = a + bX there
can be uncertainty about the parameters a and b (i.e., parametric uncertainty) or about the model itself
34

(i.e., if perhaps Y = a + bX2 better describes the real world than the original linear model considered). The
committee finds that there is uncertainty in some of the specific models used (e.g., model to estimate the
effects of technological learning, model to estimate the emissions associated with demand reduction,
model to estimate markups, engineering model to estimate the energy consumption of an equipment).
However, the higher-level models used to estimate many of the quantities affecting the final selection of
the standards (e.g., LCC, PBP) are straightforward and do not add additional uncertainty. Hence,
parametric uncertainty accounts for a large share of the entire uncertainty surrounding a final calculation
and hence, propagation of parametric uncertainty alone will add more depth and accuracy to the
uncertainty characterization of these quantities.
Communicating Uncertainty on the Results
Propagating the uncertainty through the analysis implies that the final estimate of each relevant
quantity is not a point but a probability distribution function or in some cases, a range of all of the
possible values it can take (Dudley et al., 2017). For some categories of appliances, uncertainty over the
impacts of the standards is likely to be large. Subsequent chapters discuss some of these cases. (See the
annex to Chapter 4 for an analysis of the TSD for residential dishwashers.) Presenting results as we
recommend here, with ranges of outcomes, has advantages. It can inform the agency about where its ex
post analyses should be focused: which standards are most in need of ex post evaluation due to a wide
range of possible impacts of the standard or which dimensions of the analysis are subject to maximal
uncertainty. As is developed in Chapters 3 and 4, in some cases, the actual energy use associated with
some appliances, the price and cost of appliances meeting a new standard, and the performance
characteristics and availability of new appliances can differ substantially from the ex ante “expected”
case, and can suggest where standard updates might be relevant. Characterizing and discussing the
uncertainty ex ante can inform where and how data and ex post analyses are likely to lead to regulatory
improvements.
Presenting the dependence between uncertainty and the costs and benefits of different regulatory
options can usefully guide investments in data used in the studies. As is discussed below, using emerging
technologies that allow much more granular information about appliance use, especially by heterogeneous
subgroups and expanding current surveys need not be resource intensive and may have large payoffs.
Finally, providing detailed information about the extent and sources of uncertainty behind the cost
and benefits estimates for standards can inform decisions and the public at large. Circular A-4 states “by
assessing the sources of uncertainty and the way in which benefit and cost estimates may be affected
under plausible assumptions, you can shape your analysis to inform decision makers and the public about
the effects and the uncertainties of alternative regulatory actions” (OMB, 2003).
Effectiveness in Setting a Path for Continuous Improvement of the Process to Set Energy Efficiency
Standards
Finally, the fifth criterion applied to these methods is whether they are designed to improve with time
as standards are designed, the market responds, and new data are available to test past assumptions and
considerations of variability and uncertainty. A single rulemaking resulting in establishment or revision of
a standard for appliances and equipment should be seen as just one stage in the long-term process that
revises standards every 6 years. In that context, the committee’s question is if DOE capitalizes on its
experience setting standards in the past, and sets up the infrastructure to learn from the outcomes of the
standards set today to improve its analysis in the future. Does DOE validate the assumptions made about
different factors affecting its estimates of net benefits of standards looking at past assumptions and
observed quantities? Does it conduct retrospective analysis to validate its engineering models? Does it
examine the accuracy of its models of technological learning in the manufacturing of different
equipment?
35

REFERENCES
Allcott, H., and M. Greenstone. 2012. “Is There an Energy Efficiency Gap?” Journal of Economic
Perspectives 26(1): 3-28.
Arthur, W.B. 1989. “Competing Technologies, Increasing Returns, and Lock-In by Historical Events.”
The Economic Journal 99(394): 116-131. March.
Clinton, W. 1993. “Executive Order 12866, Regulatory Planning and Review, and Amendments.”
https://www.archives.gov/files/federal-register/executive-orders/pdf/12866.pdf.
DOE (U.S. Department of Energy). 2014a. Technical Support Document: Energy Efficiency Program for
Consumer Products and Commercial and Industrial Equipment: Residential Dishwashers.
Washington, DC. December. https://www.regulations.gov/document/EERE-2014-BT-STD-0021-
0005.
DOE. 2014b. Technical Support Document: Energy Efficiency Program for Consumer Products and
Commercial and Industrial Equipment: Commercial Refrigeration Equipment. Washington, DC.
February. https://www.regulations.gov/document/EERE-2010-BT-STD-0003-0102.
DOE. 2016. Technical Support Document: Energy Efficiency Program for Consumer Products and
Commercial and Industrial Equipment: Residential Furnaces. Washington, DC. August 30.
https://www.regulations.gov/document/EERE-2014-BT-STD-0031-0217.
Dudley, S., R. Belzer, G. Blomquist, T. Brennan, C. Carrigan, J. Cordes, L. Cox, et al. 2017.
“Consumer’s Guide to Regulatory Impact Analysis: Ten Tips for Being an Informed
Policymaker.” Journal of Benefit-Cost Analysis 8(2): 187-204.
Fraas, A., and S.E. Miller. 2020. “Measuring Energy Efficiency: Accounting for the Hidden Costs of
Product Failure.” Economics of Energy & Environmental Policy 9(2).
Gayer, T., and W.K. Viscusi. 2013. “Overriding Consumer Preferences with Energy Regulations.”
Journal of Regulatory Economics 43: 248-264. https://doi-org.proxygw.wrlc.org/10.1007/s11149-
013-9210-2.
Gerarden, T.D., R.G. Newell, and R.N. Stavins. 2017. “Assessing the Energy-Efficiency Gap.” Journal of
Economic Literature 55(4): 1486-1525.
Hassett, K.A., and G.E. Metcalf. 1992. “Energy Tax Credits and Residential Conservation Investment.”
Working Paper No. 4020. Cambridge, MA: National Bureau of Economic Research.
Jaccard, M., J. Nyboer, C. Bataille, and B. Sadownik. 2003. “Modeling the Cost of Climate Policy:
Distinguishing Between Alternative Cost Definitions and Long-Run Cost Dynamics.” The Energy
Journal 24(1): 49-73.
Jaffe, A., and R. Stavins. 1994. “The Energy Paradox and the Diffusion of Conservation Technology.”
Resource and Energy Economics 16(2): 91-122.
Manning, M., M. Petit, D. Easterling, J. Murphy, A. Patwardhan, H-H. Rogner, R. Swart, G. Yohe, eds.
2004. Workshop Report: IPCC Workshop on Describing Scientific Uncertainties in Climate
Change to Support Analysis of Risk and of Options. National University of Ireland, Maynooth,
Co. Kildare, Ireland 11-13 May, 2004. Boulder, CO: Intergovernmental Panel on Climate Change
Working Group I Technical Support Unit.
NRC (National Research Council). 2010. Real Prospects for Energy Efficiency in the United States.
NRC. 2011. Modeling the Economics of Greenhouse Gas Mitigation: Summary of a Workshop.
OMB (Office of Management and Budget). 1997. Report to Congress on the Costs and Benefits of
Regulation. https://georgewbush-whitehouse.archives.gov/omb/inforeg/rcongress.html.
OMB. 2003. Circular A-4, Regulatory Analysis.
https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/circulars/A4/a-4.pdf.
OMB. 2010. Agency Checklist: Regulatory Impact Analysis.
https://www.whitehouse.gov/sites/default/files/omb/inforeg/regpol/RIA_Checklist.pdf.
36

OMB. 2011a. Circular A-4, Regulatory Impact Analysis: Frequently Asked Questions (FAQs).
https://obamawhitehouse.archives.gov/sites/default/files/omb/assets/OMB/circulars/a004/a-
4_FAQ.pdf.
OMB. 2011b. Circular A-4, Regulatory Impact Analysis: A Primer.
https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/inforeg/inforeg/regpol/circular-a-
4_regulatory-impact-analysis-a-primer.pdf.
RARG (Regulatory Analysis Review Group). 1980. Department of Energy’s proposed efficiency
standards for consumer appliances: Report of the Regulatory Analysis Review Group.
Washington, DC: Executive Office of the President. September 15.
Stern, P.C., K.B. Janda, M.A. Brown, L. Steg, E.L. Vine, and L. Lutzenhiser. 2016. “Opportunities and
Insights for Reducing Fossil Fuel Consumption by Households and Organizations.” Nature
Energy 1(5): 16043.
Sunstein, C. 2010. Memorandum for the Heads of Executive Departments and Agencies, and Independent
Regulatory Agencies: Information Collection Under the Paperwork Reduction Act. April 7, 2010.
https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/assets/inforeg/PRAPrimer_0407201
0.pdf.
37

Assessing Current Models: Engineering and Technology Cost

and Technology Performance
INTRODUCTION
The committee conducted a review of the Engineering and Screening Analysis based primarily on the
Technical Support Documents (TSDs) that support the Notice of Proposed Rulemaking for three
rulemakings supplied by the U.S. Department of Energy (DOE): standards for Residential Furnaces
(DOE, 2016), Residential Dishwashers (DOE, 2014b), and Commercial Refrigeration Equipment (DOE,
2014a). DOE tasked the committee with reviewing the regulatory analysis of these three products as
exhibits of its overall process. The committee also considered: the Federal Register notices promulgating
standards regulations and those describing the process for setting such regulations; briefings from DOE
and its contractors; and replies from DOE to the committee’s written queries. Overall the committee
found that the approach DOE used for the Engineering and Technology Cost and Technology
Performance, including the Pre-ANOPR (Pre-Advanced Notice of Proposed Rulemaking) Screening and
Analysis of Design Options was sound but could be improved.
SCREENING AND ANALYSIS OF DESIGN OPTIONS
There are at least two analyses DOE conducts prior to the publication of the Advanced Notice of
Proposed Rulemaking. The DOE Process Rule of 1996,1 which prevailed at the time the rulemakings the
committee considered were conducted, keeps these together under the heading, Pre-ANOPR Screening
and Analysis of Design Options.2,3,4 These include, first, the Screening Analysis, discussed further below.
This is followed by the Engineering Analysis, which includes a Performance Analysis and a Cost
Analysis.5 With respect to the Performance Analysis (also referred to as the Efficiency Analysis), DOE
1
U.S. Department of Energy (DOE), “Energy Conservation Program for Consumer Products: Procedures for
Consideration of New or Revised Energy Conservation Standards for Consumer Products: Notice of Final
Rulemaking,” Federal Register 61: 36974-36987.
2
This pre-ANOPR stage is to determine whether “candidate standard level is likely to produce the maximum
improvement in energy efficiency that is both technologically feasible and economically justified or constitute
significant energy savings.” Federal Register 85: 8705c.
3
See p. 36976b of DOE, “Energy Conservation Program for Consumer Products: Procedures for Consideration
of New or Revised Energy Conservation Standards for Consumer Products: Notice of Final Rulemaking.” Federal
Register 61: 36974-36987.
4
The “Process Rule” was updated in 2020. See DOE, “Energy Conservation Program for Appliance Standards:
Procedures for Evaluating Statutory Factors for Use in New or Revised Energy Conservation Standards,” Federal
Register 85(161): 50937-50944.
5
See, for example, slides 16 through 25 of DOE, “Commercial Refrigeration Equipment,” presentation to the
committee, November 19, 2019.
38

TABLE 3.1 Sequencing of Analyses Under the Process Rule

Preliminary Analyses NOPR Final Rule
Market and technology assessment Revised preliminary analyses Revised NOPR analyses
Screening analysis Life-cycle cost sub-group analysis
Engineering analysis Manufacturer impact analysis
Energy use determination Emissions impacts analysis
Markups for equipment price determination Monetization of emissions analysis
Life-cycle cost and payback period analysis Utility impact analysis
Shipments analysis Employment impact analysis
National impact analysis Regulatory impact analysis
NOTES: In the Residential Dishwashers rulemaking, DOE conducted the analyses listed in the column,
“Preliminary Analyses,” as part of the NOPR analysis. NOPR = Notice of Proposed Rulemaking.
SOURCE: DOE (2014b).
uses two different methods separately or in combination: an efficiency-level approach and a design-option
approach. Table 3.1 shows the broader context and sequencing of these analyses as the process of
standards progresses.
SCREENING ANALYSIS
Overview
The Screening Analysis precedes the Engineering Analysis.6 This first step screens all known design
options and sets aside those that (1) DOE deems technologically infeasible, (2) DOE views as impractical,
(3) will present an adverse impact on product utility or product availability, or (4) will cause an adverse
impact on health or safety. However, DOE revises screening decisions based on new information.7 The
analysis applies a strict filter that removes from further consideration technologies that do not meet
screening criteria. These criteria for exclusion will vary by the end-use appliance, but include the
following criteria:
 The technology option could cause impacts on product safety. Can another, safer option replicate
the increase in efficiency of this technology adoption?
 The option is technologically feasible but unproven. Because the definition for unproven is
vague, it would prove useful to label technologies according to their Technological Readiness
Level.
 The option causes a significant decrease in the functionality (product utility) of the appliance.
While there is no convenient method to measure functionality across the range of appliances
covered by DOE’s equipment standards, reviewing the appliance’s original purpose can
determine if compromises have been made.
6
See p. 36982c of DOE, “Energy Conservation Program for Consumer Products: Procedures for Consideration
of New or Revised Energy Conservation Standards for Consumer Products: Notice of Final Rulemaking,” Federal
Register 61: 36974-36987.
7
Ibid.
39

 The option is not practical to manufacture, install, and service. For example, DOE screens out
dishwashers that save energy and water by relying on a cold-water supply connection because it
assumes that all dishwashers in the United States are connected only to hot-water lines and is
impractical to change this. Consumers can take advantage of appliances that manage their own
water heating by turning down the thermostat on the household water heater thereby reducing
water heating energy.
 Done properly, the preliminary technical assessment phase will not exclude any reasonable
technology option and enhanced functionality, including, potentially, those not yet included in
appliances and equipment currently on the market. These may include attributes such as home
safety and improved indoor air quality. DOE should continue to consider how a standard can
reduce household energy beyond just the appliance’s energy, enhance broader household living
quality, and deliver other attributes.
In a preliminary assessment, the evaluated technologies need not be commercially available. This is
not explicitly stated in DOE’s documentation, and, for example, for Commercial Refrigeration Equipment
(CRE)—one of the three rulemakings shared in detail with the committee—DOE’s briefing material
states that: “DOE identifies technologies through general research, discussion with manufacturers and
other experts, review of equipment literature, and reverse engineering of equipment.”8 Similarly, by
eliminating alternatives that seem impractical to install -such as dishwashers certified for a cold-water
connection, DOE may be depriving customers of appliance features that can be advantageous—in this
case, reducing water-heating energy and enhancing home safety.
FINDING: Screening analyses are designed to be conservative and can therefore have the unintended
consequence of excluding specific technology options that, while not currently in use, may have
significant potential to enter the market.
Proprietary Technology
A further consideration is the treatment of propriety technology in the screening analysis. In a

briefing to the committee, DOE also suggested that they may screen out proprietary technology options.
Proprietary technology innovations can have a substantial efficiency impact on appliances. Many state-of-
the-art appliances are initially proprietary, but, later on, several firms manufacture them. Also, the time
period between rulemakings on a product is such that missing a rulemaking cycle could create a barrier to
deployment of more efficient technology lasting several years unless market forces result in deployment
of a cheaper, more efficient product or equipment. However, mandating a particular technology option as
a standard can offer a given developer an unfair advantage, which DOE cannot endorse. The DOE
program is further subject to review by the U.S. Department of Justice to ensure a candidate standard does
not lessen competition. This is important for the standard setting process since considering these nascent
technologies could provide DOE with justification for setting a higher energy efficiency standard for a
whole class of technologies, since technology developers have demonstrated (at least in the mid TRL
range) that advanced designs are possible and affordable.
Emergence of New Technology
There are several examples of DOE revising initial screening results based on new information. One
example is regarding variable-speed compressor technology implemented in portable air conditioners.
DOE’s initial evaluation for this technology determined that this technology option would offer no
8
DOE, “Commercial Refrigeration Equipment,” presentation to the committee, November 19, 2019.
40

measurable improvement (DOE, 2015). However, before DOE completed the analysis that accompanied
the Notice of Proposed Rulemaking (NOPR), variable speed compressors became available in a capacity
range suitable for portable air conditioners. DOE subsequently changed the screening results to allow
variable speed compressors to be considered.9 A second example involves using isobutane refrigerant for
residential refrigerators, refrigerator-freezers, and freezers as a replacement for HFC-134a refrigerant.
DOE had screened out isobutane as a technology option since it is somewhat flammable.10 Subsequent
studies of isobutane’s performance and safety in a controlled environment caused DOE to reconsider its
screening and allow isobutane as a refrigerant.11
In future assessments, DOE may find it useful to make a broader solicitation for comments on
screening and technical assessments while they remain in draft form. Transparency and public
engagement would likely improve the quality of the data and improve many aspects of the later analysis.
Similarly, for Residential Furnaces, DOE evaluated technology enhancements that they do not typically
include in baseline models. These enhancements have the potential to improve the Annual Fuel
Utilization Efficiency (AFUE) rating of furnaces. A key example is the development of a furnace that
fully condenses its flue gases. The process of fully condensing flue gases extracts 95% or greater of the
thermal energy of the furnace for beneficial use.
Technological Readiness
A well-developed and widely used method for identifying the readiness of a technology for
dissemination into the marketplace is the technology readiness level (TRL) taxonomy (see Box 3.1).
Developed by the NASA to manage its space-related research and development, TRLs are a convenient
means of describing the development stage of increasingly efficient technologies, devices, and their
components (NASEM, 2016).
BOX 3.1 Technology Readiness Levels
NASA and the U.S. Department of Energy have defined nine technology readiness levels:
1. Exploratory research
2. Technology concepts and application formulated, or both
3. Proof of concept validated
4. Subsystem or component validated in the laboratory
5. Limited field demonstration
6. Early field demonstration and refinements completed
7. Complete system demonstration
8. Early commercial deployment
9. Wide-scale commercial deployment
SOURCE: DOE G 413.3-4A. 2011. Technology Readiness Assessment Guide. September 15.
https://www2.lbl.gov/DIR/assets/docs/TRL%20guide.pdf.
9
DOE, “Examples of Screening Decisions Based on New Information,” personal communication from John
Cymbalsky, Office of Buildings Technology, to Martin Offutt, National Academies, June 2, 2020.
10
DOE, 2009, “Energy Conservation Standards for Residential Refrigerators, Refrigerator-Freezers, and
Freezers: Public Meeting and Availability of the Preliminary Technical Support Document,” Federal Register 74:
58915-58918. November 16.
11
DOE, 2011, “Energy Conservation Program: Energy Conservation Standards for Residential Refrigerators,
Refrigerator-Freezers, and Freezers: Final Rule.” Federal Register 76: 57516-57612. September 15.
41

The current DOE Appliance and Equipment Standards Program screens out technologies below TRLs
7, 8, or 9 depending on how the Pre-ANOPR criteria are applied. There are nonetheless technologies that
are at early, pre-competitive TRLs and have promise for use in consumer products and
commercial/industrial equipment as part of the product population analyzed. Some earlier, pre-
competitive technologies may ultimately have promise for use in appliances and equipment and bring
these into volume production so that they have a broad, positive impact on the energy efficiency. Energy
Star and utility demand side management programs, discussed in Chapter 5, may facilitate this.
Screening Analysis: Findings and Recommendations
FINDING: DOE has the ability to insert a new product for analysis after the screening analysis may have
initially ruled it out. Such can be justified especially if new information becomes available about a
screened-out technology.
RECOMMENDATION 3-1: DOE should consider technologies that are at early, pre-competitive
technological readiness levels and have promise for use in consumer products and
commercial/industrial equipment as part of product population analyzed, even if it seems plausible
that they will be screened out in later stages of the analysis such as in the Screening Analysis made
during the Notice of Proposed Rulemaking. DOE should continue to use the tools at its disposal,
such as reconsidering a previously excluded technology, to avoid prematurely screening-out
innovative technologies.
FINDING: The screening analysis may in some instances be removing new technologies that are
otherwise good candidates from further consideration.
FINDING: Ranking technologies by their proximity to market-readiness may help rationalize the
screening process.
RECOMMENDATION 3-2: DOE should adopt a taxonomy for labeling the readiness of
technologies, such as technology readiness level or “TRL” as it is widely known.
ENGINEERING ANALYSIS
Objective of DOE Engineering and Technology Analysis
The objective of the Engineering Analysis is to develop cost-efficiency relationships. DOE does this
by estimating the manufacturer’s costs of achieving increased efficiency levels (ELs) and determining the
maximum technologically feasible ELs. This relationship facilitates subsequent or downstream analyses
of consumer life-cycle cost, manufacturer impact, and employment impact. In the case of residential
dishwashers, DOE considers both electricity and water consumption. They establish these cost-efficiency
relationships for products with and without efficiency improvements.
It is unclear which future drivers of future costs, technology innovation, and consumer behavior DOE
considers in these analyses and how it formulates a framework using models to assess how these market
drivers will impact the creation and use of standards and other methods to improve end-use efficiency for
building and equipment performance. The committee offers observations on these future changes in
Chapter 6.
New technologies that DOE may consider within a product category can form two groups: (1) Those
that improve the energy efficiency of buildings and equipment and (2) those that more broadly increase
42

the functionality of devices. The second category may add features to the appliance or device or may
yield companion or secondary benefits such as reduced use of water consumption (e.g., in a dishwasher)
or reduced emissions (e.g., soapy water discharge in a clothes washer). The second category will likely
impact the market acceptance of products and therefore affect an evaluation of standards. DOE needs to
consider these exogenous factors as well.
Performance Analysis (Efficiency Analysis)
Within the Engineering Analysis, DOE conducts the Performance Analysis (Efficiency Analysis)
using either the efficiency-level approach (e.g., as with the residential dishwasher rulemaking), the design
option approach (e.g., the CRE rulemaking), or a combination of both (e.g., the Residential Furnaces
rulemaking).
Efficiency-Level Approach
This approach uses data from consumer purchases to estimate ELs for every product category. (These
ELs are ordered and numbered as EL1, EL2 and so forth.) As products reach the market, consumers
“vote” on the most desirable product that is available to them through their purchase actions. For any
product class, there is a distribution of products available of various efficiencies. Efficiency levels are
“discrete energy efficiency tiers that serve as potential standard levels.”12 DOE interviews manufacturers
to get their feedback to validate the selections. These efficiency levels correspond to “Tiers” suggested by
the Consortium for Energy Efficiency for utility energy efficiency appliance rebate programs.13
Design Option Approach
The Design Option approach uses detailed engineering calculations often based on computer
simulations of various design options. These simulations use engineering calculations that
deterministically calculate performance of end-use energy-consuming appliances and equipment. In some
cases, DOE identifies design levels through a complete physical tear down of actual products. The design
option approach is key to filling in gaps between efficiency levels and to determine the maximum level of
efficiency a given product can obtain. (Life-cycle costs including disposal are not considered in this step.)
The committee believes that understanding the impact of new and innovative technologies necessitates
evaluating the energy efficiency of innovative technologies, the consumer’s behavior in accepting the new
or updated products, and the consumer’s behavior in operating and maintaining the device where it
impacts market acceptance and energy consumption. DOE relies on expert opinions for the impacts of
consumer behavior.
FINDING: A structured program of market research that includes a combination of techniques would
provide more reliable information than techniques relying solely on expert opinion to analyze
consumer behavior. This is necessary to understand consumer behavior and appliance purchase
decisions.
12
See slide 15 of DOE, “Commercial Refrigeration Equipment,” presentation to the committee, November 19,
2019.
13
Understanding New Energy Star and CEE Tiers, J. Linsenmeyer, AC & Heating Connect, Emerson Electric
Company, 2020, http://www.ac-heatingconnect.com.
43

Consideration of Advanced Technology
The search for advanced technology involves understanding costs and performance both of devices
currently in the marketplace and of new technologies, whether proprietary, pre-competitive, or emerging.
Some of these technologies may offer still greater improvements in cost and performance. These new
technologies are often not yet incorporated in products on the market. (See Box 3.2.) It is possible that in
the course of considering which options are or are not technologically feasible,14 DOE may screen them
out at the beginning of the standards-setting process. Use of TRLs may provide useful indicator during
the process of considering this technological feasibility.
FINDING: DOE has thoroughly addressed the cost and performance of technologies on the market.
Analysis that considers new and improved technologies including the most innovative technologies
can complement the cost and performance analysis.
BOX 3.2 Advanced Technology Case Study
An example of a new technology is the scroll compressor. The scroll compressor was invented in
the 1990s. High-volume scroll manufacturing grew from 2 million in 1990 to 36 million units in 1998
to over 100 million in 2006,a enabled by technical and manufacturing improvements. Scroll technology
replaced the piston-based technology in refrigerant compressors for heating and cooling appliances.
These then-novel compressors accomplished superior compression than piston-based designs, using
only two moving parts, rather than the many components that move in a piston technology compressor.
This resulted in lower cost, higher efficiency, and longer life than the piston-based technology. Within
8 years of production, the scroll compressor achieved more than 60% market share of the residential
compressor market. The scroll compressor is a component in numerous applications for both air
conditioning and refrigeration.
During the debates concerning this new technology, some argued that scroll compressors should be
screened out since they were too novel and unproven in the marketplace. Without careful U.S.
Department of Energy oversight, the screening process would most likely have screened out this
innovative technology, which would have resulted in dramatically reducing introduction into the
market. The steps to develop a new, innovative, and “disruptive” technology like the scroll compressor
often entail high risks and large development costs.
a
J.P. Elson, N. Kaemmer, S. Wang, and M. Perevozchikov, 2008, “Scroll Technology: An Overview of Past,
Present and Future Developments,” International Compressor Engineering Conference, Purdue e-Pubs.
https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=2870&context=icec.
The pathway to develop new technologies notionally includes the following:
1. Emergence of candidate technologies that have the potential to meet or exceed the requirements
of existing products in the marketplace.
2. Development of a prototype that demonstrates the functional requirements of the appliance.
3. Funding the development, including an incubation period to achieve market acceptance.
4. Determining how to reduce the cost of new components, often caused by low volume production.
14
See slide 11 of John Cymbalsky, DOE, “Appliance and Equipment Standards Program Buildings Technology
Office,” presentation to the committee, November 19, 2019.
44

5. Exploring options to optimize the efficiency of the appliance beyond that which the new
technology contributes. This may be because of synergy between the disruptive technology and
the existing components in the product.
6. Creating a demand for the new technology and design production for high volume.
DOE participation in the development of new technology can, in certain instances, sharply accelerate
development and ultimate deployment (NRC, 2001). DOE could in principal balance its Appliance and
Equipment Standards Program with its other programs in the Building Technologies Office by crafting
standards that are more liberal in incentivizing innovation. Assessing this complementarity was beyond
the committee’s charge.
Alternative Means to Determine and Characterize Product Efficiency
Laboratory testing, field demonstrations, and modeling can augment both the efficiency-level
approach and the design option approach. DOE can also use modeling to combine computer simulations
with actual field data. These approaches are described below.
Laboratory Testing
Laboratory testing of small samples will not be sufficient for DOE’s purposes. The data reliability
and demand characteristics of an appliance population can differ substantially from data derived by a
small sample under laboratory test. However, laboratory testing can be useful to validate other results.
Empirical Data
The committee believes that engineering analysis is a useful activity but can be complemented by
monitoring performance in deployment of new or improved appliances. The optimal evaluation would
involve field tests of a population of similarly built homes or buildings to test two alternative technologies
and obtain reliable data. In residential tests, there is a dramatic difference between the energy
consumption measured in homes in differing climates and the same equipment measured in the laboratory
or simulated by a computer model. For example, without careful evaluation, it may appear that appliances
are poorly performing when in fact they were improperly installed. There can also be substantial
differences between samples due to consumer behavior. In tests of building space conditioning, some of
these could be obvious (i.e., thermostat settings) and others less so (opening windows and doors). For
example, thermostat settings can change the performance of an appliance, effectively de-rating the
BTU/kWh (British thermal units per kilowatt-hour) required.
Regarding heating, cooling and refrigeration appliances and devices, the Association of Heating,
Refrigeration and Air-conditioning Engineers (ASHRAE) has published a large portfolio of “Method-Of-
Test” procedures for various appliance types, including room air conditioners (ACs), refrigerant
expansion valves, refrigerant condensers, refrigerant compressors, liquid chillers, vending machines,
unitary ACs, commercial refrigerant cases, and air coils.
Variability and Uncertainty
As discussed in Chapter 2, in the section “Criteria and Principles Guiding This Review,” uncertainty
on factors affecting the costs and benefits of the standards must be propagated throughout the analysis to
obtain an estimate of the probability distribution of all of the relevant quantities calculated. Following the
regulatory analysis framework described in OMB Circular A-4, taking note of that circular’s guidance for
understanding and presenting uncertainty, would facilitate greater clarity that could be reflected in DOE’s
45

analyses. The correct efficiency rating of a consumer appliance or commercial equipment would reflect
performance from a “typical” installation. For example, in the case of residential air conditioning
equipment, the in situ performance could be characterized at various levels such as ideal, mid-range, and
non-ideal consumer behavior. These levels would reflect consumer behaviors involving fluctuating
thermostat settings or the opening of windows and doors.
A sufficiently large number of units of the same product/appliance needs to be installed and metered
to characterize the range of performance encountered in deployment. This may be costly and offer
unworkable delays when an accurate performance estimate for an appliance population is needed. With a
well-defined population and functioning metering equipment, the resultant load shape data collected will
represent the appliance population. In any sizeable population of end-use devices, some will perform
poorly or not at all, while other devices will perform as expected.
FINDING: The relationship of appliance performance to consumer behavior could be increasingly

relevant as appliances become smarter and are able to dynamically change their in-situ efficiency by
adapting to exogenous conditions.
RECOMMENDATION 3-3: DOE should characterize the uncertainty on the efficiency rating
in its analyses. Rather than providing a “point” estimate of efficiency, DOE should provide a
range that reflects the variability in energy consumption under different uses and consumer
behavior. This should reflect the need to account for the uncertainty arising from the range of
conditions under which a consumer may use a product and then using those use-case ranges to
establish sensitivity studies so that efficiency can be communicated as a range rather than a
single estimate.
New Data Sources
The ongoing explosion in the use of sensors, computational ability, and communications is evolving
an ever-increasing data file of consumption data at the appliance, building, and system level. For instance,
in web-connected appliances, some manufacturers are now incorporating. For example, once some smart
appliances are connected, the consumer is prompted to enter connectivity data and then receives
communications from the manufacturer offering the ability to monitor and manage consumptions. This
has the advantage of potentially reducing manufacturing cost and enabling new revenue streams. These
data can become the means to monitor end-use efficiency. Further discussion may be found in Chapter 6.
Data Aquistion and Cost Analysis
The Data Acquisition and Cost Analysis’ objective is to develop sound engineering cost analyses of
appliances with and without certain features. DOE prepares a cost-benefit analysis for various levels of
product efficiency to do so. The Engineering and Technology Cost Analysis is the first step in
determining if efficiency improvements are economically justified and is a key driver for all subsequent
analyses.
There are several elements to completing sound engineering analyses:
 Establishing representative material and labor prices.

 Choosing baseline models representative of the technology in the marketplace before
implementing the proposed rule making.
 Validating the models used to evaluate the cost-effectiveness of energy efficiency improvements.
 The Cost Analysis is necessary in providing the financial data to support a rigorous Regulatory
Impact Analysis (RIA) (see Chapter 2). DOE conducts the Cost Analysis portion of the
46

Engineering Analysis using one or more of a combination of cost approaches. (See Box 3.3.) The
choice of techniques depends on the level of public and manufacturer information available.
There are certain elements a cost analysis typically includes, which are listed and described
below.
— Ranges of cost data with accompanying bands of uncertainty. At present, DOE does not
include a representation of uncertainty in this step. These estimates are very dependent on
mark-ups by materials and costs incurred by manufacturers and internal pricing burdens.
Extra features are often market-based (what the market will bear over a base price),
which further obscures the real price, making it difficult to compare one manufacturer to
another. Also, the country of manufacture and the distribution channel adds further
uncertainty.
— Energy consumption patterns. The energy consumption patterns that are projected for an
appliance will typically include results by seasonal time-of-use (day, week, and month).
DOE’s existing practice focuses on aggregate estimates of market penetration, which are
not sufficient for the thorough Regulatory Impact Analysis (see Table 3.1 and Figure 2.1)
that it will increasingly need.
— Diversity factors. Diversity factors estimates to account for varying levels of performance
for appliance populations at the same Trial Standard Level (TSL) in these studies. In
addition to simulation or test results for one device, estimates should account for a
statistically significant population of devices.
— Energy demand estimates. Typically such estimates are derived for the product on an
hourly basis. This in addition to the availability of data on performance in different
climate zones—data that would beneficially reduce uncertainty and allow understanding
the user’s influence on the uncertainty.
— Ambient conditions. Variance of demand for Heating, Ventilation and Air Conditioning
(HVAC) devices are subjected to variance in ambient temperature and humidity.
Sampling studies done on a range of climate areas would also benefit understanding the
real energy impact of the appliances’ use and time-of-use.
RECOMMENDATION 3-4: DOE should launch a study to investigate improved methods for
data acquisition and analysis for use in setting and revising standards. This study should
include an overview of where DOE helps enable collection of these data now and how they can
improve that process at minimum impact on manufacturers.
Connecting Efficiency Levels (ELs) with Cost
DOE generates and uses cost curves to estimate life-cycle cost break-evens. These curves connect the
TSL point for energy efficiency and cost. This allows DOE to choose efficiency levels that reflect costs of
welfare-beneficial investments by consumers. Choosing ELs that yield the highest energy efficiency level
may not be economically justified if the energy savings and reduced externalities do not outweigh the
increased cost of purchase and installation. The choice of ELs considered is critical to the overall success
of an efficiency standard.
47

BOX 3.3 Data Collection Methods for Cost Analysis
Physical teardown. DOE physically dismantles a commercially available product, component-by-

component, and then estimates the costs of raw materials, production and machining processes,
purchased parts, capital depreciation, and factory overhead costs. DOE invites manufacturers to review
the results. This approach allows unbundling of features, illustrates different design paths, and
estimates manufacturer investment requirements by efficiency level. This method provides information
difficult to discern otherwise.
Catalogue teardown. In lieu of physically deconstructing a product, in a catalog teardown, DOE

identifies each component using parts diagrams. Teardown approaches are time intensive. However,
they allow DOE to consider a wide variety of capacities and features. For example, the 2016
Residential Furnaces Supplemental Notice of Proposed Rulemaking (SNOPR) analysis used data from
31 teardowns and generated an additional 46 data points using catalogue tear downs (DOE, 2016, pp.
5-9). Product teardowns are the most time-intensive of all of the costing approaches. They require
multiple iterations and often lead to several inquiries in order to determine reliable costing information.
Price surveys. When neither a physical nor catalogue teardown is feasible, DOE conducts retail price
surveys from retailer websites and other marketing materials coupled with assumptions regarding
distributor and retailer mark-ups. DOE gathers price surveys from public pricing data leveraging
technologies, including from the Internet. DOE also solicits prices from distributors.
Data scraping. Fueled by the Internet’s expansion and the increased use of sensors and
communications, data “scraping” is evolving as a less costly alternative to some costing approaches.
Scraping is taking bits of data from various sources, sometimes melded with data available from
regulatory authorities, electric utilities, state governments and public utility commissions. Trade
associations, literature reviews, and manufacturers data are increasingly online, available at trade
shows, or in trade publications publish various (what kind of) data available online. This data is not
only more accurate to use for manufacturer production cost (MPC), but the labor and time to analyze it
could result in reduced DOE expenditures. Validating this approach might involve retrospective
analyses of the impact of new or revised standards, taking into account the costs and energy savings
resulting from the consumer products and building/commercial equipment that are sold pursuant to and
are compliant with the new standard.
Findings and Conclusions
FINDING: The DOE innovation programs that comprise the Buildings Technologies Office are
supportive of new, innovative or disruptive technology used for or in consumer products and
buildings/industrial equipment. The Appliance and Equipment Standards Program is not explicitly
designed to do this. While disruptive products and equipment are a welcome find during the
Engineering Analysis, DOE’s Appliance and Equipment Standards Program has the effect of raising
the energy efficiency of existing devices and appliances incrementally.
FINDING: Disruptive or innovative technologies may emerge as a result alongside the standard
setting process of DOE’s appliance efficiency standards, and these may have a large effect on
increasing the efficiency with which energy services are delivered.
48

FINDING: A cost analysis typically includes a number of elements such as: ranges of cost data with
accompanying bands of uncertainty; energy consumption patterns; diversity factors; energy demand
estimates; and ambient conditions—all of which could be included in the Engineering Analysis to
improve its fidelity.
RECOMMENDATION 3-5: DOE should expand the Cost Analysis segment of the Engineering
Analysis to include ranges of costs, patterns of consumption, diversity factors, energy peak
demand, and variance regarding environmental factors.
ELECTRIC POWER SYSTEM
Many of the devices subject to the Appliance and Equipment Standards Program are electricity-
consuming. The standards consider the performance of such appliances and equipment to be the kilowatt
hours (kWh) of electricity consumed at the point of end-use. Nonetheless, any change in consumption
implied by the more efficient appliance is by itself incomplete without consideration of the timing and
source of the electricity that is to be consumed. This section discusses a few of the notions that take this
wider perspective on electricity consumption and how such notions dictate the value of the reduction or
change in electricity consumption beyond the mere kWh that might be saved.
Load Shape Modeling
In addition to relying on Engineering Technical performance for standards-making, the efficiency

data used to develop standards are critical for other energy analyses, sometimes referred to as “load shape
modeling.” In planning and ultimately operating an electric utility, the collective electricity demand of the
customer is critical. Load patterns determine the degree to which power plants are operated. Planners also
use the patterns to determine what type and what generation capacity of power plants a balancing
authority needs, how to operate them, what other resources to deploy, and the need for spinning reserves.
Making estimates of the future patterns in the timing and amounts of electricity demand is called load
shape modeling. These analyses rely on data that help characterize the end-use of energy-efficient
appliances. In that regard, estimating the impact of DOE standards is critical. Examples of load shape
modeling include Public Service Electric & Gas Company’s ELCS (Electric Load Curve Synthesis)
model (Gellings and Taylor, 1981) and the Electric Power Research Institute’s HELM, the Hourly
Electric Load Model.15 In the development of these models, it was important to replicate the behavior of
major electric end-use appliances, both concerning their demand over various time horizons and their
performance relative to variations in outside temperature and humidity. In addition, variations in
consumer behavior were simulated by assuming variations in the demand relative to outside air
temperature in winter conditions and weighted temperature-humidity in summer conditions.
Demand Response
The cost-efficiency relationship in the Engineering Analysis assumes that the only two determinants
for consideration in appliance standards are cost and efficiency. However, demand response (DR)
readiness is increasingly an important dimension in evaluating end-use appliances. There are three types
of demand response: emergency demand response; economic demand response; and ancillary services
demand response. DR readiness assures that a product can respond to each. This functionality is a
15
The Hourly Electric Load Model (HELM) was developed by ICF Incorporated for the Electric Power
Research Institute under EPRI Project RP-1955-1.
49

prerequisite for the product to become part of a portfolio of demand response capability for responding to
electricity market demands and system operator needs.
Electric power systems operate optimally when the demand imposed on them at each point in time is
equal to the supply available. Demand for electricity varies over the course of a day and year, while the
cost of generating electricity changes as well, depending on available resources. Economic demand
response can judiciously decrease demand so as to offset expensive generation, for example on hot
afternoons or when renewable power is unavailable.
Demand response programs can contribute to system operator needs as well. Ancillary services are
electrical services necessary to support the transmission of electric power from producers to energy
service companies and consumers. These services are determined in real time by grid operators within
those control areas to maintain reliable operations of the interconnected transmission system. Typical
services are those which provide various types of reserve capacity and assure system stability.
Emergency demand response is used in times of emergency such as a critical weather event, for
example, Superstorm Sandy or a large area blackout such as the Northeast blackout of 2003.
Demand response programs may include utility activities such as: the use of innovative rate designs
(e.g., Time-of-Use Pricing [TOU]; Critical Peak Pricing [CPP]; Extreme Day Pricing; or Real Time
Pricing [RTP]); use of direct load control of air conditioners or electric water heaters; the use of utility-
controlled customer-owned electric energy storage; or load control programs where a partial or full
customer outage is pre-negotiated with individual large consumers. As the amount of renewable resources
are added to the power system, the role of DR will become increasingly important in order to manage the
variability of these resources and keep the system in balance.
Consumers who adopt appliances that are DR-ready have the potential to participate in demand
response programs that lower consumer electric cost, increase the availability of electric service to
everyone else during demand peaks and enhance system reliability. For example, a study by the National
Renewable Energy Laboratory (NREL) revealed a potential saving ranging from 0.5% to 2.2% (Stoll et
al., 2017). DR is already an element of virtually every electricity market in the United States and is part of
the system economics, which affect nearly every consumer.
FINDING: The value of DR programs is substantial. While mandating Demand Response-ready is

beyond the charter of appliance energy efficiency standards, since it may not reduce on-site energy
use, DR often reduces system energy and reduces cost of supply capacity to meet demand.
FINDING: DOE Appliance Efficiency standards do not impede the inclusion of Demand Response
functionality in appliances and, where possible, may encourage the ability to take advantage of it.
RECOMMENDATION 3-6: DOE should consider Demand Response readiness as a factor in

cost-efficiency calculations. This necessitates the inclusion of power system benefits not
currently considered. (See also Recommendation 4-10.)
FINAL OBSERVATIONS
The committee believes that the most effective appliance efficiency standards are those which
incrementally improve the nation’s energy efficiency, and do not discourage or interfere with the
development and eventual adoption of disruptive technology. The Market and Technology Assessment
and Screening Analysis offer the best window through which to focus on that outcome. This is likewise
the analytical stage in which to include technology options that are near-commercial and offer promising
reductions in energy consumption and peak demand.
Some of the data and projections used in the suite of engineering and technology analyses discussed
in this chapter will have important consequences both for appliance and commercial equipment standards
and in other regulatory development mechanisms. The energy consumption patterns and amounts that are
50

projected for an appliance will need to include results by seasonal time-of-use (day, week, and month);
these projections will be important for the RIA as well. The use of Actual Test Data, including experience
from previously implemented standards, in place of computer simulation or to validate computer
simulations will improve the analytical outcomes. The analysis of the TSLs will hinge on properly
accounting for the variability and uncertainty in the cost and efficiency of the respective TSLs. Lastly,
there is an opportunity to modify models and approaches used in future analyses by performing a
retrospective review of some past predictions.
REFERENCES
0005.
Commercial and Industrial Equipment: Portable Air Conditioners. Washington, DC. February.
Gellings, C.W., and R.W. Taylor. 1981. “Electric Load Curve Synthesis—A Computer Simulation of an
Electric Utility Load Shape.” IEEE Transactions on Power Apparatus and Systems 100(1): 60-65.
January.
NASEM (National Academies of Sciences, Engineering, and Medicine). 2016. The Power of Change:
Innovation for Development and Deployment of Increasingly Clean Electric Power Technologies.
NRC (National Research Council). 2001. Energy Research at DOE: Was It Worth It? Energy Efficiency
Stoll, B., E. Buechler, and E. Hale. 2017. “The Value of Demand Response in Florida.” The Electricity
Journal 30(9): 57-64. November.
51

The Economic Analysis of Standards
INTRODUCTION
This chapter reviews the models and analyses that DOE uses to justify its proposed standards
economically. As is discussed in Chapter 1, the Energy Policy and Conservation Act (EPCA) requires
DOE to consider seven factors in assessing the benefits and costs of a standard (42 U.S.C. 6295
(o)(2)(B)(i)), including (1) the economic impact of the standard on the manufacturers and consumers of
the affected products, (2) the savings in operating costs throughout the estimated average life of the
product compared to any increases in the initial cost or maintenance expense, (3) the total projected
amount of energy savings likely to result directly from the imposition of the standard, (4) any lessening of
the utility or the performance of the products likely to result from the imposition of the standard, and (5)
the impact of any lessening of competition, as determined in writing by the Attorney General, that is
likely to result from the imposition of the standard.1 Economic modeling helps address these factors.
DOE uses the screening and engineering analyses discussed in Chapter 3 as its starting point for the
economic analysis. Those models estimate changes in unit energy use, performance, and production cost
associated with a potential efficiency level (EL). DOE’s economic models then trace through their
implications for consumer welfare and industry profits. DOE also considers whether a proposed standard
will have a disproportionate impact on subgroups such as lower-income consumers and small businesses.
In addition to EPCA, DOE considers executive orders in its review and evaluation of proposed
standards. Executive Order 12866 references the EPCA factors, but recognizing the difficulties in
calculating costs and benefits, focuses on broader principles underlying the imposition of regulations,
including why market conditions would lead to an expectation that a standard (or other form of
government intervention) would improve “the well-being of the American public”—what economists call
welfare. When markets function “perfectly,” market outcomes are economically efficient, meaning that
market intervention is not expected to yield positive net benefits to society. Roughly, economists deem a
market “perfect” when it features many buyers and sellers exchanging a good or service that does not
create externalities at publicly known prices, and neither buyers nor sellers have advantageous
information that the other lacks. Therefore, DOE’s economic analysis of standards proceeds against a
backdrop of presumed imperfection in markets. These imperfections, often called market failures, may
come from pollution externalities, manufacturers’’ market power, or consumer mistakes. Accordingly,
DOE’s economic analysis includes a discussion of the market failures relevant to the product in question.
In this chapter, the committee articulates several ways in which DOE’s analysis of standards’ impact
on producers and consumers deviates from a theoretical ideal suggested by economic reasoning. In some
cases, the committee suggests refinements. However, attempts to tweak the DOE process to address
1
Additional factors are (6) the need for national energy conservation and (7) other factors the Secretary
considers relevant.
52

individual shortcomings relative to the theoretical ideal may not always lead to a better answer. DOE
often analyzes products and equipment about which there is limited information about key economic
parameters. While the DOE process attends to both uncertainty and variability, it deals with complicated
multi-step processes, where it is often impossible to forecast how sensitive results are to assumptions
about uncertain parameters.
Thus, instead of calling for more detailed economic modeling, the committee’s overarching
recommendation is for DOE to seek additional data where feasible, increase the transparency of its
analyses by in many cases making explicit the sources and consequences of uncertainty, and give greater
attention to ex post analysis, which can offer insight into the reliability of its judgments and the
performance of its standards. Over time, DOE can fold those insights into the forward-looking standards
process. In brief, given the challenges of modeling complex competition in a data-poor environment,
DOE should take every effort to use past data and learn lessons ex post about firms’ behavior and the
evolution of products affected by standards.
The committee organized the discussion as follows. There is first a discussion of DOE’s modeling of
manufacturer behavior and the analyses of markups and markets. Then there is a look at consumers’
behavior and DOE’s characterization of consumer costs and benefits from an appliance standard. Energy
savings and environmental benefits affect both individual consumers and society at large; these are
discussed in subsequent sections. Finally, the committee returns to a discussion of market failures.
THE EFFECT OF STANDARDS ON PRODUCERS
From an economic perspective, appliance manufacturers come into the analysis of a proposed
standard in two ways. First, DOE’s statutory authority is to set standards that are economically justified,
and executive orders require it to consider whether the benefits of the standards exceed its costs, where
benefits and costs can include impacts on public health and safety and the natural environment. The
actions of producers (appliance manufacturers) and consumers contribute toward these benefits and costs.
There may also be externalities on both the consumer and producer sides. Therefore, DOE needs to
consider impacts on producers and consumers in performing the benefit-cost test and analyzing market
failures. Second, EPCA requires DOE to consider the impact on manufacturers and of any competition
reduction among manufacturers resulting from a regulation. That requires understanding manufacturers’
decision-making processes and the intricacies of competition among manufacturers.
This section considers DOE’s analysis of these topics on the manufacturer side. The conceptual
underpinning for that analysis is the economic theory of the producer—the theory of the firm and the
industry. The economic issues involved are not at all simple. The manufacturer of a regulated appliance is
a multi-product producer—the manufacturer produces more than one type of that particular appliance and,
often, produces other appliances. Thus, the firm operates in multiple markets. For each particular product,
the manufacturer must decide its price, features, method and quantity of production, and marketing. The
manufacturer typically makes these choices regarding multiple products, and there is likely
interdependence among their decisions. There can be joint costs and other forms of interaction among
decisions involving multiple products. It is important to note that the existing economic theory of firm
behavior does not operate at the level of granularity involved in these decisions. Moreover, the empirical
data required to implement theoretical models and test alternative theories at this level of granularity are
generally lacking. The text makes no assumption with regard to whether material costs or labor costs are
the dominant influence on the cost of manufacturing a given appliance. The point being made is that,
while DOE analyzes the potential impact of an efficiency standard on the appliance’s price in isolation,
without considering the pricing of other products that the manufacturers may produce, the reality is that
multi-product manufacturers are free to determine their product prices simultaneously and inter-
dependently across different products.
DOE greatly simplifies these issues in its analyses. It assesses the impact of an efficiency standard for
a given class of appliances on pricing, investment decisions, manufacturer profitability, and industry
53

competitiveness through the lens of the particular appliance viewed in isolation. DOE projects the markup
governing the appliance’s selling price and the impact on corporate investment requirements, financial
needs, and cashflow based on that one individual product in isolation. These projections deserve
significant attention because they simultaneously have substantive implications for the analysis and are
based on relatively thin evidence. In the absence of better data, more sophisticated economic modeling
may add little value. Thus, the committee’s overarching recommendation regarding the analysis of
producers is for DOE to collect and analyze real-world data that facilitate evaluation of assumptions about
markups, cashflow impacts, and other aspects of producer choice (Recommendation 4-1, below).
Below, the committee describe a conceptual framework for assessing producer behavior and profit.
The committee argues for a holistic view of the appliance manufacturer that posits that manufacturers sell
a product portfolio in imperfect competition. This market-setting implies that a standard that forces a firm
to alter a product impacts its profits by affecting that product and through indirect effects on substitute
products sold both by that firm and its competitors.
Conceptual Model of the Manufacturers
DOE wishes to characterize the likely impact of a standard on producers of the good. How should it
do so? For sellers, the relevant metric is profit, and so the pertinent question is whether a standard will
cause profits to rise or fall.
Most appliance markets feature many competing products that a handful of large firms produce.
Products are closely related but differentiated in their performance, size, styling, and other features.
Products’ design and production require substantial fixed costs (i.e., costs that do not vary with the
volume of goods produced). For most but not all appliances, products reach end consumers after passing
through a retail intermediary.
These conditions suggest that most producers sell a portfolio of competing products and set prices for
those products considering the degree of competition they face. This implies that the pass-through of costs
depends not just on a supply-and-demand elasticity but also on the degree of competition. An appliance
standard can cause the exit of existing products or introduce new products, which might impact the degree
of competition facing a given product. A critical feature of these markets is that each product’s price and
characteristics depend on the entire portfolio of related products. The field of industrial organization in
economics has well-established models that describe these conditions.2
Producers compete in price, energy efficiency, and other product features and attributes. Simplifying,
the committee refers to other features as “quality.” Standards may impact quality in one of two ways.
First, standards may force a change in product quality. Producers may sacrifice performance or other
features to achieve efficiency while limiting cost increases. Second, standards may cause firms to
compete differently on quality. This could lead to significant effects on consumer well-being and the
economic efficiency of the market.
One main reason for the effects of standards on consumer welfare and market efficiency is that sellers
can increase profits by segmenting a market into customers with higher and lower willingness to pay for
the product by offering products with different quality levels. Sellers may use energy efficiency to
achieve that product differentiation—both within and across sellers. If so, standards may limit the sellers’
ability to use energy efficiency to achieve product differentiation, as it becomes more expensive for the
seller to maintain differences in efficiency when the minimum is higher. This can lead to a drop in profits
and a shift in welfare toward consumers. It can also increase quality if firms add new features to achieve
product differentiation when energy efficiency differences have been compressed.
The committee notes that product differentiation may benefit consumers because it tailors products to
consumers’ needs and wants with different usage profiles. Where firms have substantial market power,
however, which appears to be the case in many of the product and equipment markets regulated by DOE’s
2
A standard approach, which motivates this discussion, is the Nash-Bertrand price competition model.
54

efficiency standards, the product differentiation can enhance that power. Moreover, in some
circumstances, incumbent producers can benefit from standards if they create barriers to entry that limit
future competition.
The Determination of Markups
How manufacturers respond to an appliance efficiency standard affects consumers’ welfare in two
ways: via the mix of features that the manufacturer decides to build into the appliance, as mentioned
above, and by the final retail price.
One component of the consumer’s final price is the manufacturer’s price as the product leaves the
factory, which in turn partially depends on the cost of manufacturing determined in the engineering
analysis in the Technical Support Documents (TSDs). The consumer’s final price also depends on
incremental price increases over cost added along the distribution chain. In its analysis, DOE combines
the manufacturer’s markup over its production cost together with the markups over cost that occur
through the distribution chain.
The overall markup is the difference between the manufacturers’ marginal cost of producing an extra
unit of the appliance product and the consumer’s final price. DOE routinely uses financial statements
(U.S. Securities and Exchange Commission Form 10-Ks)3 for publicly traded companies to estimate the
average markup on all products sold. They then average these results across firms in an industry to arrive
at an estimate of typical industry markups. In some cases, DOE may adjust these results based on
stakeholders’ comments, though this process is not transparent.
Given sparse data, this is a reasonable process for assessing the order of magnitude of markups
among manufacturers. However, it is far from perfect and leaves considerable uncertainty. Financial
statement data do not provide breakdowns among product classes, so all calculations are blind to markup
differences across appliance categories or different models. Moreover, many manufacturers are
subsidiaries of larger companies, so their data is obscured even further. DOE omits manufacturers that are
not publicly traded.
Even if DOE accurately measures the average markup for a class of appliances before changing the
standard, markups may change because of a new standard. DOE’s process makes this distinction in the
distribution and retail markup analysis, but not in consideration of manufacturer markups. Generally, the
marginal markup may be greater or smaller than the average.
An alternative approach would be to model the firms competing to sell differentiated products along
the lines discussed above. Methods of estimating markups as equilibria outcome of market competition
are standard in academic economics. Standard economic approaches rely on detailed data on prices and
market shares of alternative products (Berry et al., 1995). For some appliances, these data may not exist.
Moreover, these models may not accommodate the granularity of detail required for the appliance types
being DOE considers. These models have come under some criticism (Knittel, 2014), and they often
require additional adaptation to model market outcomes when products are changing.
The markup values derived from the current analysis are subject to considerable uncertainty, but DOE
uses this single point estimate of the markup in all downstream steps. Given the imperfections highlighted
above, the committee’s view is that greater attention to the markup analysis’s uncertainties is warranted.
The preceding discussion focused on the manufacturer markup. DOE also models markups in the
subsequent steps of the supply chain. For each product category, DOE models the transaction chains that
occur before a product reaches its final consumer. For many products, DOE considers multiple channels
when goods reach final users via different routes.
DOE estimates the markups for these steps in the wholesale or retail supply chain from financial
statements and sources like the U.S. Census Annual Retail Trade Survey (ARTS), which contains data on
operating expenses from each economic sector. Recently, DOE has focused attention on the ‘incremental
3
The form may be downloaded at https://www.sec.gov/files/form10-k.pdf.
55

markup’ in these steps, which attempts to isolate the input components, but is sensitive to assumptions
about operating costs and performance4
The incremental approach appears to be reasonable, but the committee notes the same issues as
above. DOE mostly pulls the markup estimates from financial filings and surveys representing costs and
revenues for firms that typically sell many product categories. The financial data DOE uses do not
represent all sellers. Uncertainty is thus substantial.
The committee finds that the DOE process of evaluating markups is sensible, but the true economic
context is complicated in ways that imply substantial uncertainty about how manufacturing cost will
ultimately impact consumer prices. DOE currently lacks the relevant data to verify its assumptions or
distinguish between alternatives, leaving significant uncertainty underrepresented in the current analysis.
The goal of the markup analysis is to estimate how an increase in the manufacturing cost translates
into market outcomes. Given the complexity of the markets in question, predictions about markups will
have substantial uncertainty unless much more data becomes available.
DOE is frequently evaluating technologies and products that are in the marketplace today. In such
cases, DOE can validate markup analysis against existing market prices. Such data may be difficult to
obtain, but DOE could put greater emphasis on systematically gathering such information.
In addition to current market data, ex post analysis could also serve an important validating role. In
some cases, DOE can obtain data on final consumer prices after it regulates an increase in energy
efficiency. The empirical evidence shows that, in some cases, the projected price increases did not
materialize. Indeed, in some cases, the outcome appears to be some degree of price reduction. Where
these outcomes occur, they have two economic implications. One implication is that the markup analysis
was not reliable. Another implication is that DOE’s economic analysis may have mischaracterized
manufacturers’ decision-making behavior. Moreover, if standards can lead to lower product prices and
better quality, more aggressive standards might be justified.
Spurlock (2013) found that the prices of given models of clothes washers, which had been declining
in real terms before the minimum energy efficiency standards in 2004 and 2007, dropped significantly at
the time the standards came into effect and thereafter began trending downward more quickly. The
average real price across all clothes washers did not change significantly at that time. Brucal and Roberts
(2019) reported similar results for clothes washers, dryers, refrigerators, and room air conditioners.
What can explain this? One explanation is learning-by-doing (learning curve effects): as cumulative
production grows, the manufacturer gains more experience in making the product and finds ways to lower
production costs. Learning-by-doing has certainly occurred with some appliances (Van Buskirk et al.,
2014), and DOE has incorporated learning curve phenomena in its price projections. But learning-by-
doing cannot account for the entire change—otherwise, the prices of regulated models of clothes washers
would not have declined so much more than the average price of all clothes washers.
Houde and Spurlock (2016) suggested standards may have spurred a reduction in market power and
an incentive to increase innovation. With clothes washers, the greatest within-model price reduction
occurred for the lowest efficiency washers. Moreover, there was a distinct increase in the number of
models offered in the highest efficiency group. There were also within-model price declines for washers
in the highest efficiency group. These changes are consistent with a model in which a firm engages in
strategic pricing and product differentiation. If there is significant heterogeneity in how consumers pay
attention to energy efficiency, it enables a firm to strategically segment customers and allows it to
exercise its market power.
Standards can also spur investments in research and development to generate innovations in the
mandated activities; indeed, there is likely to be low-hanging fruit in such areas because consumers
previously undervalued the activity. Newell et al. (1999) noted, for example, that an appliance
manufacturer might under-invest in energy efficiency innovation, something that government efficiency
standards could correct. In a dynamic context, firms might strategically withhold or delay cost-saving
4
See Dale et al. (2004). The article assumes cost pass-through does not affect a good’s demand,
notwithstanding changes in operating costs.
56

innovations from implementation to further exercise market power (e.g., Karp and Perloff, 1996; Kutsoati
and Zabojnik, 2005; Loury, 1979). DOE does not consider these are factors when it assesses impacts on
firm profits and consumer prices from an appliance standard.
As noted above, some economic models address these issues, but not with the granularity required to
deal with the real-world complexity of the industries DOE regulates. Moreover, sufficiently detailed data
to test these models may be lacking. Instead, empirical evidence, both current and after the imposition of
a standard, is needed to validate models, evaluate standards, and direct subsequent policies.
RECOMMENDATION 4-1: DOE should put greater weight on ex post and market-based
evidence of markups to project a more realistic range of likely effects of a standard on prices,
including the possibility that prices may fall. This would improve future analyses.
Manufacturer Impact Analysis
DOE conducts a manufacturer impact analysis (MIA) to estimate the financial impact of a proposed
appliance efficiency standard on manufacturers and assess the standards’ impacts on employment and
manufacturing capacity. DOE conducts the analysis in three phases and uses quantitative analyses and
qualitative evaluation.
Phase I, “Industry Profile,” consists of preliminary research directed at characterizing the appliance
manufacturing industry, which involves collecting data on market share, sales volumes and trends,
pricing, employment, and the industry financial structure.
In Phase II, “Industry Cash Flow,” DOE employs the Government Regulatory Impact Model (GRIM),
an industry cash-flow model customized for this rulemaking, to model the economic impact of appliance
standards on cash flow in the manufacturing industry as a whole. Appliance standards can affect
manufacturer cash flows in three ways: (1) by creating a need for increased investment, (2) by raising
production costs per unit, and (3) by altering revenue through higher per-unit prices or possible changes
in sales volumes or both (DOE, 2014, p. 12-2). The GRIM model attempts to account for these potential
cash flow impacts. GRIM draws on DOE’s engineering analysis, shipments model, census data, and
financial data from corporate reports and 10-K filings. It also incorporates information obtained by DOE
from interviews with manufacturers. GRIM projects annual cash flows using standard accounting
principles. The key GRIM output is the industry net present value (INPV), which is the discounted
present value of annual industry cash-flows over the analysis period. The analysis period is the
announcement year of amended energy conservation standards until several years after the standards
compliance date. DOE discounts the cash-flows by the industry weighted average cost of capital, as given
in S&P credit reports. DOE assesses the financial impact of alternative appliance efficiency standards by
comparing discounted cash flow under each standard. The qualitative part of the MIA addresses trends in
product characteristics, manufacturer characteristics, and standards’ impact on manufacturer subgroups.
In Phase III, “Subgroup Impact Analysis,” DOE evaluates the impacts of energy efficiency standards
on manufacturer cash flows, investments, and employment. Phase III also evaluates any impacts on
manufacturer subgroups, specifically focusing on the potential for disproportionate impacts on small
business manufacturers.
DOE’s interviews with manufacturers play an important role, providing feedback on the approaches
and data used in the GRIM analysis. The interviews provide DOE information to evaluate the impacts of
appliance standards on manufacturer cash flows, manufacturing capacities, and employment levels.
Examples of data obtained through interviews include capital investment costs for one-time changes in
plant, property, and equipment; costs for one-time investments in research, product development, testing,
and marketing; product cost structure, or the portion of the manufacturing production costs related to
materials, labor, overhead, and depreciation costs; and projected total shipment and shipment distribution
mix.
57

The interviews also provide manufacturers an opportunity to voice their concerns with potential new
appliance standards. DOE invites manufacturers to identify key issues they feel DOE should explore and
discuss further with them. With dishwashers, manufacturers raised issues about the impact on dishwasher
performance and issues with test procedures. Manufacturers of higher-efficiency products were also
concerned that they would experience increased competition as manufacturers that previously focused on
low-efficiency products moved into their target segment of the dishwasher market.
The GRIM analysis is a financial analysis rather than an economic analysis. However, it raises some
of the same issues as those discussed above connected to DOE’s markup analysis. Published corporate
financial data typically do not provide breakdowns among product classes, meaning they are blind to
differences in investment requirements and financial impacts across appliance categories or different
models. Some manufacturers are subsidiaries of larger companies, so their data is obscured even further.
Using average cost assumptions to develop an industry-wide cash flow estimate may not identify
differential impacts of appliance standards among different manufacturer subgroups such as small
companies manufacturing niche products. DOE uses the industry interviews to identify heterogeneity
within the industry, and it individualizes or segments the GRIM analysis to some degree to reflect inter-
firm differences. However, this process is not entirely transparent. Moreover, it is not clear how
accurately the GRIM cash flow analysis can reflect corporate behavior regarding the cross-product
allocation of production costs, strategic pricing, and market segmentation that could affect an efficiency
standard’s cash flow consequences for one of multiple products manufactured by a company. Finally, the
manufacturers impact analysis does not provide any information on how the current standard helps or
hinders the manufacturers’ ability to respond to a more stringent energy efficiency standard that DOE
may enact 6 and 12 years later.
RECOMMENDATION 4-2: To account properly for uncertainty and variability across

manufacturers, DOE should report ranges for the input values that feed the GRIM model and
run GRIM with the lower bound and upper values in the observed ranges. To make the MIA
more transparent, DOE should present its estimates of financial parameters and cost of capital
from publicly available sources and then report the adjusted values after the responses to
interviews have been considered.
THE EFFECT OF STANDARDS ON CONSUMERS
The consumer chooses from whatever appliance products the manufacturers decide to produce at
whatever final prices emerge from the distribution chain. They also base their choice on the energy and
water prices set by the retail utilities. The appliance has to work alongside the configuration of other
energy-using appliances, including with the existing fuel supply to the building, typically electricity or
natural gas, and with its energy management system, if one exists.
DOE wishes to characterize the likely impact of a standard on consumers. How should it do so?
When a household or individual uses an appliance in their home, the relevant metric is the change in
consumer utility (or welfare). Changes in consumer utility encompass changes in (1) the cost of
purchasing the appliance, (2) the cost of using it (and in the manner and frequency with which the
consumer uses it), and (3) product performance or other attributes that are of concern to the consumer.
When a business uses an appliance, the relevant metric is the impact on the business’ profit. In this
case, businesses determine profit by both cost and other product attributes that determine performance
and function. Moreover, businesses may sometimes have broader concerns than short-term profit (e.g.,
positioning itself as a socially responsible corporate member), which would require a broader metric.
This report will primarily discuss the household consumer case, understanding that concepts generally
carry over to commercial users.
This sub-section considers DOE’s analysis of consumer behavior and welfare. The committee first
describes a conceptual framework to evaluate DOE’s analysis. This framework begins with consumer
58

utility as the foundational metric. It then delineates the substantive choices made by consumers
throughout the life-cycle of the product: (1) ownership (e.g., does one own zero, one, or more
dishwashers), (2) product choice (e.g., which model a consumer chooses), (3) fuel choice (e.g., electric
versus gas-powered), (4) time, frequency, and mode of use (incorporating rebound), and (5) turnover
dynamics (including repair versus replace and disposal). The committee also discusses how to account for
possible consumer mistakes or lack of information.
The DOE analysis focuses on life-cycle cost (LCC) and payback period (PBP). The committee’s
discussion emphasizes that LCC and PBP estimates are imprecise due to the uncertainty and variability of
key inputs such as production costs, manufacturer and distributor markups, discount rates, electricity/fuel
prices, equipment performance, and consumer behavior. The committee discusses these issues in detail in
subsequent sections. Moreover, product attributes and behavioral responses imply that LCC and PBP are
insufficient to characterize consumer utility.
The conceptual model may not be the best one for regulatory purposes when economic models
require unverifiable assumptions, critical inputs are highly uncertain, or relevant data are unavailable. As
a result, the committee’s recommendations do include specific ideas on how to improve DOE’s analysis,
but even more, they emphasize the critical need for:
 Retrospective analysis to confirm assumptions and characterize error/uncertainty,

(Recommendation 4-1);
 Appropriate treatment of uncertainty and variability of findings (Recommendation 4-2);
 Analysis of how sensitive the results are to different assumptions so that DOE can prioritize data
collection efforts, (Recommendation 4-5); and
 A preference for simplicity where complexity may not improve accuracy.
Conceptual Model of Consumer Choice
Buyers of an appliance make a series of decisions. Heuristically, it is useful to outline these decisions
sequentially, although they may overlap or be made jointly. First, on the “extensive margin,” buyers
choose to have a number of appliances in their homes. In a given period, they decide whether to purchase
a new appliance as a replacement or an addition or to stick with what they have. In the former case, but
not the latter, they are “in the market” for an appliance. A consumer who decides to buy an appliance in a
given period must choose which appliance to buy out of those available in the market. Next, given the
consumer’s appliances, there is a choice on the “intensive margin,” which is how much to use each
appliance. One example of intensive margin is whether to run the dishwasher after every meal or less
frequently and in which operating mode (e.g., quick wash, full wash, etc.). The choice may depend on the
amount of energy used when running the appliance in a given mode and the energy’s costliness. However,
it may also depend on other features (attributes) of the appliance, like how noisy it is or how long the
cycle takes.
Like all consumer choices, the choices on both the extensive and intensive margins depend on the
consumer’s preferences and budget constraints. The latter reflects, in turn, how much money the
consumer has available to spend on goods and services.
DOE conceptualizes the consumer’s choice as a trade-off between two costs: the up-front purchase
price of an appliance (plus cost of installation) and the annual operating costs over the period the
consumer uses the appliance (e.g., how long the consumer lives in that house). At the heart of this trade-
off is the consumer’s discount rate: if there is a choice between buying a more expensive model but
saving on energy operating costs, how does a consumer compare those costs? If the consumer employs a
higher discount rate, they will weigh the upfront purchase price more heavily than the lifetime savings in
operating costs; if they use a low discount rate, they will do the reverse. The answer depends, in part, on
how the consumer finances the up-front purchase. Do they use a high-interest credit card, sell stock from
their portfolio, withdraw money from savings, or does the appliance store offer a deal for no payment for
59

the next 12 months? In some cases, the consumer may not be able to finance the purchase, which could
generate the same outcome as a high discount rate, namely avoiding appliances with a significant upfront
cost regardless of everything else.
However, the conceptualization of consumer choice as a trade-off between upfront appliance cost and
ongoing operating energy costs is likely to be overly simplified for several reasons.
For one, there is empirical evidence that many consumers do not know how much energy or water
they use when they run an appliance’s operating cycle. There is no meter that shows them how many
kWh of electricity or gallons of water they currently use. Surveys have found that consumer estimates
versus actuals of how much electricity or water they use tend to be widely off the mark.5 Moreover, many
consumers have little sense of what they are paying per unit for water and electricity.6 While an
appliance’s upfront purchase price is obvious, the operating cost in electricity and water is relatively
shrouded for many consumers, impairing their ability to make an informed trade-off. Moreover, it is
difficult to precisely predict the operating costs over the lifetime of the appliance. The challenge of
calculating these costs will vary across appliances and with the availability of official labels and
information from third-parties organizations or consumer reviews, all of which can influence the impact
of standards by providing greater information to consumers to improve choice.
Consumer preferences may also be considerably more complex. Appliances and their services boost
consumer well-being (utility); with a dishwasher, they can handle dirty dishes conveniently and
effectively, avoid streaks when washing glassware, and have an appliance that looks good in the kitchen.
The evidence from market research is that many attributes might affect appliance purchase decisions—
both qualitative and quantitative, both subjective and objective. (See discussion in Chapter 5 based on
EPRI, 1994.) Examples for dishwashers include brand loyalty, noise, reliability, repair costs, ease of
operation, convenient rack layout, drawer configuration, ability to hold extra-tall items, salt container,
rinse agent dispenser, etc. The attributes invoked for the purchase of a given commodity vary across
consumers and, for a given consumer, can vary across choice occasions depending on the context. The
presence of attributes in the utility function tends to lessen the importance of price as a driver of consumer
choice.
Another source of complexity in consumer choice is the numerosity of alternative products. For
example, in the case of residential dishwashers data plotted in DOE’s Technical Support Documents
implies that there are more than 230 basic models of standard residential dishwashers (DOE, 2014, p. 3-
36, Figure 3.14.1). Such numerosity is by no means uncommon among consumer products.7 Given the
numerosity of different models and brands of a given commodity, the evidence is that consumers often
choose from a limited subset of the available alternatives. In the marketing literature, this is referred to as
a consideration set. If an item is not within the consideration set, the consumer will not consider or select
it, regardless of price or attributes. The product set a consumer considers can be distinctive to that
individual and influenced by the consumer’s past shopping experience, habit, or exposure to information
or advertising.8
An appliance efficiency standard thus impacts consumer utility in several ways. First, it will alter the
set of products on the market by driving out non-compliant ones. Second, it will directly impact the
efficiency (hence operating cost) of some available appliances. Third, it will affect the equilibrium price
through incremental cost and incremental markup. (It may further impact prices through altering the
5
For energy, see Baird and Brier (1981), Kempton et al. (1985), Attari et al. (2010), Chen et al. (2015), Houde
and Myers (2019). For water, see Hamilton (1985), Beal et al. (2013), Attari (2014), and Araya et al. (2020).
6
See Ito (2014) and Brent and Ward (2019). There is a stark contrast with automobiles and gasoline use. As you
fill your car each week, you get a clear sense from the pump of both how much gasoline you have used and how
much it costs per gallon.
7
Nor is it a recent phenomenon. In 1896, the H.J. Heinz company introduced the marketing slogan, “57 pickle
varieties.” The slogan Heinz 57 varieties (omitting “pickles”) endured at least into the late 1970s.
8
The effect of appliance regulation may be to exclude certain products from the consideration set.
60

degree of competition among sellers.) Fourth, altering one attribute for efficiency may alter the other
attributes’ utility (e.g., increased energy efficiency may increase the run time of a dishwasher).
The committee finds that changes in demand due to a new appliance standard are influenced by
changes in installed cost and operating cost, but also by many other factors, including the substitutes
available on the market, product characteristics other than cost, and other related private and public
policies and campaigns. Because of the dependence of both markups and individual demand on the
complete set of alternatives available in the market, a standard can affect both characteristics and demand
for substitute appliances that exceed the efficiency standard. Given data on the appliances available in a
market (prices and attributes) and on choices by a sample of consumers, it has become common practice
in economics and market research to estimate what is known as a discrete choice model to explain
consumer choices. The model produces estimates of the utility weights placed by consumers on the
various attributes included in the choice model. These weights quantify how consumers make trade-offs
among alternative attributes and can be used to calculate the monetary value (willingness to pay) that
consumers place on changes (improvements or reductions) in attributes. Estimating these models requires
some judgment calls by the analyst, including choosing attributes, using objective data or subjective
perceptions to characterize them, and modeling consumer choices as limited by a consideration set and, if
so, how to compose that set.9 If DOE collected the appropriate data, it could formulate and estimate
discrete choice models of consumer behavior to quantify the trade-offs that consumers face from changes
in appliance attributes (performance). Alternatively, it could collect qualitative survey data about how
consumers value some appliance product attributes relative to others.
The committee recognizes that acquiring data for a study of the relation between appliance attributes,
consumer choice and consumer value is a challenge. The Energy Information Agency’s own surveys of
residential and commercial energy use provide valuable information on some appliances, as well as
private sources of information such as Nielsen; however, both the scope and timeliness of these data limit
their usefulness. The committee nevertheless urges DOE to attempt such analysis where feasible.
RECOMMENDATION 4-3: DOE should collect data on consumer choices in appliance

markets and estimate a discrete choice model of consumer behavior to quantify the trade-offs
that consumers face from changes in appliance performance.
While a discrete choice model can significantly improve both the assessment of relative standards and
the demand projections, the complexity of appliance markets means that projections of consumer
behavior in response to standards necessarily will be uncertain. As is discussed in Chapter 2, rigorous
evaluation of standards requires transparent treatment of uncertainty. Ex post analysis of prior standards
would be useful to validate the usefulness of these tools.
DOE’s Forecast of Product Purchases (Shipments Analysis)
DOE estimates future products’ purchases (i.e., shipments) using computer models calibrated against
historical data. These are stock-and-flow models representing the additions and subtractions of product
from the stock of in-use units any given year of the analysis period. The models begin with an estimate of
units’ stock at a reference year, broken down by age or vintage. DOE combines this estimate of the
reference year’s stock with estimates of product additions and product retirements to forecast the stock for
each year of the projection horizon. Depending on the product, the model may also differentiate products
by class (e.g., compact or standard dishwasher), market sector (e.g., residential or commercial), and
geographic region in addition to the vintage.
DOE estimates the initial stock of units from historical shipment’s data from resources like
Appliances Magazine and Appliance Design (AHAM), and from its assumptions on the lifetime of the
9
See, for example, Swait and Ben-Akiva (1987) and Shocker et al. (1991).
61

products to determine the proportion of units shipped in the past that remain in use in the present and their
vintage. DOE then combines this estimate of the reference year’s stock with estimates of product
additions and product retirements. Product additions are estimated as the sum of new installations and
replacements. Depending on the product (e.g., residential dishwashers or residential furnaces), it may be
assumed that new housing construction is the primary driver of new installations (installations for new
owners of the appliance). Shipments to replace damaged equipment are projected based on estimates of
the products’ lifetimes, sometimes assuming that the lifetime is affected by region. For example, Non-
Weatherized Gas Furnaces are assumed to have an average lifetime of 19.5 in the North and 23.5 years in
the rest of the country.
To assess the effect of standards, DOE estimates shipments both under the base-case scenario, when
there are no new energy efficiency standards, and assuming each of the Trial Standard Levels (TSLs) has
been adopted. To estimate the impacts of standards on consumer purchase decisions, DOE calculates the
Relative Price Elasticity of Demand. The relative price is the sum of the purchasing price and present
value of operating costs, divided by household income. The calculated elasticity is then combined with
the TSL-induced changes in the purchase price and operating costs to estimate shipments’ impacts.
The shipments model provides DOE with point-estimates of the number of units that will be retired
and added to the stock every year. The committee finds that these point-estimates fail to represent all of
the uncertainty surrounding the country’s trends in appliance/product purchases. Even if DOE accurately
estimated the changes in the purchase price and operating costs caused by a TSL, it would still be subject
to too much uncertainty about the customers’ discount rate to estimate the operating expense’s present
value. Additionally, even if DOE had perfect information about customers’ implicit discount rate, it
would fail to assess the relative-price elasticity of demand; the method DOE currently uses ignores
changes in appliance quality and consumer preferences. As pointed out in the above section, Conceptual
Model of Consumer Choice, changes in appliance quality and consumer preferences are important
determinants of consumer choice.
To consider the uncertainty on the input parameters, DOE must use the ranges (or probability
distribution, if possible) of the data available, not just the averages. For example, to estimate shipments of
new appliances, rather than using the estimates of new housing units for future years under the Annual
Energy Outlook (AEO) reference case, DOE can consider the range given by the AEO high-economic
growth and low-economic growth cases. Similarly, instead of using a point estimate of implicit
customers’ discount rate (e.g., 37% for dishwashers), it can use the range reported in the literature. The
upper and lower bound estimates of implicit costumers’ discount rate will propagate toward the
calculation and be reflected in upper and lower bound estimates of relative-price elasticity of demand.
RECOMMENDATION 4-4: The committee recommends that DOE propagate the uncertainty
in the shipments model’s input parameters and present the full range of shipment estimates.
Life-Cycle Cost and Payback Period
The primary input to DOE’s analysis of consumer benefits and cost is the life-cycle cost (LCC) and
payback period (PBP) analyses. While the calculation of an LCC from a societal perspective would
include the costs associated with all stages in the life cycle of a product, including manufacturing, use,
repair, re-use, recycle, and final disposal, to assess direct consumer’s costs, DOE includes only the costs
of acquisition, installation, and operation of an appliance. To estimate the economic impact of a standard,
DOE calculates the life-cycle savings for an average household, and for specified subgroups, taking into
account the appliance purchased with and without a standard (i.e., calculates the net benefit of the policy).
The final calculation thus relies on cost estimates for the compliant product and other available substitutes
and on purchase decisions of households. This section considers issues that arise in DOE’s estimates of
installed costs, operating costs, and overall household savings. Common concerns across the three
62

components are how DOE captures variability across households from the available data and models and
how it models and presents residual uncertainty.
Life-Cycle Cost Savings
To assess the effects of new or amended standards on consumers, DOE considers not just the LCC of
a particular product but the LCC savings relative to a base-case. To do this, DOE estimates the difference
between the LCC of the product a consumer would choose when there is a new or amended standard and
the LCC of the product the consumer would choose when there is not a standard (i.e., the “base-case
choice”). Therefore, DOE must assume consumers’ choices when there is and there is not a standard. The
shipments analysis provides guidance, including information about which households will realize the
savings by changing their purchasing decisions from a non-compliant to a compliant product. As is
discussed above in the section, Conceptual Model of Consumer Choice, such estimates are subject to very
significant uncertainty, compounded from uncertainty about the prices of different options and consumer
behavior. Consequently, the estimates of LCC savings provide a higher-level assessment of a standard but
are difficult to evaluate or interpret.
The committee finds that although DOE must consider the estimates of future shipments to calculate
the average of LCC savings of an EL for all consumers in the nation, it could better communicate the
results of its analysis and the variability and uncertainty of its LCC estimates by clarifying the
interpretation of its estimate LCC Savings given its dependence on shipments estimates.
It is helpful to look at the probability distributions and ranges of LCC savings in Figures 8.4.2
through 8.4.5 in the residential dishwasher TSD (DOE, 2014) to see the consequence of aggregating
product costs with consumer demand. They refer to the weighted LCC savings from purchasing a product
with an efficiency level equal to the TSL only for the consumers for whom this TSL is higher than the
efficiency level they would have chosen in the absence of the new or amended standard.
Without further explanation, presenting the LCC savings like this may make readers think that a
distribution of LCC savings concentrated around $0 indicates there is not much difference between the
energy efficiency of the baseline and the TSL. This is misleading because LCC savings of $0 may
correspond to a case when in fact, there is a significant difference between the energy efficiency of the
baseline product and the TSL, but the modeling has concluded that most consumers are already choosing
products with an energy efficiency that is equal to the TSL or higher. The standard may have considerable
upside risk should demand for the compliant standard be underestimated.
Considering multiple plausible base cases in the calculations of LCC savings would shed additional
light on the impacts that standards have on consumers. For example, in the case of dishwashers, three
base cases include (1) households without automatic dishwashers choosing between continued hand-
washing or the purchase of a new appliance; (2) households without an automatic dishwasher, choosing
between the baseline product and one that complies with a potential new standard; and (3) households
looking to replace a currently operational dishwasher.
Additional metrics that would illuminate a standard’s potential impact include the LCC savings from
a new purchase of a TSL relative to the baseline and the LCC savings from a replacement of a baseline
product with a new TSL.
The first calculation presents the difference between the LCC of purchasing a baseline product (i.e.,
the product that meets the current standard) and the LCC of purchasing the TSL product. It represents the
savings to a consumer purchasing a TSL product instead of purchasing the baseline. The committee thinks
that by presenting the distribution of LCC savings for each TSL relative to the baseline, DOE can clearly
convey the uncertainties on the initial and operating costs of a particular efficiency level without
contaminating this estimate with an estimation of shipments during the compliance year.
Lower values of the metric of LCC savings of a new purchase, assuming projected shipments, do not
necessarily indicate that savings potential of a TSL is low. DOE may want to note that low LCC estimates
63

-assuming estimates of future shipments- may signal that most consumers would be unaffected by the
standard because they already choose a higher efficiency level.
The second metric of value is the probability distribution of LCC savings of a TSL relative to the
replacement of a baseline product that a consumer already owns and can continue to use. By considering
both, the LCC savings from purchasing a TSL product instead of a baseline product and also considering
the LCC savings from replacing a currently owned baseline product with a TSL product DOE will obtain
and communicate additional information on the value of setting the standard at a TSL, of interest for
products where operating costs are a significant portion of life-cycle costs. Such savings are essential for
estimating the benefits of appliance replacement programs that state governments, utilities, or other
entities may consider to incentivize consumers to upgrade their appliances. Therefore, the calculation of
LCC savings of replacements may allow DOE to improve its estimates of shipments and the final
estimate of LCC savings at the national level. It is important to note that the committee’s recommendation
to present two additional LCC metrics does not impose any additional burden on DOE because DOE
already estimates all of the parameters necessary for these calculations.
RECOMMENDATION 4-5: DOE should make changes to the Technical Support Documents
underpinning its rulemakings to clearly communicate the dependence of the life-cycle cost
(LCC) calculation on shipments assumptions and thereby add clarity on the interpretation of
LCC savings. In order to clarify the engineering scope of a standard, apart from consumer
demand estimates, the technical support documents should include (1) LCC savings for one
consumer choosing between purchasing a baseline product or purchasing a TSL and (2) LCC
savings for one consumer that could continue to own a baseline product or replace it with a
TSL and (3) life-cycle cost savings for products or equipment that meet a given TSL as
compared to the baseline without adjusting for the assumed current and future distribution of
sales (shipments).
Variability and Uncertainty in the Estimate of Installation Costs and Operating Costs for Baseline
and TSL Products
DOE defines the consumer LCCs for appliances as the sum of installed costs, manufacturing cost,
manufacturer markup, distributor/retailer markup, taxes, installation costs (ICs), disposal costs, and
operating costs (OCs), which are a function of the life-cycle energy and water consumption, the prices of
water and energy throughout the lifetime of the product, and the costs of maintenance and repair. DOE
constructs an estimate of a product’s LCC to consumers by estimating each of these components. End of
life costs are generally omitted by DOE; of the three case studies reviewed by the committee, these are
only included as part of the installation costs of non-weatherized gas furnaces (furnaces; see DOE, 2016).
Life-cycle component costs vary significantly from consumer to consumer, depending on the
country’s region where the consumer lives, their socioeconomic conditions, and the product’s usage.
There is also significant uncertainty stemming from unknown values such as the future cost of materials,
labor, energy, and water, and the uncertainty over decisions made by manufacturers and retailers over
wholesale and retail markups.
Both variability and uncertainty raise challenges for estimating average IC and OC and subgroup
analyses. DOE addresses both challenges in its analyses, using the wealth of household-level data in the
Residential Energy Consumption Survey (RECS) for Monte Carlo simulations of operating cost. They
integrate some regional variation in the estimated installed costs and use random variables for some
parameters (such as frequency of dishwasher use) to characterize uncertainty. The committee identified
several possible modifications and extensions to the analyses that would allow DOE to better simulate the
spread of LCC values that result from disparate regions and user’s socioeconomic conditions and
behavior. These are summarized here and detailed in the annex to the chapter.
64

It is useful to start with an example. For residential dishwashers, DOE uses Monte Carlo simulation
based on data from the 7,382 households that own such an appliance according to the 2009 RECS. This
RECS sample represents 67.4 million households, each of the 7,382 representing a different number of
households (which is accounted for in each household’s “weight”). DOE takes 10,000 random draws of
the RECS sample, based on each household’s weight, and calculates the LCC savings for each efficiency
level (relative to 2019 expected shipments) for each one. DOE takes information about the state or region,
water heating fuel type, number of cycles the dishwasher runs per year, and household income group from
each RECS household record. DOE estimates the value of some components of installation and operation
costs with this information and compounds these estimates with other values previously estimated—that
do not incorporate household or region characteristics—to calculate the LCC savings. DOE then
generates the ranges and distributions of LCC savings from the 10,000 LCC Monte Carlo trials.
DOE could augment this process by expanding the number of variables in the trials to other
variability sources, expanding the number of trials,10 and substituting point estimates with random
variables for uncertain parameters. For example, additional regional sources of variation for installed
costs include labor costs, retail markups, and in some cases real estate costs. Greater use can be made of
RECS information, although as discussed further below, it is limited in the range of appliances included
in the survey as well as its structure and timeliness (NRC, 2012).
Uncertainty and Installed Costs
Installed costs are the expenses that consumers must incur before the appliance or piece of equipment
is ready to be used in their residences or businesses. DOE estimates installed costs for the baseline and
TSL products, adding sales taxes and installation costs to the result of the markup analysis. See Box 4.1
for the consumer expenses DOE calculates.
BOX 4.1 Consumer Expenses Included in Analysis
 The manufacturer cost for the baseline product in the year of analysis
 The factor to account for manufacturers cost declines due to learning, between the analysis
year and the compliance year
 The manufacturer’s cost increase for each TSL
 The manufacturer’s markup for the baseline product
 The retailer markup for the baseline product
 Incremental retailer markup for EL products
 Sales taxes
 Installation costs
DOE estimates most components of installed costs without considering the specific location of the
household. For dishwashers, the exception is the sales tax, which DOE estimates based on the RECS
information about the specific state or state-group where the household is located.
As is discussed above, uncertainty is likely to be important in estimating both the cost of new
products and the wholesale and retail markups. Ex post evaluations are critical to assess the impact of a
10
The committee recommends increasing the number of Monte Carlo runs to ensure that all households in
RECS are sampled at least once. In the case of dishwashers, the number of Monte Carlo runs should increase by at
least 70 times (i.e., from 10,000 to 700,000) to increase the chances of drawing each of 7,382 households in the
RECS sample at least once in the Monte Carlo trials.
65

standard, validate models, and inform policy changes. However, in addition, explicit incorporation of
uncertainty can better inform the LCC. For some parameters, the uncertainty characterization can be
informed by the same data sources DOE already uses. For other essential parameters, the committee
recommends extending its data collection efforts in the engineering analysis.
For example, DOE estimated the incremental retailer markup for dishwashers of different efficiency
levels—relative to the baseline—as 11% (i.e., the retailer price of a dishwasher is estimated by DOE to be
1.11 times the manufacturer price). DOE could instead characterize the parameter as a random variable
with a minimum value of 10.3% and maximum of 11.8% based on the same data that DOE already uses,
as discussed in the Annex.
An example of a parameter for which DOE can collect probabilistic information instead of a point
estimate is the incremental cost of manufacturing products of different efficiency levels. For the case of
residential dishwashers, the engineering analysis indicates that incremental costs for levels EL1 through
EL4 are $9.52, $36.53, $74.72, and $74.72 dollars in 2013 dollars. Rather than point estimates, the
engineering analysis could yield probability distribution functions representing uncertainty on the
quantity and cost of the materials, labor, and specific design used to produce an appliance with the given
efficiency level.
RECOMMENDATION 4-6: DOE should improve the representation of variability and

uncertainty on Installed Costs by considering the variation in costs components across states
and by leveraging the engineering analysis to obtain a probabilistic characterization of costs
components.
Estimation of Operating Costs
Dishwashers provide a useful example of both the complexity of estimating operating costs and
opportunities for further refinements. Operating costs include all of the expenses included by owners of
the appliance or equipment between the moment it is installed and the end of its useful life. DOE
estimates operating costs as a function of over a dozen parameters, characterizing how households use the
appliance (number of cycles), household discount rates, appliance characteristics such as energy and
water use, and the price of electricity, of heating water by various means, and repairs. See Box 4.2 for a
list of operating costs DOE includes in its estimates.
RECS only includes household data on some of these parameters: frequency of use (i.e., cycles per
year), the fuel used for water heating (i.e., electricity, gas, or oil), and the geographical variability in
annual prices of water and energy. Other essential components of operating costs include water heater
efficiency and water temperature-rise requirements which vary across hydroclimatic regions and across
seasons, but this is not accounted for in DOE’s analysis. In this case, using RECS data to find the state of
the household and select the water heater’s appropriate efficiency and required water temperature rises
would be preferable.
66

BOX 4.2 Factors Used in Calculating Operating Costs Included in Analysis
 Number of cycles per year

 Duration of the cycle
 Electricity consumption during a cycle for baseline product and products of different efficiency
levels
 Electricity consumption during standby
 Number of standby hours per year
 Energy consumption for water heating per cycle
 Annual water use
 Energy consumption for water heating per cycle
 Annual energy consumption from electric, gas, or oil water heaters
 Electricity, natural gas, and LPG
 Oil and water prices in the analysis year
 Future changes in electricity, natural gas, LPG, oil and water prices
 Electricity prices in the compliance year
 Repair and maintenance costs
 Product lifetime
 Consumer’s discount rates
For some other parameters, DOE considers geographic variability but not at the level or resolution
that is both possible and desirable. For example, while DOE considers energy prices for 27 states or
groups of states, they only consider the resolution of water (and wastewater) price variability by looking
at the values in four regions. In this case, using the water prices specific to the state of the household in
the RECS sample, instead of using an average value for one of four regions to which the state belongs
would be preferable.
The committee also finds that the representation of uncertainty on operational costs can significantly
improve if DOE modifies how it records the responses to questions already asked by RECS. For example,
in the case of dishwashers, the RECS asks users about the frequency of use of this appliance. The
response is stored in one of five categories: less than once per week, once per week, 2-3 times per week,
4-6 times per week, at least once a day. Because the responses stored in these five categories cannot be
directly used by DOE as an input into the LCC calculations DOE must make assumptions without any
empirical support. The modeling of this parameter as random variable with a distribution and parameters
“guessed” by DOE, could be avoided by changing the way RECS collects responses to a question it
already asks. In this case, RECS could give respondents the opportunity to specify the number of times
they use the dishwasher per day, week, month, or year.
The LCC calculations can also significantly improve if DOE changes how it documents the
engineering analysis results concerning annual energy use for each efficiency level. For dishwashers, the
LCC calculations rely on an estimate of each efficiency level’s yearly energy consumption. However,
instead of annual energy consumption, LCC calculations require data on energy consumption per cycle,
disaggregated by function to correctly differentiate the electricity consumption from energy for water
heating. Similarly, DOE must make assumptions about the duration of a cycle and the number of hours a
dishwasher is idle to estimate standby energy. DOE could avoid making these guesses if it documented
the results from the engineering analysis at the disaggregation level needed for LCC calculations.
67

Finally, the committee recommends that DOE prioritize collecting data on parameters that are
essential to LCC calculations, such as the lifetime of the products analyzed. Until enough data is collected
to use in its analyses, DOE must represent this crucial input’s lack of knowledge by characterizing
uncertainty based on information from focus groups, expert elicitations, or educated guesses based on
engineering analysis. For dishwashers, DOE notes that although products with higher efficiency may have
a higher probability of failure due to the increased complexity and number of parts, there is not enough
data to include this consideration, notwithstanding that the mere possibility of a reduced lifetime of higher
efficiency products warrants consideration in the LCC.
RECOMMENDATION 4-7: DOE should improve the accuracy of its estimates of all of the
LCC calculation components by (1) taking full advantage of disaggregated data to account for
geographical and temporal variability when available, (2) specifying probability distributions
instead of one-point estimates and compounding or propagating the uncertainty they represent
throughout the calculation, (3) better recording the data collected by RECS to avoid losing
information provided by respondents, (4) better documenting the engineering analysis to obtain
disaggregated probabilistic information necessary for the LCC, (5) prioritizing the collection of
information for parameters likely to have a significant impact such as the lifetime of a product
(i.e., durability), and (6) validating the assumptions made in previous analyzes with data
collection through the engineering analyses, focus groups with manufacturers, retailers,
consumers, and other means.
As noted before, DOE, in general, omits the consideration of consumer disposal fees in the
calculation of consumer LCC savings. This omission is inconsequential in calculating costs to consumers
if there is no correlation between a product’s disposal fees and its energy efficiency. This is often the
case, as disposal fees are generally determined by the appliance’s category and size (i.e., one large
refrigerator). However, from a societal perspective, differences in the materials and components used for
more efficient equipment may cause changes in the environmental consequences associated with disposal
or scrappage. The committee recommends that DOE’s engineering analysis collects data on the entire life
cycle of baseline and EL products, conducts a comparative environmental life-cycle assessment (LCA),
and accounts for any differences in its calculation of the NPV of proposed standards.
ENERGY CONSERVED
In Situ Performance
DOE develops a rigorous test procedure to judge the efficiency of products and equipment.
Standardized laboratory tests are necessary for offering consistent comparisons across standard levels and
eventually for certifying products. However, in situ performance may differ significantly from laboratory
tests because of usage patterns, maintenance, installation quality, or ambient conditions. At the
certification stage, products may be tailored to the test procedure, or test results may even be manipulated.
The committee refers to any difference between a test rating and in situ performance as the performance
gap.
Currently, DOE simply assumes that the test procedure rating is accurate. For some appliances, DOE
uses consumer data to look at heterogeneity in use intensity and sometimes calculates regional differences
in average ambient conditions. But it does not appear that DOE attempts to determine the in-situ
performance gap. This could matter when assessing technologies because some may perform worse in
practice than others. A performance gap may also lead DOE to choose the wrong efficiency level if test
ratings provide a biased estimate of true energy savings.
There are several reasons to suspect that a performance gap may be significant. Recent literature has
demonstrated that the performance of energy efficiency retrofits is often far below ex ante expectations
68

(Fowlie et al., 2018) and that the quality of installation is an important determinant of performance
(Blonz, 2019; Christensen et al., 2020). Where appliance performance depends on complementary
components or regular maintenance, a performance gap can arise if users do not follow best practices.
An assessment of in situ efficiency and the possible performance gap is difficult for several reasons.
The main obstacle is that appliance-level energy consumption data taken in the field is rarely available.
Even then, however, DOE could develop test procedures that quantify the degree of expected variability
and test the robustness of alternative technologies or designs to variance in maintenance, use, or
installation. Such a modeling effort may or may not be warranted depending on costs and expected
variation.
Fortunately, data on the performance of individual appliances will probably be available soon. Much
data of this sort is now being collected implicitly by utilities and manufacturer, trade, professional, and
consumer associations and groups. In terms of direct production cost, it would be inexpensive to greatly
expand this sort of data collection among new appliances. DOE should do everything possible to foster
the generation of these data, to collect them, and to analyze them to judge appliances based on their actual
performance in the field, which is what matters for energy conservation and consumer welfare.
Of course, even where data are available, it may be difficult to judge in situ performance for a
prospective standard or feature where the products and technologies in question are not in the market or
that not many consumers use. Even in this situation, data on existing products could be useful in assessing
typical performance gaps and variations in performance across space and settings and over time.
RECOMMENDATION 4-8: DOE should seek to gather and make use of in situ performance
data wherever possible to account for any performance gaps. When estimates of situ
performance data are unavailable, DOE should include a qualitative assessment of the potential
for a performance gap. Indicators of performance include maintenance requirements and
product lifetime as well as energy and water consumption.
The Rebound Effect
An improvement in the energy efficiency of an appliance lowers the operating cost of the appliance. It
thus lowers the cost that end-users face to obtain services from the appliance. This may induce a rebound
effect (or “take-back”): because energy services are cheaper, end users may increase their usage of the
appliance and therefore their energy consumption. Whether this happens depends, in part, on the extent to
which end users are aware of the amount of energy they are using when they run the appliance and how
much that energy costs them. This may vary with both the type of end user (commercial versus
residential) and the type of appliance.
The rebound effect has distinct implications for quantifying energy conservation and for evaluating
consumer welfare. Regarding energy conservation, the rebound effect should be accounted for whenever
it exists. The DOE process allows for consideration of rebound, but it does not appear to be applied
uniformly. Of the three case studies considered by the committee, DOE only discusses rebound in one
(furnaces; see DOE, 2016). In that instance, the DOE technical support document adjusts for the rebound
effect.
The implications of the rebound effect on consumer welfare are somewhat different in nature.
Whenever something occurs to lower the price of a commodity—in this case services from the operation
of an appliance—the consumer experiences an increase in welfare (utility), regardless of whether her
demand for the commodity (her utilization of the appliance) increases.
Furthermore, any price change has two distinct types of impact on a consumer—a substitution effect
and an income effect. The substitution effect is associated with the change in the price of the given
commodity relative to the prices of other commodities—this is what underlies the rebound effect. The
income effect is associated with a price change that leaves the consumer with more money (in the case of
a price decrease) or less money (in the case of a price increase) to spend on commodities. In the case of a
69

price decrease, such as is implied by the improved energy efficiency of an appliance, the extra income
that becomes available may be spent to some degree on other goods that require the consumer to use more
energy overall. This has been called an indirect rebound effect—it is driven by the consumer’s propensity
to spend income on energy-using consumption activities. DOE does not currently consider this, which the
committee finds appropriate for purposes of calculating consumer welfare; that is, after accounting for
monetary savings, welfare analysis does not require determining how that money is spent.
The case study that considers rebound (furnaces) says that the welfare effect on consumers of rebound
is approximately zero because the benefits roughly cancel out with the additional utilization costs. It is
correct that these factors offset, but it is more precise to say that rebound must be a benefit, and it is
feasible to approximate it and see if it is of potential significance.
An exact calculation of the welfare gain from the rebound effect requires information about the
demand curve for appliance services. But it is straightforward to use an approximation of the welfare gain
for consumers by assuming a linear approximation to the energy services demand curve, and then
calculating the welfare gain “triangle” by multiplying the estimated rebound effect by the change in the
operating cost. This requires no more information than is used to calculate the rebound effect’s
implications for energy take-back.
RECOMMENDATION 4-9: For purposes of calculating changes in energy use, DOE should
consider direct rebound wherever possible; if DOE believes there to be minimal rebound, they
should document the reasons why. However, consumer welfare should be understood to benefit
from rebound, rather than be harmed by it, notwithstanding the implied increase in energy use.
Approximations of the welfare gain from rebound can be incorporated wherever sufficient
information allows.
Energy Saved Versus Dollars Saved
Energy efficiency standards pursue the dual goals of conserving energy and saving money. EPCA
directs DOE to design standards to “achieve the maximum improvement in energy [or water] efficiency
. . . which the Secretary determines is technologically feasible and economically justified” (42 U.S.C. §
6295(o) and (42 U.S.C. § 6313(a)(6)(B)(iii)). Conserving energy and saving money are closely related,
but there has always been some divergence between the two because the cost of producing electricity
varies significantly at different hours of the day and at different times of the year. As such, energy saved
at different hours and on different days has different economic benefits.
Until recently, this nuance could safely be ignored when evaluating the impact of a standard on
residential and commercial customers because nearly all such electricity users faced flat rates. That is,
they paid the same price for a kWh regardless of the time of day.
But this fact is changing. Smart meters that allow for more complex rate designs are now widespread,
with more than 95 million installed in the United States (EIA, 2020). Customers increasingly face rates
that vary across hours of the day, weekdays versus weekends, and seasons of the year. Currently, all of
California’s investor owned utilities default their customers into time of use rates. On one extreme, some
retail rates in Texas essentially pass wholesale electricity prices directly onto consumers, which creates a
real-time electricity price that varies every hour. Rates will continue to change in this direction, and
consumers can expect large changes to the nature of electricity prices within the lifetime of appliances
sold today.
Energy conservation is not an end in itself but it stands as a proxy for the preservation of economic
resources, including externalities. It has been an effective proxy, but where the two diverge, it is
important to focus on the conservation of economic value rather than units of energy per se. Although the
statutory law is stated in terms of energy conservation, it is also explicit that standards must be
economically justified. For this reason, the committee believes that DOE should do whatever it can to
ensure that it is appropriately focused on saving money and economic value.
70

What does this mean in practice for DOE’s analysis?

First, for appliances with a highly seasonal component, such as air conditioners, furnaces, and
dishwashers (due to the need to raise the water temperature above that of the supply), it is critical that
DOE’s analysis includes appropriate adjustments of prices to reflect seasonal variation in the cost of
electricity. It is also important that DOE accounts for seasonal variation in energy consumption for all
appliances where such variation exists.
Second, DOE’s process should recognize and quantity consumer financial savings from features that
facilitate load shifting or other so-called “smart appliance” features. Potential features and savings are
discussed further in Chapter 6. The committee notes here that in addition to the reliability and
environmental benefits these technologies offer today, they can facilitate and spur innovation needed for
an energy transition and hence deserve credit now as economic benefits. DOE has in some cases treated
internet-enabling features as an energy efficiency equivalent; they are perhaps better conceptualized as
direct economic benefits. A smart dishwasher that moves a cycle from a high-price hour to a low-price
hour saves no energy but delivers consumer savings. It also may deliver economic benefits by replacing
electricity generation from more polluting sources with cleaner ones.
Finally, appliances that adjust energy consumption to the grid’s needs at different time-scales increase
grid reliability. While reducing pollutant emissions from energy use is a clear reason to pursue end-use
energy efficiency, reliability enhancement is equally important. As discussed in the next section, DOE’s
analysis pays great attention to the quantification of benefits from reduced pollutant emissions, but the
analysis on reliability benefits is lacking. The committee recommends heeding reliability benefits in equal
fashion.
RECOMMENDATION 4-10: DOE should credit as economically valuable those features and
innovations that save consumers money and enable appliances to contribute to grid efficiency
and reliability.
POLLUTANT EMISSIONS FROM ENERGY USE
The clearest rationale for policies that boost energy efficiency is the pollution externalities associated
with electricity generation, or the appliances’ direct use of fossil fuels. Where standards reduce energy
consumption, the social benefits from reduced externalities should be monetized and included as a
benefit. This is done in two steps. First, estimated reductions in energy usage are translated into changes
in emissions of various pollutants using a model of the energy sector. Second, these changes in pollution
are monetized using existing estimates of the damages associated with emissions.
Changes in Emissions
The DOE process takes an estimated change in energy consumption that follows from the LCC
analysis and translates this into estimates of emissions reductions that include both combustion emissions
for electricity generation (or direct use of the appliance) and upstream emissions due to extraction,
processing, transportation, and fugitive emissions of fuel.
The direct emissions (i.e., from combustion) of methane (CH4) and nitrous oxide (N2O) are estimated
from emissions intensity factors calculated by the U.S. Environmental Protection Agency (EPA).11 Direct
emissions of carbon dioxide (CO2), sulfur dioxide (SO2), nitrogen oxides (NOx), and mercury (Hg) are
estimated using projections presented by the Energy Information Administration in the most recent
Annual Energy Outlook (AEO). The AEO is generated by running the National Energy Model System
11
U.S. Environmental Protection Agency (EPA) Center for Corporate Climate Leadership, undated, “GHG
Emission Factors Hub.” https://www.epa.gov/climateleadership/ghg-emission-factors-hub.
71

(NEMS) model under different scenarios that vary in assumptions about energy supply, demand, and
technological change.
The DOE process backward-induces marginal emissions factors from AEO forecasts in a process
described in detail in a Lawrence Berkeley National Laboratory (LBNL) report (Coughlin, 2014) and
summarized in the TSDs. The method considers the results from AEO side-cases and compares them to
those of the reference cases to estimate changes in supply (fuel use, emissions, generation capacity) and in
demand. Then it allocates the demand changes to each of three load categories: peak, off-peak, and
shoulder, and estimates the fraction of electricity demand in each load category that is met by each fuel
type.
To estimate the fuel use reductions caused by lower on-peak, off-peak, or shoulder demand, DOE
assumes merit-order. It assumes that all oil-fired electricity is used to meet peak demand, that natural-gas-
fired electricity is used to meet the peak-demand not satisfied by oil, that nuclear and coal are used
proportionally to meet demand at the three load types, and that the remaining fuels are used to serve off-
peak and shoulder load. To estimate the emissions reductions caused by a reduction in fuel use at a given
time, DOE applies a statistical regression analysis model that “produces coefficients that define the
change in total annual emissions of a given pollutant resulting from a unit change in total annual
generation for each fuel type, as a function of time.” (DOE, 2014, p. 15-3) DOE then applies these
coefficients to the estimates of changes in annual generation from each fuel type to obtain estimates of
emissions reductions.
Because the NEMS model is a well-studied and transparent government model, and the assumptions
of the AEO reference and side cases are extensively documented and publicly scrutinized the committee
finds they are the appropriate tools for bounding the uncertainty about the impacts of standards in the
future.
Nevertheless, the committee finds DOE’s method has two weaknesses. First the method provides one-
point estimates of emissions reductions when in fact this quantity is impossible to estimate with that
precision. Second, the method uses a very coarse representation of the temporal and spatial dimension of
electricity generation and consumption and hence it is unlikely to assess the true impact that energy
efficiency has on emissions. By apportioning the energy savings into three load categories DOE attends to
the fact that the marginal generators vary with the amount of load served at a given time, but by ignoring
the seasonality of energy supply and demand DOE is likely to get the emissions of the marginal generator
wrong. For example, the natural gas electricity generating unit that is called to provide power during a
peak hour in the spring is likely to be less polluting than another natural gas unit required to provide
electricity during the peak-hour in the summer. Similarly, whether the reduced demand occurs in
Kentucky or in Texas, the reductions in emissions are likely to be different. This heterogeneity has been
documented in the literature (e.g., Graff Ziven et al., 2014; Siler-Evans et al., 2012) estimated using high
resolution data on emissions and demand.
In order to obtain a more precise estimate of the emissions changes that would occur in the first years
of implementation of the energy efficiency standard, the committee recommends that DOE estimates the
spatial and temporal specific changes in electricity demand, and then runs NEMS with these changed
inputs to compare results with the reference case of no-standards. The runs should be completed under
varying assumptions of fuel prices and composition of the power generation fleet as is standard in the
literature (e.g., Alqahtani and Patino-Echeverri, 2019). Instead of NEMS, DOE could build a simplified
electricity cost model representing the composition of the U.S. electric power system interconnects.
RECOMMENDATION 4-11: To estimate changes in emissions during the compliance year,

DOE should estimate for each state, the changes in the hourly load curve that would result from
the adoption of an energy efficiency standard during a full year. Using these estimates, the
National Energy Modeling System can incorporate different assumptions about
decarbonization of the U.S. electricity system such as the natural gas prices and the penetration
of renewable energy and energy storage and estimate a range of emissions changes for each
relevant region and time.
72

Following an approach that uses annual hourly load curves and data-based specific emissions factors
for individual regions and times will improve the estimates of emissions reductions due to a standard,
taken as a given the energy mix and performance of the current U.S. electric power generation fleet. This
approach will provide time and region-specific estimates useful to monetize benefits from emissions
reductions of pollutants for which health impacts are time and space dependent. However, to project the
impact of energy standards on future years, it is necessary to use projections of the U.S. electricity grid’s
future status and performance. One source of such predictions is NEMS. If possible, the committee
recommends that DOE accounts for the spatial and temporal heterogeneity of U.S. marginal emission
factors by running side-cases of NEMS that considers region-specific changes in annual hourly load curve
resulting from the adoption of different TSLs. The uncertainty that is inherent to NEMS as a model (i.e.,
is NEMS representation of power plant dispatch processes accurate?) and to the input values assumed for
its parameters (i.e., prices of coal and gas to electric generators, base demand levels, costs of growing
renewable generation capacity, etc.) implies that a single run of NEMS for a projected change in the load
curve will be insufficient to characterize the uncertainty on future emissions.
If DOE continues to use the data of the existing AEO side cases the committee recommends that DOE
estimate the uncertainty on the marginal emissions intensity factors it derives and, rather than reporting a
single point estimate, it presents a range. DOE can estimate ranges of emissions changes associated to a
specific region by considering the results from the two most extreme calculations obtained from AEO
side-cases. Although as discussed in the Annex, the committee considers it most desirable to present
results at the highest temporal and spatial resolution possible (e.g., seasonal emissions for each U.S. state)
if DOE continues to present one estimate for the nation, it must compound the uncertainty derived from
different assumptions with the discrepancies in emissions changes due to regional variability. One way to
find the bounds of possible results for a national average of emissions reductions—after accounting for
both uncertainty and variability—is to estimate emissions changes for two states in the opposite spectrum
of impacts from the standards.
As discussed above, a primary source of uncertainty on the benefits of energy efficiency to consumers
and society is the unknown composition of the U.S. electric power grid and the associated costs of
electricity and emissions. As the energy transition unfolds there will be a great deal of variation year to
year in the type of power plants that supply electricity and their marginal emissions (Holland et al., 2019).
While it is not reasonable to expect DOE to try to predict that future beyond using the forecasts produced
with NEMS and summarized in the AEO, the committee recommends that DOE conducts backward-
looking studies to estimate the error in its past estimates of marginal emissions rates, and to use its
findings in a robust analysis of the uncertainty in these parameters.
While the committee believes that estimating the benefits of reduced energy consumption from a new
standard must be calculated over the course of the lifetime of the product, it recommends that DOE
focuses its efforts on improving its estimates of emissions reductions with today’s fleet. DOE could
estimate emissions reductions from reduced demand today and then extrapolate that estimate to account
for the 15 or 30 years into the future as required by its analytic horizon. DOE can conduct retrospective
analyses to determine whether the estimates of marginal emission rates estimated from AEO forecasts are
better predictor than estimates derived from static forecast that takes the current grid as given or some
other simple heuristic.
Given that more than 85% of the Clean Air Act’s public health benefits are attributed to reductions of
particulate matter (PM) emissions (EPA, 2011), the committee recommends that DOE considers
estimating the energy efficiency standards impacts on PM emissions reductions and their economic value.
DOE cans use EPA’s National Emissions Inventory (NEI) is a source of PM2.5 emissions data.
As mentioned, DOE also estimates reductions in upstream emissions. The same recommendations
made about direct emissions are made by the committee regarding this analysis. The uncertainty on these
estimates is known to be very large, particularly when fugitive emissions are considered (Schwietzke et
al., 2014).
73

Another fundamental issue in calculating emissions is to treat fugitive emissions associated with
natural gas production and distribution. This is critical for natural gas appliances and electrical appliances
that may draw from natural gas power plants on the margin.
Monetizing Emissions Reductions
The DOE process monetizes changes in emissions quantities using outside studies and government
reports on the economic and health damages associated with emissions. The DOE process follows the
presidentially mandated procedures for calculating these numbers. However, it also appears not to
monetize some pollutants, nor does it represent or communicate all of the uncertainty and variability in
these numbers. As with changes in energy use, emissions reductions from new appliances may
significantly depend on other public policies and interventions as well as standards; in consequence
absent consideration of the dependencies it is difficult to predict the impact of a standard.
In particular, DOE adopted the federal estimate of the Social Cost of Carbon (SCC) that arose
following President Trump’s Executive Order 13783 (March 2017) which disbanded the Interagency
Working Group (IWG) that had been convened by the Obama Administration in 2009 to develop
estimates of the SCC, and withdrew the Technical Support Document and updates issued by the IWG in
2010, 2013, 2015 and 2016. In compliance with this Executive Order, the EPA put out a new estimate of
the SCC which first appeared in the Regulatory Impact Analysis for the Repeal of the Clean Power Plan
(EPA, 2019) and was then used by other federal agencies for the duration of the Trump administration,
including DOE. The new federal estimate is based on the same analysis of integrated climate assessment
models as the prior IWG, but changes in two key assumptions: (1) only domestic U.S. damages from
climate change were considered, not global damages; (2) the discount rates applied to value future
impacts were changed from 2.5%, 3% and 5%, with 3% being used for the central SCC estimate to 3%
and 7%. Omitting non-domestic damages, while using a 3% discount rate, reduced the SCC from a prior
value of $50 per ton of CO2 emissions in 2020 to $7 (GAO, 2020). Using a 7% discount rate reduced it to
$1 (EPA, 2019). On his first day in office, President Biden issued Executive Order 13990 reviving the
IWG and, the following month, the IWG revived the prior 2016 estimates of SCC, with a central value of
$51 per ton. It also introduced values for the social cost of methane and the social cost of nitrous oxide
(IWG, 2021).
The committee finds that DOE did not monetize all emissions, however, and no other monetized
values feature attention to uncertainty. The default appears to monetize only NOx and carbon, and DOE
calculates reductions in sulfur dioxide, nitrous oxide, and methane, but does not monetize them. The
committee believes regardless of the current atmospheric concentration of different pollutants it is
important to quantify and monetize the emissions of particulate matter and any precursors. The latter can
now be monetized using the values promulgated in IWG (2021).
In addition, the economic damage associated with a ton of emissions for local air pollutants varies
substantially across space (Muller and Mendelsohn, 2007; Muller et al., 2013) and time. Attention to
geographic detail and times of emissions may be relevant to DOE’s findings, which are likely to depend
on the appliance under consideration. For example, air conditioners have very different utilization rates
across regions of the country, and this may lead to substantial differences in the associated benefits from
emissions reductions.
Finally, the appropriate monetized value of emissions reductions may sometimes depend on the
existence of other regulations. For a capped pollutant, such as SO2 from the power sector, emissions
reductions can take on one of two values. If the cap is not binding, then the appropriate value is the direct
health benefit taken from the epidemiology literature. However, if the cap is binding, then the social
benefit of reducing emissions is not the health benefit because the total emissions do not actually change.
Instead, the social benefit is the marginal cost of complying with the binding cap, which is measured by
the permit price in markets with tradable permits (such as sulfur dioxide [SO2]). Thus, where a binding
74

cap exists, the appropriate value for emissions reductions is the permit price, which can be higher or
lower than the health impact in practice.
RECOMMENDATION 4-12: DOE should monetize all emissions impacts for which meaningful
estimates of social impacts are available, considering policy interactions when those interactions
are deemed significant.
MARKET FAILURES THAT RATIONALIZE STANDARDS
Economic theory concludes that an efficient allocation of resources is provided by well-functioning

markets—that, in a broad sense “welfare” can be improved by government intervention only when
markets are problematic. Such problems are called market failures and provide the opportunity for
regulatory action to yield benefits in excess of cost. Federal regulations require an analysis of market
failures for regulatory actions, and DOE does this as part of its analysis of new standards. Given the
uncertainties inherent in benefit and cost estimates of proposed standards, both a robust analysis of market
failures and an analysis of how standards address the failures are critical because the case for market
failure may not be universally strong across all appliance products.
Several potential market failures might justify minimum efficiency standards. The committee has
sorted these possibilities into three broad categories: environmental externalities, inefficiencies due to
consumer behavior, and inefficiencies due to producer behavior.
Environmental Externalities
The extraction, processing, transportation, and combustion of fossil fuels generates pollution. In the
absence of comprehensive pollution pricing or other regulations, the price of energy does not reflect its
full social cost because the burden of the pollution is borne by third parties outside the market. When
energy prices are artificially low, market actors will underinvest in energy efficiency from the point of
view of social welfare, even if consumer and producer behavior is consistent with models of full
rationality and perfect competition. This provides economic justification for policy intervention.
Standards are one way of addressing this market failure. Pollution externalities are clear, compelling,
and uncontroversial as a case for market failure and a rationale for policy. DOE routinely calculates
estimates of environmental gains in their analysis. Environmental benefits are often substantial and should
be uncontroversial as a rationale. That said, quantifying the economic costs of pollution is challenging
and is subject to substantial uncertainty. This report comments further on this quantification in the
section, Pollutant Emissions from energy use. Standards are often not the most efficient way to address
pollution externalities. This report comments further on the importance of considering policy alternatives
in Chapter 5.
Inefficiencies in Consumer Behavior
A second possible market failure is that buyers in a market are privately making inefficient choices.
That is, they could save themselves money by investing more in energy efficiency. If true, then a
substantial benefit of standards could come from correcting inefficiencies in consumer choice.
The energy efficiency gap, or energy efficiency paradox, refers to the phenomenon whereby a
consumer makes what is considered an inefficient choice by choosing an item that does not appear to
offer the lowest combination of capital cost plus operating cost (Jaffe and Stavins, 1994; Stern et al.,
2016). These terms refer to analysis that suggests that the private energy savings from improved
efficiency more than offsets additional product costs, yet the market reveals low take-up of the product.
75

This phenomenon is ubiquitous, and associated with significant increased energy use in the consumer
sector.12 The question is: why does this occur?
One reason a gap might exist is because of split incentives, or the landlord-tenant problem.
Sometimes the person who decides what appliance or piece of equipment to purchase is not the person
who pays the future energy bills. This can lead to fully rational self-interested behavior that is
nevertheless socially inefficient. Descriptive evidence of appliance ownership from surveys is consistent
with renters using less efficient appliances, supporting this view (Davis, 2010), and recent studies some
provide empirical evidence that split incentives are associated with inefficiencies in energy use (Myers,
2020). Demand for rental properties has surged in the United States over the past decade, and now
accounts for more than 36% of households, suggesting that the issue of split incentives may become
increasingly important in the appliance sector (Harvard University, 2017). Standards can be an appealing
policy solution to these sorts of inefficiencies, but the relevance of split incentives will vary across types
of appliances and types of customers. There is for example a history of government-funded programs to
incentivize the use of energy efficient appliances in properties that are not owner-occupied. In the 1990s
the New York Power Authority implemented a $38 million program of demand side management in the
New York City Housing Authority that targeted efficient residential refrigerators among other measures
(Nolden and Morgan, 1996).
A second reason for the energy efficiency gap is that consumers undervalue future cost savings
relative to upfront cost. The committee alluded above to factors that might cause appliance buyers not to
make these “rational” choices. First, the operating cost (and, therefore, the energy savings) may be
unclear to some consumers because they are not aware of the energy or water their appliance consumes in
real-time when they run it. They also may have only a vague awareness of how much they are paying for
electricity, natural gas, or water in their daily use. Secondly, attributes other than price usually matter to
consumers and may be more important to them. For example, despite the introduction of new lighting
products (compact fluorescent lamps) that were more energy-efficient than the incumbent (incandescent)
technology, consumers still preferred the latter owing to its better consumer attributes such as quality of
light (NRC, 2013, p. 28). After binding regulations effectively phased-out the incumbent technology,
many consumers still preferred the least efficient among the compliant alternatives—itself a variant of the
incumbent technology (NRC, 2013, p. 31). Finally, choice is complex in real-world markets with a
numerosity of alternative products. Consumers may not make the best choices when facing complexity,
particularly if sellers strategically market products to benefit from consumer fallibility.
When economists think of rational decision making, they tend to think about a decision-maker
performing a global optimization; the decision-maker systematically considers all alternatives and all
possible attributes associated with each alternative. In order to give due consideration to their budget
constraint, the decision-maker considers every other possible use of the money that they would spend on
the item in question. They make a thorough assessment of these things and choose the best course of
action.13
In reality, choices—by firms and households—often fall short of this idealized conceptualization.
This was first identified in economics by Herbert Simon (1955), who argued that the conventional
characterization of decision making is unrealistic and lacks psychological veracity. Performing the
implied analysis would be computationally demanding, if not intractable; it would require information
that might not always be available; and it would demand a massive amount of time and attention for
information processing and calculation. Instead, Simon argued, people display “bounded rationality.”
They seek to simplify their decision making in various ways. One way is to limit their attention to a
subset of the available alternatives—a consideration set—based on familiarity, habit, or other factors.
Another way is to simplify their assessment of attributes—focusing mainly on some while dropping
12
See Hirst and Brown (1990) and Stern et al. (2016). These papers review studies that conclude foregone
energy savings in the consumer sector of perhaps up to 50% due to a broad set of policies, including structural
barriers and pricing distortions as well as the behavioral attributes that underlie the energy efficiency gap.
13
This passage adapted from NASEM (2020).
76

others and looking to attain a good enough level of the attributes rather than an optimal level (Tversky,
1972). They employ heuristics or rules of thumb to evaluate the options and their attributes. More
generally, Simon suggested that consumers “satisfice” rather than optimize—seek a reasonably good
choice if not the absolute best.14
To deny that consumers may make mistakes would be unreasonable—they are, after all, humans. But
to hold consumers to an idealized standard of global, all-considering optimization would also be
unreasonable. There exist cases where the apparent energy savings from improved efficiency more than
offsets the additional upfront cost, yet consumers do not choose those appliances—there is an energy
efficiency gap on the consumer side. Note that mistakes are not necessarily only in one direction. Some
consumers may overinvest in energy efficiency that does not pay back (i.e., because they have low
utilization rates), which complicates the consumer benefits that derive from tighter standards.
Substantial literature has sought to examine whether consumers make systematic mistakes due to
some combination of these factors. Empirical evidence remains mixed (see, e.g., Allcott and Greenstone
[2012] and Gerarden et al. [2017]). Moreover, the case for underinvestment in efficiency varies
substantially across types of products or equipment. For example, many products lack official energy
efficiency labels, making it much more plausible that consumers make poor choices. For some products,
energy consumption is a small part of the total cost of ownership. This makes it more plausible that
consumers might ignore energy efficiency when making choices, perhaps even as a rational approach to
minimize hassle and effort (Sallee, 2014). Likewise, the buyers’ sophistication is likely to vary across
products, particularly when buyers are themselves businesses. It is hard to imagine, for example, that
supermarket chains are inattentive to the operating costs of commercial refrigeration.
In sum, there is compelling evidence from many domains that consumers do not always choose the
options that seem to offer the lowest overall cost. However, the evidence around the energy efficiency gap
is more mixed (Allcott and Greenstone, 2012), and there are strong reasons to suspect that mistakes vary
substantially across products. Small firms may not have energy managers (Janda et al., 2014), while even
owners or managers of large commercial buildings where the former may be tenants often have no
knowledge of the size of the energy bills of their properties (NRC, 2010, p. 101). Either might avail
themselves of demand-side management (DSM) provided or encouraged by utilities, discussed in Chapter
5, or continuous commissioning discussed in Chapter 3, among other services. Considering all this, the
committee’s recommendation is for the DOE process to engage more robustly with these issues and
attempt a more significant context-specific quantification of any market mistakes.
RECOMMENDATION 4-13: DOE should place greater emphasis on providing an argument

for the plausibility and magnitude of any market failure related to the energy efficiency gap in
their analyses. For some commercial goods in particular, there should be a presumption that
the market actors behave rationally unless DOE can provide evidence or argument to the
contrary.
This energy efficiency gap is central to DOE’s analysis. DOE’s regulatory analyses that support
tighter standards consistently premise upon an energy efficiency gap. That is, standards that have large
benefit-cost ratios do so because DOE’s engineering analysis demonstrates that readily available
technologies save far more in energy costs than they require in upfront cost, but the regulatory impact
analysis does relatively little to interpret this fact. The question is where the burden of proof should lie.
The committee’s recommendation calls both for the gathering of more evidence, and for allowing the
burden of proof to vary across types of products and equipment. One source of such relevant evidence
could be additional customer interviews, for both household and business customers and for rental and
owner-occupied housing, that seek to document potential biases or information gaps in choices.
14
Note that these phenomena are less likely to occur with commercial equipment, since many commercial
buyers are themselves large companies with sophisticated energy planning and management systems. However, not
all are.
77

Not all interpretations of the energy efficiency gap may imply a market failure. Other explanations for
the energy efficiency gap include biases in the cost estimates, a failure to properly account for
performance trade-offs, a failure to account for irreversibility and option value, or a failure to account for
borrowing constraints, and mismeasurement of actual energy efficiency or actual transaction prices.
Energy efficiency gap estimates are premised on engineering estimates of cost and savings that may
simply be incorrect. DOE’s analysis may sometimes overstate energy savings in situ. Some recent
evidence calls into question the realized performance of home energy retrofits (Christensen et al., 2020;
Fowlie et al., 2018), appliance replacement programs, and automobile fuel economy ratings (Reynaert
and Sallee, 2021). While this evidence does not directly comment on the in-situ performance of
appliances, it suggests the value of research or data that could assess whether test procedures accurately
reflect performance in the field.
Analysis will often fail to measure actual transaction prices correctly, as prices fluctuate with
promotions and retailer discounts, and these data are unavailable in many settings. Moreover, an apparent
energy efficiency gap may be due to some users having low utilization rates, which may not be fully
characterized by the data used by analysts.
In addition, some households may face severe constraints accessing credit, which could lead to
apparently inefficient trade-offs between up-front cost and future operating costs. The irreversibility of an
appliance investment and uncertainty over future prices of electricity implies that consumers will
rationally only buy an energy efficient product if its operational cost-savings are significantly above its
additional upfront cost (Hassett and Metcalf, 1995), though these considerations do not necessarily imply
slower diffusion of the high efficiency products (Baker, 2012). Given these alternative possibilities, it is
important for DOE to establish evidence in favor of interpreting an energy efficiency gap as evidence of
consumer mistakes that can be corrected by standards.
Inefficiencies in Producer Behavior
There also can be instances of market failure on the manufacturers’ side. This can come from
monopoly power that distorts markets, or from failures to innovate efficiently. Under EPCA, DOE must
consider impacts that a standard might have on the degree of competition in a market. DOE makes a
qualitative assessment of any such impact in its manufacturer impact analysis, and it outsources further
analysis to the U.S. Department of Justice. Both analyses appear to focus on whether a standard might
shrink the number of firms competing in a particular market.
This is an important issue, but this analysis attends to only some of the potential distortions due to
market power (monopoly power). This narrow view of market power simply asks whether manufacturers
charge prices that are artificially high overall. A related but distinct issue pertains to what economists call
second-degree price discrimination. The basic idea is that sellers attempt to segment buyers who differ in
their willingness to pay by offering a higher and lower quality version of a product. Economic theory
shows that when firms use energy efficiency to price discriminate, market outcomes are inefficient and
standards can improve market outcomes (Fischer, 2010). Further research has placed greater emphasis on
the possibility that price discrimination is an important factor in appliance markets (Spurlock, 2013). In
particular, price discrimination is a possible explanation for empirical evidence showing that tightening of
standards has led to price decreases (Houde and Spurlock, 2016). These findings suggest, but are not
conclusive toward, the idea that standards may enhance efficiency because they counteract market
inefficiencies due to price discrimination.
A separate issue regarding possible inefficiencies in producer behavior relates to innovation.
Competitive forces generally spur sellers to accelerate diffusion to gain a market advantage. But advances
in the development and diffusion of technology (including the marketing of a new technology to buyers)
often create what economists call spillovers. This is meant to indicate situations where the technological
advance of one producer benefits its competitors by demonstrating how something can be done. When
spillovers exist, the free market does not lead to efficient levels of innovation. Barriers to development
78

and diffusion of new technology also exist in oligopolistic industries, as is the case for much of the
appliance sector, where firms may be in a position to eschew investing in new technology that would cut
into the profits of existing products or where access to retailers by new firms is limited. In each of these
cases, standards might then enhance efficiency.15 Note that such spillovers and lack of investment need
not be restricted to cutting edge technologies. It can relate to the more prosaic task of educating and
acclimating buyers to a new feature.
Market outcomes may also be distorted if producers are able to collude on technological rollout. To
the extent that technological development and diffusion requires fixed costs that firms do not immediately
recover in market equilibrium, firms would find it more profitable to roll out technological change at a
slower pace. Firms would thus like to agree on a rate of innovation but face anti-trust violations if they
explicitly coordinate on the roll out of product features and embedded technologies. As a recent example,
a group of automakers has been accused of anti-competitive behavior surrounding a technical working
group that coordinated the diffusion of features in the luxury automobile market (Beene and Mehrotra,
2017). In such circumstances, standards could protect against such distortions.
The committee notes, however, that standards might work in the opposite direction by facilitating
collusion rather than mitigating it. This may be a particular problem for standards developed by industry
coalitions or through negotiated rulemaking. In principle, the standards-setting process itself could
constitute a vehicle through which such industry coordination is made possible and legal. Economic
theory has described how quality standards can facilitate collusion, though results can go either way and
are sensitive to model assumptions. More generally, standards could potentially act as barriers to entry
against future competitors, thus serving the interest of incumbent producers.
In many product markets, a large fraction of all products achieves exactly the DOE minimum or
exactly the Energy Star rating, which is most commonly a specific efficiency percentage beyond the DOE
minimum. Moreover, the appliance industry initiates a substantial fraction of the standards that DOE
develops through the Direct Final Rule process or Negotiated Rulemakings or both. These factors are not
direct evidence of collusion, but they suggest the value of further scrutiny. It is the committee’s view that
DOE ought to develop higher sensitivity to the possibility that this process facilitates collusive behavior,
particularly via the coordination of a slower than competitive adoption of new technologies. A formal
quantitative analysis may not be feasible around the possibility of collusion, but DOE can at least make it
routine to report out what information they have that makes them disregard this concern for each
standard.
RECOMMENDATION 4-14: The committee recommends that DOE give greater attention to a
broader set of potential market failures on the supply side, including not just how standards
might reduce the number of competing firms, but also how they might impact price
discrimination, technological diffusion, and collusion.
The existence of market failures justify public interventions in appliance markets, including setting
minimum energy efficiency standards for appliances: the social costs of energy use are generally not
internalized by individual firms or consumers, while other behavioral and structural issues may result in
some consumers and firms purchasing equipment that, from either a social of individual perspective,
makes an uneconomic tradeoff between the purchase price and subsequent operating costs, thereby
inefficiently increasing energy consumption. While DOE considers market failures, it does not include an
analysis that attempts to identify and estimate the importance of specific market failures associated with
appliances. (See Recommendations 4-13 and 4-14.) The review in this chapter underscores the
15
Promoting efficient innovation in energy efficiency is a complex problem that is largely outside the scope of
this DOE program, which is intended to focus on existing technology, although as is discussed in Chapters 2 and 6,
the program can nevertheless contribute in important ways to more ambitious energy goals. More generally,
standards can play an important role in energy innovation as part of a suite of public and private actions. See Jaffe et
al. (2005), Henderson and Newell (2011), and Mazzucato (2011).
79

importance of tailoring consideration of market failures to each category of appliances that are under
consideration for a standard. The committee finds the existence and importance of energy-related market
failures differs across appliance categories, together with the likelihood that a standard will yield
improvements in economic efficiency.
FINAL OBSERVATIONS
Legislation and Executive Orders require the DOE energy efficiency standards to be “economically
justified.” To do so, DOE conducts detailed analyses that attempt to calculate benefits and costs of
alternative efficiency levels. However, the results are necessarily uncertain, in part because of data
limitations but also because of inherent complexities in the response manufacturers, consumers and
markets to a standard that changes the available types of appliances. As a result, quantifying, to the extent
practical, the uncertainty of benefit and cost estimates, enhancing data collection, and focusing on market
failures are critical to the economic justification of a standard. The value of the retrospective study is not
primarily in estimating ultimate energy savings, which of course depend on a host of factors outside the
standard itself,16 but rather in assessing the validity of assumptions made in the initial analysis and
informing subsequent analyses conducted for other potential appliance standards.
This chapter’s focus on uncertainty and empirical data apply across the range of analyses that make
up the economic analyses of the Appliance and Equipment Standards Program, as is summarized below.
RECOMMENDATION 4-15: In order to evaluate the economic costs and benefits of a

standard, DOE should present the distribution of costs and benefits estimated in its models
when (1) uncertain parameters are represented by probability distributions and (2) parameters
that vary across geographic and other relevant dimensions are disaggregated. The uncertainty
or variability the parameters represent should be compounded or propagated—properly
accounting for any correlations—throughout the calculation. This methodology is necessary for
the markup analysis and manufacturer impact analysis (Recommendation 4-2), the shipments
analysis (Recommendation 4-4), and all components of the life-cycle cost analysis.
(Recommendations 4-5 and 4-7). Where multiple sources of uncertainty must be combined for
the final benefits result, as with net benefits depending on both the shipments analysis and the
appliance unit cost and performance, the subcomponents should be reported as well.
(Recommendation 4-5).
RECOMMENDATION 4-16: DOE should obtain better data for improving the economic
analyses of appliance and equipment performance standards. Empirical data are needed on
markups (Recommendation 4-1), consumer choices in appliance markets (Recommendation 4-
3), and in situ performance (Recommendation 4-8). Some of this information can come from
relatively simple changes to current surveys and studies, including engineering analyses of the
Appliance and Equipment Standards Program and the Residential Energy Consumption
Survey (Recommendation 4-7).
RECOMMENDATION 4-17: Ex post analyses can validate assumptions made in prior

standards and evaluate the implications of prior forecasts’ inaccuracies and mistakes. DOE
should use such ex post analyses routinely to improve forward-looking standards iteratively.
16
See, for example, Newell et al. (1999).
80

REFERENCES
Allcott, H., and M. Greenstone. 2012. “Is There an Energy Efficiency Gap?” Journal of Economic
Perspectives 26(1): 3-28.
Alqahtani, B., and D. Patiño-Echeverri. 2019. “Combined Effects of Policies to Increase Energy
Efficiency and Distributed Solar Generation: A Case Study of the Carolinas.” Energy Policy 134:
110936. https://doi.org/10.1016/j.enpol.2019.110936.
Attari, S.Z., M.L. DeKay, C.I. Davidson, and W.B. de Bruin. 2010. “Public Perceptions of Energy
Consumption and Savings.” Proceedings of the National Academy of Sciences U.S.A. 107(37):
16054-16059.
Beene, R., and K. Mehrotra. 2017. Top German Automakers Sued in U.S. Over Two-Decade “Cartel.”
Bloomberg. July 27. https://www.bloomberg.com/news/articles/2017-07-28/porsche-audi-bmw-
and-others-acted-as-cartel-lawsuit-says.
Berry, S., J. Levinsohn, and A. Pakes. 1995. “Automobile Prices in Market Equilibrium.” Econometrica
63(4): 841-890.
Blonz, J.A. 2019. The Welfare Costs of Misaligned Incentives: Energy Inefficiency and the Principal-
Agent Problem. Finance and Economics Discussion Series 2019-071. Washington, DC: Board of
Governors of the Federal Reserve System.
Brucal, A., and M.J. Roberts. 2019. Do Energy Efficiency Standards Hurt Consumers? Evidence from
Household Appliance Sales. Journal of Environmental Economics and Management 96: 88-107.
Christensen, P.F., E. Myers, and M. Souza. 2020. Decomposing the Wedge Between Projected and
Realized Returns in Energy Efficiency Programs. E2e Working Paper 046. Berkeley, CA:
University of California, Berkeley.
Coughlin, K. 2014. Utility Sector Impacts of Reduced Electricity Demand. LBNL-6864E. Berkeley, CA:
Lawrence Berkeley National Laboratory. https://eta.lbl.gov/publications/utility-sector-impacts-
reduced.
Dale, L., D. Millstein, K. Coughlin, R. Van Buskirk, G. Rosenquist, A. Lekov, and S. Bhuyan. 2004. An
Analysis of Price Determination and Markups in the Air-Conditioning and Heating Equipment
Industry. LBNL-52791. Berkeley, CA: Lawrence Berkeley National Laboratory.
Davis, L.W. 2010. Evaluating the Slow Adoption of Energy Efficient Investments: Are Renters Less Likely
to Have Energy Efficient Appliances? Energy Institute at Haas Working Paper 205, June.
Berkeley, CA: Energy Institute at Haas.
DOE (U.S. Department of Energy). 2014. Technical Support Document: Energy Efficiency Program for
0005.
EPA (U.S. Environmental Protection Agency). 2011. The Benefits and Costs of the Clean Air Act from
1990 to 2020: Final Report—Rev. A. April. https://www.epa.gov/sites/production/files/2015-
07/documents/fullreport_rev_a.pdf.
EPRI (Electric Power Research Institute). 1994. CLASSIFY Profiles: Volume 1: Residential Customer
Needs and Energy Decision Making. Report TR-104567-V1s. Palo Alto, CA. December.
FERC (Federal Energy Regulatory Commission). 2020. Form No. 714—Annual Electric Balancing
Authority Area and Planning Area Report. https://www.ferc.gov/industries-data/electric/general-
information/electric-industry-forms/form-no-714-annual-electric/data.
Fischer, C. 2010. Imperfect Competition, Consumer Behavior, and the Provision of Fuel Efficiency in
Light-Duty Vehicles. RFF DP 10-60. December.
81

Fowlie, M., M. Greenstone, and C.D. Wolfram. 2018. Do Energy Efficiency Investments Deliver?
Evidence from the Weatherization Assistance Program. Becker Friedman Institute for Research in
Economics. Working Paper No. 2621817. January 22.
GAO (U.S. Government Accountability Office). 2020. Social Cost of Carbon: Identifying a Federal
Entity to Address the National Academies’ Recommendations Could Strengthen Regulatory
Analysis. GAO-20-254. Washington, DC. June.
Gerarden, T.D., R.G. Newell, and R.N. Stavins. 2017. “Assessing the Energy-Efficiency Gap.” Journal of
Economic Literature 55(4): 1486-1525.
Graff Zivin, J.S., et al. 2014. “Spatial and Temporal Heterogeneity of Marginal Emissions: Implications
for Electric Cars and Other Electricity-Shifting Policies.” Journal of Economic Behavior and
Organization 107: 248-268.
Harvard University. 2020. American’s Rental Housing 2020. Joint Center for Housing Studies.
https://www.jchs.harvard.edu/sites/default/files/reports/files/Harvard_JCHS_Americas_Rental_H
ousing_2020.pdf.
Hassett, K.A., and G.E. Metcalf. 1995. “Energy Tax Credits and Residential Conservation Investment:
Evidence from Panel Data.” Journal of Public Economics 57(2): 201-217.
Henderson, R.M., and R.G. Newell, eds. 2011. Accelerating Energy Innovation: Insights from Multiple
Sectors. Cambridge, MA: National Bureau of Economic Research.
Hirst, E., and M. Brown. 1990. “Closing the Efficiency Gap: Barriers to the Efficient Use of Energy.”
Resources, Conservation and Recycling 4(3): 267-281.
Holland S.P., E.T. Mansur, N.Z. Muller, and A.J. Yates. 2020. “Decompositions and Policy
Consequences of an Extraordinary Decline in Air Pollution from Electricity Generation.”
American Economic Journal: Economic Policy 12(4): 244-274.
Houde, S., and C.A. Spurlock. 2016. “Minimum Energy Efficiency Standards for Appliances: Old and
New Economic Rationales.” Economics of Energy & Environmental Policy 5(2): 65-84.
Jaffe, A.B., and R.N. Stavins. 1994. “The Energy Paradox and the Diffusion of Conservation
Technology.” Resource and Energy Economics 16: 91-122.
Jaffe, A.B., R.G. Newell, and R.N. Stavins. 2005. “A Tale of Two Market Failures: Technology and
Environmental Policy.” Ecological Economics 54(2): 164-174.
Janda, K.B., C. Bottrill, and R. Layberry. 2014. “Learning from the ‘Data Poor’: Energy Management in
Understudied Organizations.” Journal of Property Investment & Finance 32(4): 424-442.
Knittel, C.R., and K. Metaxoglou. 2014. “Estimation of Random Coefficient Demand Models:
Challenges, Difficulties and Warnings.” Review of Economics and Statistics 96(1): 34-59.
Mazzucato, M. 2011. The Entrepreneurial State. London, UK: Demos.
Muller, N.Z., and R. Mendelsohn. 2007. “Measuring the Damages of Air Pollution in the United States.”
Journal of Environmental Economics and Management 54(1): 1-14.
Muller, N.Z., R. Mendelsohn, and W. Nordhaus. 2011. “Environmental Accounting for Pollution in the
United States Economy.” American Economic Review 101(5): 1649-1675.
Myers, E. 2020. “Asymmetric Information in Residential Rental Markets: Implications for the Energy
Efficiency Gap.” Journal of Public Economics 190: 104251.
NASEM (National Academies of Sciences, Engineering, and Medicine). 2020. Reducing Fuel
Consumption and Greenhouse Gas Emissions of Medium and Heavy-Duty Vehicles: Phase Two.
Newell, R.G., A.B. Jaffe, and R.N. Stavins. 1999. “The Induced Innovation Hypothesis and Energy-
Saving Technological Change.” Quarterly Journal of Economics 114(3): 941-975.
Nolden, S.L., and S.J. Morgan. 1996. “Super-Efficient Refrigerators for Apartments: The NYPA/NYCHA
Project as a New Market Transformation Model.” Pp. 2.181-2.185 in 1996 Summer Study on
Energy Efficiency in Buildings: Volume 2—Residential Buildings: Program Design,
Implementation, and Marketplace Issues. Washington, DC: American Council for an Energy-
Efficient Economy.
82

NRC (National Research Council). 2010. Real Prospects for Energy Efficiency in the United States.
NRC. 2012. Effective Tracking of Building Energy Use: Improving the Commercial Buildings and
Residential Energy Consumption Surveys. Washington, DC: The National Academies Press.
NRC. 2013. Assessment of Advanced Solid-State Lighting. Washington, DC: The National Academies
Press.
Reynaert, M., and J.M. Sallee. 2021. “Who Benefits When Firms Game Corrective Policies?” American
Economic Journal: Economic Policy 13(1): 372-412.
Sallee, J. 2014. “Rational Inattention and Energy Efficiency.” Journal of Law and Economics 57(3): 781-
820.
Schwietzke, S., W.M. Griffin, H.S. Matthews, and L.M.P. Bruhwiler. 2014. “Natural Gas Fugitive
Emissions Rates Constrained by Global Atmospheric Methane and Ethane.” Environmental
Science & Technology 48(14): 7714-7722.
Siler-Evans, K., I.L. Azevedo, and M.G. Morgan. 2012. “Marginal Emissions Factors for the U.S.
Electricity System.” Environmental Science & Technology 46(9): 4742-4748.
Simon, H. 1955. “A Behavioral Model of Rational Choice.” Quarterly Journal of Economics 69: 99-118.
Shocker, A.D., M. Ben-Akiva, B. Boccara, and P. Nedungadi. 1991. “Consideration Set Influences on
Consumer Decision-Making and Choice: Issues, Models, and Suggestions.” Marketing Letters
2(3): 181-197.
Spurlock, C.A. 2013. Appliance Efficiency Standards and Price Discrimination. LBNL-6283E. Berkeley,
CA: Lawrence Berkeley National Laboratory. https://eta.lbl.gov/publications/appliance-
efficiency-standards-price.
Stern, P.C., Janda, K.B., Brown, M.A., Steg, L., Vine, E.L., and L. Lutzenhiser. 2016. “Opportunities and
Insights for Reducing Fossil Fuel Consumption by Households and Organizations.” Nature
Energy 1(5): 16043.
Swait, J., and M. Ben-Akiva. 1987. “Incorporating Random Constraints in Discrete Models of Choice Set
Generation.” Transportation Research Part B: Methodological 21(2): 91-102.
U.S. Census Bureau. 2013. Annual Retail Trade Survey: 2011. March.
https://www.census.gov/data/tables/2011/econ/arts/annual-report.html.
Van Buskirk, R.D., C.L.S. Kantner, B.F. Gerke, and S. Chu. 2014. “A Retrospective Investigation of
Energy Efficiency Standards: Policies May Have Accelerated Long Term Declines in Appliance
Costs.” Environmental Research Letters 9(11): 114010.
83

ANNEX 4A: DETAILED COMMENTS ON THE LIFE-CYCLE COST ANALYSIS FOR

RESIDENTIAL DISHWASHERS
This annex provides greater detail of the committee’s analysis of the LCC analysis associated with the
residential dishwasher’s rulemaking. The information in this annex, contained in two tables, comprises a
review of the LCC as described in the Technical Support Document (DOE, 2014). Table 4A.1
summarizes DOE’s approach to estimating each input to the installation cost (IC) estimation for
dishwashers and the committee’s recommendation to improve this estimation process by accounting for
uncertainty and variability. Table 4A.2 summarizes DOE’s approach for each of the input variables for
the operating costs (OC) of dishwashers and also includes recommendations.
REFERENCE
DOE (U.S. Department of Energy). 2014. Technical Support Document: Energy Efficiency Program for
0005.
84

TABLE 4A.1 Review of DOE Approach to Estimate the Value of Installed Costs (ICs) for Residential Dishwashers
DOE Approach
Units and
Variable Description/Relevance Estimate Method Data Sources Committee Comments
Baseline This is the cost to manufacture 2013$ Association of Home This is a key input into DOE’s analysis. The committee finds
Manufacturer Cost the baseline dishwasher as Appliance Manufacturers that a single-value estimate, with the precision of cents of a
in the analysis year estimated during the year DOE Standard: dollar, conveys a false sense of certainty around a value that
conducts the analysis 203.72 is, in fact, highly uncertain. DOE could provide a range
Compact: instead of point estimates by propagating the uncertainty on
187.68 the costs of different materials and labor. Nevertheless,
because DOE’s interest is in comparing the baseline with the
alternative ELs to find LCC Savings, this value ends up
being irrelevant (what really matters is the incremental cost
of the ELs).
Factor to account Multiply the Manufacturers Unitless DOE derives cost BLS Producer Price Index • The cost declines of baseline costs are irrelevant for the
for Manufacturer cost of year 2013 by this factor declining annual rates (PPI) data. GDP Chained LCC Savings calculation (because they appear on both
cost declines due to to obtain the forecast of 0.905 (due to experience or Price Index. Data on annual sides of the equation and cancel out). For other
learning, between manufacturer cost for 2019 in learning) from historical shipments of dishwashers calculations, where they are useful, DOE should account
the analysis year 2015$ PPI data for other for 1972-2012 for the uncertainty around this value.
(2013) and the miscellaneous • DOE’s analysis of cost assumes decreases due to
compliance year households’ appliances experience (i.e., reductions in costs related to shipments)
97
(2019) for years 1988-2013 and estimates a 95% confidence interval for the experience
shipments data by fitting rate (defined as the percent reduction in price when
a power-law function cumulative production doubles) as 25.7±1.5%. To
account for uncertainty, DOE can sample from this range
in the Monte Carlo simulation. While it is not clear from
Chapter 8 or Appendix 8C if DOE (2014) already treats
this value probabilistically, it seems possible given that
the Excel file presents a value slightly different from
0.905 (i.e., 0.926).
Baseline This is the cost to manufacture 2015$ DOE multiplies the 0.905 x Baseline See recommendations in prior row.
Manufacturer Cost the baseline product during the Baseline Manufacturer Manufacturer cost in the
in the compliance compliance year (i.e., 2019 for Standard: Costs in the Analysis year compliance year
year dishwashers) 148.31 by the Factor
Compact: representing cost
168.33 reductions due to learning
(prior row in this table)
Manufacturer cost Add to baseline cost to find the 2013$ From engineering Given that different products already in the market or that
increases from ELs total manufacturer cost of each analysis and cost models will be developed can achieve the same efficiency level
EL Standard; that involve physically through the adoption of different designs and components, it
EL1:9.52 disassembling is clear that the manufacturer cost increases will differ
EL2:36.53 commercially available among products within the same EL category. The
EL3:74.72 products, reviewing committee recommends adopting one of two possible
EL4:74.72 publicly available cost approaches:

DOE Approach
Units and
and performance • Calling this parameter “Average Manufacturer Cost
Compact; information, and increase,” and, rather than providing a point estimate,
EL1:8.01 modeling equipment cost providing a range or
EL2:21.50 • Changing the name of this parameter to “Maximum
manufacturer cost increase” and adopting the maximum
value from the range of possibilities uncovered in the
engineering analysis.
Manufacturer Multiply this factor by the Unitless Weighted average of Securities and Exchange • Supplement the SEC’s information with expert’s
Markup for Manufacturer Production Cost manufacturers’ markup Commission SEC 10-K interviews to understand if the estimates of the
Baseline product (MPC) to obtain the final 1.24 reported in the data reports from 2004-2009 for manufacturer’s average markup for all products is
manufacturer price source manufacturers of major representative of the markup for dishwashers.
household appliances whose • Estimate the weighted average of the manufacturer’s
product offerings include markup for all of the years for which there are SEC 10-K
dishwashers reports and calculate the minimum, the maximum and
the average for those years. Adjust the relevant statistics
based on expert’s judgments. Then assume markup is a
random variable that follows a triangular probability
distribution function and use this characterization to
estimate the Installed Costs for each household in the
RECS sample.
• For transparency, provide all of the data necessary to
98
allow readers of the TSD to replicate DOE’s analysis
and obtain the same estimates. The committee could not
calculate a range for this variable because DOE did not
provide a list of the manufacturers that they included in
the original analysis.
• Consider reporting to 3 or 4 decimal places for this
factor, given that sales prices are reported with the
precision of dollars and cents.
Retailer Markup for Multiply this factor by the Cost Unitless Use the reported Sales 2011 Annual Retail Trade • Estimate the markup for Electronics and Appliance
Baseline of Goods Sold (CGS) to obtain and Gross Margin for Survey (ARTS)a stores for all of the years reported in the most recent
the final sale price of the 1.36 Electronics and ARTS surveys (i.e., 1993-2011).
baseline product to consumers Appliance stores in year • Characterize the Markup for baseline as a random
2007 in the Formula: variable following a triangular distribution with
Baseline Markup = parameters estimated from the ARTS survey. (The
(CGS+GM)/GM or committee finds this variable is within the range
=Sales/GM (1.321,1.405) and the parameters of the triangular
distribution will be: Minimum=1.321, Maximum=1.405,
Average=1.363.b.
• Also, use historical data to identify any past trends in
retail markups that can shed light on the projection of
future values. Because DOE is estimating future markup
it is important to assess whether historical data supports
the hypothesis of increasing or declining trends.

DOE Approach
Units and
• Supplement the SEC’s information and adjust the
triangular parameters with info from experts’ interviews
about whether the estimates of the manufacturer’s
average markup for all products is representative of the
markup for dishwashers.
• Consider reporting 3 decimals for this factor. The third
and fourth decimal impact the final calculation because
sales prices are reported with a precision of cents, and
the Costs of the Goods Sold may exceed $1,000.
Incremental Retailer Multiply this factor by the Unitless Breakdown Operating 2007 Annual Retail Trade • Use the coefficient of variation (CV) reported for each of
Markup incremental cost of an EL Expenses of Electronics Survey Detailed Operating the Operating Expenses of Electronics and Appliance
(relative to the baseline 1.11 and Appliance stores into Expenses (ARTS).c The Stores in the data source. The CV of Operating Expenses
product’s cost), to obtain the two categories: invariant ARTS collects detailed data for Electronic Appliance Stores varies between 1% (for
retailer price and variant + profits. on Operating Expenses on Purchased advertising and promotional services) and
Then apply the formula: years ending in 2 and 7 20.8% (for Expensed Equipment). Calculate Standard
IncrementalMarkup=(CG Deviation of each Operating Expense and use it to
S+VariantCosts+Profits)/ estimate a range of plausible incremental markup values.
CGS • Using this approach, the committee finds that the
Incremental markup can be characterized as a random
variable likely to take a value in the range (1.103, 1.118).
99
It can be characterized, for example, as following a
triangular distribution with the parameters:
Minimum=1.103, Maximum=1.118, Average=1.11. The
lower and upper bounds of this range are estimated by
applying the specific Standard Deviation to each of the
Variant Operating Expenses, and assuming sales, profits,
and Invariant Expenses stay as average.
• Consider reporting 3 decimals for this factor. The third
and fourth decimal have an impact in the final
calculation because sales prices are reported with a
precision of cents.
• Use historical data to identify any past trends in retail
markups that can shed light on the projection of future
values.
Sales Taxes Add to the retailer price to % Find the specific rate
obtain the final purchasing cost according to the state
to consumers Varies by state where the household is
or groups of located
states
Average Sales This value is presented in the % DOE takes the Average Sales Tax Clearinghouse • Account for interstate variability. Take the population-
Taxes (Population TSD but not used for the LCC Sales Tax Rate for 9 http://thestc.com/STrates.st weighted average sales tax in a state (city tax + county
Weighted National 7.11 Census Divisions plus the m tax + state tax) for each of the 50 states to calculate a
Average) four largest states in year

DOE Approach
Units and
2014 and weights them range of Sales Taxes for the country. Estimate minimum
by population in 2013 to and maximum tax rates or quartiles or both.
obtain an average for the • Using data of State and Local Sales Tax Rates as of
whole country January 1, 2014d—for 50 states and DC.- and estimates
of the population for the same year,e the committee finds
that the population-weighted average of state and local
sales tax rates is 7.24%. The minimum is 0% (in four
states with 2.3% of the population), and the maximum is
9.45%.
• The committee used a different data source than the one
provided in the TSD because the committee does not
have access to the 2014 data on combined city, county
and states tax rates from the Sales Tax Clearinghouse
(and the data provided for free in the Website is for an
unspecified year and different from the data presented by
DOE).
• Clarify when an average national sales tax used.
• Use public data sources for both tax rates and population
estimates to ensure replicability.
• Estimate the range of sales tax rates for the country.
• Estimate population-weighted average using data for all
100
of the states.
Installation cost Add to the consumer product 2013$ DOE equates labor costs RS Means. Residential Cost Because DOE is estimating LCC Savings and assuming
price to obtain total Installed 2015$ to installation costs. DOE Data 2013f installation costs are the same for all products, this value is
Costs (IC) obtains Labor costs with irrelevant, because it cancels out in the calculation of
149.5 overhead and profits by savings. When DOE estimates LCC savings from a newly
157.93 subtracting material costs purchased baseline product, it should account for the
from total costs variation of labor costs by state, rather than assuming a
single value for the entire country. In the Monte Carlo
simulation, DOE can use the information on the state (or
group of states) where the RECS household is located to
retrieve the corresponding installation costs.
NOTE: ARTS = annual retail trade survey; BLS = Bureau of Labor Statistics; CGS = cost of goods sold; EL = efficiency level; GDP = gross domestic product; GM = gross margin; LCC = life-cycle
cost; RECS = Residential Energy Consumption Survey.
a
https://www.census.gov/data/tables/2011/econ/arts/annual-report.html.
b
The minimum markup value was observed in 2011, the average value was observed in 2007, the maximum value was observed in 1993 (the first year for which there is data in this ARTS report), while
values very close to the average were observed in 1995, 2007 and 2009.
c
https://www.census.gov/data/tables/2007/econ/arts/detailed-operating-expenses.html.
d
https://taxfoundation.org/state-and-local-sales-tax-rates-2014/ Table 1. State and Local Sales Tax Rates as of January 1, 2014.
e
https://www.census.gov/data/datasets/2014/demo/saipe/2014-state-and-county.html.
f
See p. 672 of Gordian. 2013. “RS Means. Residential Cost Data 2013: 32nd Annual Edition.” Kingston, MA: Gordian.

TABLE 4A.2 Review of DOE Approach to Estimating the Value of Operating Costs (OCs) for Dishwashers
DOE Approach
Variable Description/Relevance Units and Estimate Method Data Sources Committee’s Comments
Number of cycles Number of times that a Cycles/year A random draw from a RECS • Were this question in the RECS to be amended to
per year dishwasher completes an distribution depending on give respondents the opportunity to provide the
operational cycle in a Varies by the answer provided by the number of times in a week, month or year that a
year household and household in RECS: dishwasher is used in their household, DOE could
Monte Carlo run a. Less than once per avoid having to resort to guesses and random
week: Triangular, 1 to variable generation to reproduce the number
52 needed.
b. Once per week: So far as probability distributions continue to be
Uniform, 26,78. used, the committee recommends:
c. 2 to 3 times per week:
Uniform, 78 to 182 a. Utilizing all of the parameters (e.g., what is the
d. 4 to 6 times per week: “most likely” value parameter assumed for the
Uniform, 182 to 338 triangular distribution representing the responses
e. At least once per day: “less than once per week”? only the minimum
Triangular, and maximum are specified)
300/500/548 b. Justifying the choice of distribution (e.g., why a
Uniform instead of a Triangular?)
c. Justifying the choice of parameters (e.g., why
does DOE assume that a household that responds
“once per week” has a 50% chance of using it less
101
than once per week, as implied by the Uniform
(26,78)?)
*The title of Table 7.4.1. “RECS dishwasher Usage

data” is misleading. This title is appropriate to
describe the content in columns 1-3 but gives the
wrong impression that the probability distributions of
column 4 are also part of data collected or directly
inferred from RECS. Also, it is unclear if the
probability distributions presented in column four
already account for the adjustment of the total
average of cycles per year, form 171 (as obtained
from the 2009 RECS) to 215 (as obtained from Little
[2001]).
Average number of This number is used to Cycles/year DOE bases this estimate on Arthur D. Little (2001). Review • This number would not play any role in the
cycles per year translate the estimate of a survey of 26,000 of Survey Data to Support calculation of energy consumption per cycle if the
annual energy 215 households conducted by Revisions to DOE’s engineering analysis presented this value.
consumption of an EL Arthur D. Little in 2001 Dishwasher Test Procedure. • DOE should present the per-cycle energy
into per cycle energy December 12, 2001. Prepared consumption both for operating the dishwasher
consumption and to for the U.S. Department of and for heating the water as a result of the
adjust the RECS data to Energy by Arthur D. Little, engineering analysis and make explicit the
obtain an average of Cambridge, MA. assumption about cycles/year in the presentation
of EL curves.

DOE Approach
cycles per year deemed to • In the TSD, DOE (2014) explains that the 2009
be more accurate RECS indicates that out of the 12,100 households
surveyed, 7,382 had dishwashers and used them
171 cycles/year on average. DOE prefers to use
the 215 cycles/year estimated because the Little
(2001) survey has data from more households
(three times more) and is more comprehensive. In
the committee’s judgment, disregarding the data
from the RECS—which is 8 years newer than
Little (2001)—isn’t entirely justifiable. The
average of 171 obtained by DOE from RECS data
may be artificially low because DOE assumes a
probability distribution that implies that
households responding that they use the
dishwasher once a week, have a 50% chance of
using it less than once a week. By adjusting this
distribution DOE can obtain a higher average of
cycles/year and may not need to take the 215
Little (2001) number as a reference.
• The Little (2001) reference does not appear to be
available to the public and could not be found by
the committee.
102
• DOE should base the analysis on publicly
available sources whenever possible and if
needed adjust assumptions.
Duration of the This value is necessary to Hours The caption of Tables 7.2.3 Section 5.5.2.3 of the Engineering Analysis states
average estimate the number of and 7.2.4 imply DOE that DOE expects manufacturers of EL2 to increase
dishwashing cycle Standby hours per year 1 assumes the average the duration of the cycle to compensate for decreased
dishwashing cycle for all water use and maintain washing performance with
Standard Dishwashers and improved efficiency. Given that DOE expects ELs 3
Compact dishwashers is 1 and 4 to improve upon the design options of EL2, it is
hour quite likely that in practice, cycle duration will be
different from 1 hour, depending on the dishwasher
options and the consumer preferences. While the
duration of a cycle has a direct impact on the
estimation of standby energy consumption, it has a
small impact on the operating costs of the
dishwasher. Nevertheless, a more rigorous estimation
of this number in the engineering analysis would
benefit from an improvement in the presentation of
the engineering analysis, which would characterize
different cycles available at the different ELs and for
each cycle, specify its duration, energy and water
consumption.
Standby Power Use Necessary to estimate Watts Tables 7.2.1. and 7.2.2 present estimates of the
Standby Energy Use Standby Power for the baseline and EL of standard

DOE Approach
Standard: and of compact dishwashers without providing any
Baseline: 0 reference. Presumably standby power consumption is
EL1-EL4: 0.5 measured in some existing models and assumed to
Compact: stay unchanged for others but for transparency DOE
Baseline 2.3 should state the conditions of the test and the
EL1: 1.7 assumptions about Standby Power estimates. In
EL2: 0.5 particular, it would be useful to explain the great
discrepancy in standby power use between the
Standard and the Compact product classes.
Standby hours per This value is necessary to Hours/yr DOE estimates the number DOE should characterize the Standby hours per year
year estimate the Annual of Standby hours by based on the characterization of the cycles for each
Standby Energy Use 8551 assuming there are 8766 EL. By assuming that all cycles last 1 hour, DOE
hours in 1 year, and may be overestimating the standby energy
assuming the dishwashers consumption from EL1, EL2, EL3, and EL4.
are run for 215 cycles per According to the estimates presented, Standby energy
year, each taking an represents less than 3% of the annual energy
average of 1 hour to be consumed by the Standard dishwashers and between
complete. Therefore the 3% and 8% of the compact models.
number of standby hours is
8766-215=8551.
103
Annual Standby This value is subtracted kWh/yr
Energy Use from Annual Energy
Use to obtain Per-cycle Standard
Energy Use Baseline 0
EL1-EL4:
4.3kWh/yr
Compact
Baseline 19.7
EL1 14.5
EL2 4.3
Annual Energy Use This is the energy kWh/yr Subtract the estimate of This intermediate quantity is not reported explicitly
dedicated to consumed by the Standby Annual Energy in the TSD but can be calculated from Tables 7.3.1
Dishwashing Only. dishwasher to operate Standard consumption from the and 7.3.2
(I.e., total energy (i.e., heat water, operate Baseline: 307 Estimate of Annual
consumption minus the motors and dry the Energy Use (kWh/yr)
standby energy dishes) EL1: 290.7
consumption) EL2: 275.7
EL3: 229.7
EL4: 175.7
Compact:
Baseline: 204.3

DOE Approach
EL1: 188.5
EL2: 136.7
Per-cycle energy Average energy used by a kWh/cycle Divide Annual Energy • Section 5.5.3 in the Engineering Analysis Chapter
use for each EL dishwasher per cycle of Standard; Use Dedicated to of the TSD presents DOE developed cost-
operation. Calculated for Baseline 307/215 Dishwashing Only efficiency curves for standard and compact
each EL. by the Number of Cycles residential dishwashers based on teardowns and
EL1: 290.7 in a Year cost modeling. The cost efficiency curves relate
the Estimated Annual Energy Use of each EL to
EL2:275.7 Standard; the Incremental Manufacturing Cost ($).
Baseline 307/215 Unfortunately, the section does not explain the
EL3:229.7 process followed to estimate Annual Energy Use.
EL1: 290.7/215 There is no mention of the assumed number of
EL4:175.7 cycles/year or the energy used during standby.
EL2:275.7/215 Later, in section 7, DOE takes this estimate of
Annual Energy Use and divides by an estimate of
Compact: EL3:229.7/215 215 cycles/year to obtain the energy used per
Baseline: 19.7 cycle.
EL1: 14.5 EL4:175.7/215 • DOE’s cost-efficiency curves should relate
EL2: 4.3 Incremental Manufacturing cost and Energy use
per cycle (instead of Annual Energy Use). In
Compact: chapter 5, the TSD reports estimates of the energy
104
Baseline: 204.3/215 use per cycle and standby energy for the baseline
EL1: 188.5/215 dishwasher and for each EL.
EL2: 136.7/215 • DOE should describe in detail all of the
assumptions and/or test conditions that affect the
estimate of energy use per-cycle. For example,
what is the average inlet water temperature? what
is the operating water temperature for each ESL?
Also, any other information that can be derived
from engineering analysis should be reported in
Chapter 5. For example, how much energy is used
to heat water and how much to power the motors
and dry the dishes? Assumptions on the number
of cycles per year should be presented and
discussed in Chapter 7 of the TSD.
Per-cycle Water This estimate is necessary kWh/cycle Assume the use of an Per-cycle water use • The engineering analysis presents an estimate of
Heating Energy to calculate the operating electric water heater with (gallons/cycle) from the Annual Water Consumption and Per-cycle Water
Consumption costs of a dishwasher Standard: 100% efficiency and engineering analysis. Consumption but does not discuss related results
taking into account Baseline 0.82 multiply the per-cycle and assumptions that are necessary for the LCC.
different sources of EL1 0.7 water consumption by an • DOE should present any tests results and/or
energy for water heating EL2 0.58 assumed temperature rise assumptions used in Chapter 5, related to (1)
(e.g. electricity, gas, oil) EL3 0.51 of 70F (21C) and a specific water temperature rise, (2) assumed efficiency of
EL4 0.37 heat of 0.0024 kWh/gal-F the water heater.
(4.186 joule/gram-C)
Compact:

DOE Approach
Baseline 0.58 • The values reported in the sixth column of Tables
EL1 0.51 7.2.3 and 7.2.4 (per-cycle energy use for water
EL2 0.33 heating) are a bit lower than the values the
committee obtained by multiplying the water use
per cycle (i.e., 3rd column in the tables) by 70 F
and by 0.0024kWh-gal-F. (For the Baseline and
ELs 1-3 of the Standard Dishwasher the
committee finds 0.84, 0.71, 0.59, 0.52 versus the
reported values of 0.82, 0.7, 0.58, 0.51.) For the
Compact Dishwasher the committee finds 0.59,
0.52, 0.34 vs the reported values of 0.58, 0.51,
0.33.
Per-cycle energy This value is reported for kWh/cycle Subtract the Per-cycle This value is not used in any LCC or PBP calculation
use for machine and the baseline and each EL Water-heating Energy
drying (motor in tables 7.2.3 and 7.2.4 Compact; Consumption from the per-
energy for pumping but does not play any role BL: 0.36 cycle Energy Consumption
water and electrical in the LCC calculations EL1 0.37
heating element for EL2 0.31
dish drying)
105
Annual Water Use Intermediate calculation Gal/yr Multiply per-cycle Water Per-cycle Water Consumption
Consumption by Number from Engineering Analysis and
Standard; of Cycles in a year Number of cycles from Little
BL:1075 (2001)
EL1:914
EL2:753
EL3:667
EL4:477
Compact;
BL: 753
EL1: 667
EL2: 430
Annual Energy Necessary to estimate the kWh/yr for electric The caption of Tables 7.3.1 • The efficiency of water heaters and water-
Consumption for cost of water heating for water heaters and 7.3.2 indicates that the temperature rise requirements varies
water heating when the baseline and ELs as a calculation assumes water significantly across geographical regions and
Electric, Gas or Oil function of prices of MMBtu/yr for gas heater efficiencies of 98% seasons. This variability will greatly affect the
heaters are used electricity, gas and oil. and oil water electric, 80% for gas and estimates of the energy efficiency of water
The energy consumption heaters. 78% for oil. heating models.
for water heating reported • DOE should account for geographical and
by DOE is 44%-57% of Standard seasonal variability in water heater efficiency
the total energy Electric and temperature rise requirements by collecting
consumed by data and developing technical models. Then

DOE Approach
dishwashing operation for BL: 177, EL1: develop estimates of energy consumption for
the standard models and 150, EL2:124, water heating that are specific to each state,
5%-56% for the compact EL3:110, EL4:79. month and fuel type.
models Gas: • The calculation of Per-cycle Water-heating
BL:0.74, Energy consumption assumes electric water
EL1:0.63, heaters efficiency of 100% and this calculation
EL2:0.52, EL3: assumes 98%.
0.46, EL:4 0.33. • DOE’s engineering analysis should provide
Oil; details on assumptions and test results. If there
BL:0.76, are not any tests results on water heating energy
EL1:0.64, consumption then DOE needs to back up its
EL2:0.53, assumptions on water heater efficiencies with
EL3:0.47, references from the literature or water heater
EL4:0.34 standards.
• The committee could not replicate the numbers
of the third column in Table 7.3.1 and Table
7.3.2. When multiplying annual water use by
70F and 0.0024kWh/gal-F the committee
obtains values of Annual Energy Consumption
for water use that are ~3kWh higher than those
reported. If the committee took into account the
98% efficiency of electric water heaters, then it
106
would obtain values even higher than those
reported.
Electricity prices in Annual average 2013$/kWh Estimate annual average EIA Form 861 which contains, DOE should use the monthly average electricity
the analysis year residential electricity residential electricity prices for every utility serving final prices to residential customers by state calculated by
price in 2012 A different value for each of 27 RECS consumers, annual electricity EIAa. Multiply monthly electricity prices in each
for each of the 27 regions by weighting the sales, revenues from electricity state by monthly electricity consumption in each state
RECS regions annual residential sales, number of customers in to obtain state-specific, monthly electricity costs.
electricity price for a utility each sector.
by the number of
households served in a
region
Natural Gas Prices Annual average of natural 2013$/MMBtu Estimate annual average of EIA publication, Natural gas Residential natural gas prices are likely to be higher
in the analysis year gas residential prices in A different value prices by state and then Navigator which presents than industrial and commercial prices. By using an
2012 for each of the 27 estimate the average price monthly averages of volumes average of the price for the three sectors, DOE is
RECS regions in the RECS region of natural gas deliveries and likely underestimating natural gas costs to residential
weighting each state in the prices by state. customers.
region by its number of DOE should adjust the monthly average of natural
households gas prices by state for all sectors based on the relative
difference between national residential, commercial
and industrial monthly prices reported by EIA in its
natural gas monthly publicationb

DOE Approach
LPG prices in the Annual average of LPG 2013$/MMBtu Estimate average price in EIA’s 2012 State Energy DOE should use monthly residential prices of
analysis year prices for residential A different value the RECS region by Consumption Price and propane by state provided by EIAc to estimate
customers in 2012 for each of the 27 weighting each state in the Expenditures Estimates monthly energy costs for households with LPG water
RECS regions region by its number of (SEDS). heaters. Then use monthly prices by state/region to
households estimate monthly energy costs.
Oil prices in the Annual average 2013$/MMBtu Estimate average price in EIA’s 2012 State Energy DOE should use monthly residential prices of heating
year of analysis residential oil prices in A different value the RECS region by Consumption Price and oil by state provided by EIAd to estimate monthly
2012 for each of the 27 weighting each state in the Expenditures Estimates energy costs for households with LPG water heaters.
RECS regions region by its number of (SEDS). Then use monthly prices by state/region to estimate
households monthly energy costs.
Water prices Annual average of water 2013$/thousand Due to small sample size Water and Wastewater Rate There is numerosity and heterogeneity of the
prices in 2012 gallons of utilities in the survey, Survey conducted by Raftelis municipal water supply industry, and the surveys may
DOE estimates water Financial Consultants and the suffer from response bias. Nonetheless, there are not
A different value volumetric prices for the American Water Works better sources for data than those used by DOE.
for each of the four U.S. Census Regions. Association with data from 290
four census It first finds a volume water utilities and 214
regions (Northeast, weighted average price by wastewater utilities on fixed
Midwest, South state and then finds a and volumetric charges.
and West) regions price by weighting
states averages by the
107
state’s population
Future changes in Necessary to project Unitless DOE calculates annual Annual Energy Outlook 2014 DOE should account for uncertainty on future energy
electricity, natural electricity prices for the changes in prices forecast reference case. prices by considering the AEO projections under the
gas, LPG and oil compliance year (2019) For each fuel, one by the AEO 2014 reference low economic growth and high economic growth
prices and 20 years beyond (i.e., value for each year case for the period 2012- cases. DOE can assume that each of the AEO cases
through 2040) between 2012 and 2040. Then for all of the has equal likelihood and can sample accordingly in
2019 and another years between 2030 and its Monte Carlo simulation. DOE can also look
value for all years 2040 it assigns the average retrospectively at AEO projections, estimate their
between 2030 and annual change for the error and apply that error to current projections to
2040 period obtain an upper and lower bound estimate.
Future changes in Necessary to project Unitless DOE estimated the linear National U.S. city average U.S. city data are only sources available.
water (and waste water prices for the growth in water prices water price index from 1970
water) prices compliance year (2019) One value of from 1970 through 2012 through 2012.
and 20 years beyond (i.e., annual price and extrapolated the trend
through 2040) change for all of to forecast prices through
the projection 2048
years
Repair and Lack of data force DOE to • DOE states that although products with higher
maintenance cost assume that there are no efficiency may have a higher probability of
incremental costs in repair failure due to the increased complexity and
or maintenance from number of parts there is not enough data to
changes in product include this consideration.
efficiency

DOE Approach
• DOE should extend the engineering analysis to
project the impact of standards on the lifetime of
appliances and the cost of maintenance and
repairs. It also recommends that DOE collects
data on the lifetime of equipment and uses it to
inform this variable.
• DOE should infer the duration of key product
components from RECS data for each appliance
type and vintage and given household usage
whenever possible.
• An increase in the costs of repair and
maintenance of a product is likely to play a bigger
role in the LCC than other variables that DOE
estimates in more detail, such as the modest
increase in manufacturer costs due to the cost of
labor.
Product lifetime Necessary to estimate Years DOE adjusts RECS data to • RECS data on the age of • DOE assumes the same lifetime for products of
operating costs for the reflect use outside of the appliance (age response different energy efficiency levels. This
entire lifetime of the Weibul primary residence based on is in the form of bins assumption may be unrealistic in light of the fact
product distribution with AHS data and also takes comprising several years). that as stated by DOE, the increased complexity
shape parameter into account shipments • U.S. Census’s American and number of parts of more efficient products,
108
16.25, and location data from the Appliance Housing Survey (AHS) of increases the cumulative probability of failure.
parameter 2.18 magazine and trade second and vacant homes. Hence, the committee’s recommendations
estimated in associations • Data from Appliance presented in the row above, for obtaining data on
Crystal Ball by magazine and trade the impacts of energy efficiency on repair and
using the inputs: DOE takes as maximum associations on known maintenance costs and product lifetime, also
average=15.4 the 99th percentile history of appliance apply here.
years, minimum=5 shipments. • The committee did not find a detailed explanation
years and on the approach followed by DOE to combine
maximum=50 information from the three data sources described
years in Section 8.2.3 of the TSD.
• DOE should provide a detailed explanation of the
method followed to combine information from
different data sources.
• Appendix 8.D.3 suggests that DOE estimates the
shape and scale of the Weibul distribution from
its estimates of the minimum, median, and
maximum lifetime of a dishwasher. The
committee believes this approach does not take
advantage of the information on the standard
deviation of lifetime of dishwashers that can be
estimated from the RECS sample and its
adjustments.
• DOE should estimate the parameters of the
Weibul distribution using the method of moments

DOE Approach
(or Maximum likelihood) to all of the (adjusted)
observations of the lifetime of dishwashers in the
sample. Also, note that the two-parameter Weibul
with the parameters specified implies there is a
7% chance that the dishwasher fails before 5
years. If 5 years is believed to be the true
minimum of the distribution then consider using a
3-parameter Weibul.
Discount rates Necessary to find the % • DOE estimates this • Federal Reserve Board’s The committee finds DOE’s approach reasonable.
present value of discount rate as an Survey of Consumer
Operating Costs of the One probability opportunity cost of Finances (SCF) containing
dishwasher distribution (30 funds that consumers data on shares of debt and
bins) for each of 6 will spend paying for equity for each of the six
Income groups. the operating and income classes (years 1995,
Income groups: 1: maintenance costs of 1998, 001, 2004, and 2010)
1-20 percentile, 2: the appliance. as well as data on interest
21-40 percentile, • DOE estimates a rates on mortgages, home
3: 41-60 distribution of discount equity loans, credit cards,
percentile, 4: 61- rates for each of 6 installment loans, other
80 percentile, 5: income brackets by residence loans and lines of
81-90 percentile, taking into account the credit.
109
and 6: 90-99 average shares of • Cost of Savings Index Data
percentile) different debts and on interest rates of savings
equity for each group accounts and money market
during 1994-2010, the accounts.
probability distribution • Federal Reserve Board
of interest rates for time-series data on annual
different types of debt, interest rates for
and historical data on certificate of deposit, saving
returns from equity bonds. and AAA corporate
investments bonds.
Rate of return on stocks from

Standard and Poor’s S&P 500.
NOTES: BL = baseline level; EL = efficiency level; kWh = kilowatt-hour; LCC = life-cycle cost; LPG = liquefied petroleum gas; mmBtu = million British thermal units; PBP = payback period; RECS =
Residential Energy Consumption Survey; SCF = standard cubic feet. See A.D. Little, 2001, Review of Survey Data to Support Revisions to DOE’s Dishwasher Test Procedure, Prepared for the U.S.
Department of Energy, Cambridge, MA, December 12.
a
For example, monthly prices for April 2020 are reported in Table 5.6.A in Electric Power Monthly, “Average Price of Electricity to Ultimate Customers by End-Use Sector,”
https://www.eia.gov/electricity/monthly/epm_table_grapher.php?t=epmt_5_6_a.
b
See, for example, Table 3 in “Selected national average natural gas prices, 2015‐2020” in the Natural Gas monthly, June 2020, https://www.eia.gov/naturalgas/monthly/pdf/table_03.pdf.
c
For example Weekly Propane Prices (October-March) for residential customers is provided at https://www.eia.gov/dnav/pet/pet_pri_wfr_a_EPLLPA_PRS_dpgal_w.htm.
d
For example Weekly Heating Oil Prices (October-March) for residential customers is provided at https://www.eia.gov/dnav/pet/pet_pri_wfr_a_EPD2F_PRS_dpgal_w.htm.

Consideration of Alternatives to a National Standards Program
INTRODUCTION
For over four decades, Presidents of both major parties have required agencies to quantitatively and
qualitatively assess expected regulations before issuing them. Executive Order 12866, issued by President
Clinton in 1993, continues to guide today’s regulation development. It requires agencies to conduct a
regulatory impact analysis (RIA) that (1) identifies the public need for regulation, (2) evaluates alternative
approaches to address that need, and (3) estimates the benefits and costs of those alternatives.1
In fulfilling this requirement, the U.S. Department of Energy (DOE) prepares RIAs for its energy and
water conservation standards, and the latter are referred to as the National Standards Program in the
context of the RIA. This chapter considers the RIAs as they appeared in the Technical Support
Documents for the three rulemakings the committee was asked to evaluate (i.e., residential dishwashers,
commercial refrigeration equipment [CRE] and residential furnaces). The discussion in this chapter is
reviewing the analysis of various government interventions but is not advocating for any of the
government interventions ‘real world.’
Broadly speaking, these RIA documents for the three rulemakings evaluate a small set of alternatives
(see Box 5.1) in isolation rather than as an integrated part of the standard-setting framework. Because the
purpose of the RIA is to ensure regulations do more good than harm—based on available ex ante
information—and endeavor to maximize public welfare, greater attention to the RIA could improve the
consumer impacts and societal outcomes of DOE’s standards.2 As previous chapters discuss, the costs and
benefits of efficiency standards depend critically on context, including characteristics of the environment
within which the appliance or equipment is installed; how homeowners or commercial businesses use the
appliance and; characteristics of the manufacturing market. These and other drivers are affected by
regulatory regimes that can both directly shape the environment or shape household and business
incentives in a way that affects appliance and equipment purchases and use. While RIAs are used by DOE
to identify alternatives to a proposed standard, RIAs also identify the context within which a potential
standard can be evaluated, both improving the standard setting process within DOE as well as elucidating
alternatives for other regulatory entities.
In evaluating how the RIA treats these alternatives to the National Standards Program, the committee
is not advocating for any of them. Nor is there any judgment implied by the omission of any one
1
See OMB, 2003, “Circular A-4, Regulatory Analysis,” last modified September 17,
https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/circulars/A4/a-4.pdf.
2
H. Beales, et al., 2017, “Government Regulation: The Good, the Bad, and the Ugly,” Regulatory Transparency
Project of the Federalist Society, June 12, https://regproject.org/wp-content/uploads/RTP-Regulatory-Process-
Working-Group-Paper.pdf.
98

intervention from the list. Rather, the committee is recommending these alternatives be considered in the
RIA analysis.
BOX 5.1 Alternatives to National Standards Evaluated in the Regulatory Impact Analysis
 No New Regulatory Action

 Consumer Rebates
 Consumer Tax Credits
 Manufacturer Tax Credits
 Voluntary Energy Efficiency Targets
 Bulk Government Purchases
SOURCE: DOE 2014a, Chapter 17, 2014b, 2016.
ENERGY EFFICIENCY STANDARDS AND THE BUILT ENVIRONMENT
The built environment is the building or system that the appliance and equipment performance
standards can influence. Prominent examples include residential, commercial, and industrial buildings.
Technology facets of buildings that substantially contribute to appliance and overall energy efficiency
include the appliance’s surrounding built environment and the synergetic operation of the appliance with
the building and adjoining components.
One example of this relationship is that the actual energy consumed by an HVAC device is a function
of its rated efficiency over a range of expected performance; its size in relation to the building; the
installation quality; the building’s windows’ and doors’ insulation; expected ventilation; ambient
temperature and humidity vs. design parameters; and its operation. Similarly, for domestic water heating,
the actual energy consumed depends on the ambient conditions, the water delivery systems’ thermal
integrity, and the input water temperature and the water heater’s controls.
These relationships, as well as the independent operation of appliances, are far more complicated in
actual use than in laboratory conditions. In an ideal world, energy-efficient appliances would be installed
appropriately, optimally operated, and well maintained. In reality, appliances are not perfectly installed or
well-maintained and operate under various circumstances unintended by designers. However, as is
discussed further below, efficiency in larger commercial buildings is becoming more controllable by
remote monitoring and control of connected loads.
Building commissioning addresses the integrated use of energy in buildings, from houses to high-
rises to industrial buildings. Commissioning involves monitoring key energy consumption parameters and
using the building’s controls to make needed adjustments in an initial or start-up phase. Since the building
and its major end-use devices are continually monitored, it also allows for optimizing maintenance and
promptly identifying and rectifying malfunctions.
Some studies have suggested that building’s energy efficiency can degrade from 3-6% per year
(Toole, 2006). One approach to address this is continuous commissioning, in which commissioning is not
be limited to a building’s start-up phase, but used to optimize buildings through all seasons. Consequent
operations monitoring of the building’s heating, ventilating, and air conditioning (HVAC) equipment can
detect and notify when retuning is needed. Once optimized, the building naturally starts a new
degradation process if left alone. Connectivity provides a method to economically monitor the building
equipment’s ongoing performance by reporting consumption and parametric data that can be analyzed to
find (1) when the loss of efficiency occurs and (2) what to do to return the efficiency to optimal
operations. In many cases, this can be accomplished autonomously or by a remote capability. In other
99

cases, a technician or engineer can be dispatched to repair the problem. The key is to know when and
what as soon as degradation occurs. This ongoing monitoring and response service can be provided at a
lower cost than the increased cost due to degradation, primarily because of connectivity. One study cites
paybacks for continuous improvements identified at typically less than 3 years and energy savings of
between 15 and 35%.3 Manufacturers’ literature describes the initial operation of end-use energy-
consuming devices and appliances and many already includes commissioning guidelines for those
devices.
The committee considered whether appliance and equipment standards could be designed to regulate
the minimum efficiency of a device, as it now does, and also regulate attributes of how and where it is
installed and the controls which are later applied to it and decided that aside from possibly adding
attributes of appliance control, such changes are complex and impractical. Current state energy efficiency
building codes address some of the relevant issues; geographic diversity from both a technical and
political perspective adds additional complexity to national standard setting efforts.4 However, the
attributes of appliance controls strongly influence the energy consumption of appliances and devices.
Chapter 6 outlines a possible path aimed at including digital communications and sensors that could
address this opportunity.
ENERGY EFFICIENCY AND CONSUMER BEHAVIOR
DOE’s RIAs consider the impact of several alternatives to the National Standards Program intended
to modify consumer behavior to enhance energy efficiency, including consumer rebates and consumer tax
credits (see Box 5.1). However, the RIAs consider each alternative individually, while the policy analysis
literature emphasizes the effectiveness of a portfolio of responses, so that current RIAs are likely to
understate the effectiveness of these options. In addition, these policies can be complementary to
standards. For example, one study considered the combined effect of appliance standards, labeling and
financial incentives and found them to be complementary (Ürge-Vorsatz et al., 2012). Similar results are
found in the “Consumer Behavior Studies” funded by DOE5 under the American Recovery and
Reinvestment Act of 2009, which attempted to use randomized control experiments to rigorously
investigate how demand-side policies such as informational pamphlets and programmable thermostats
complement real-time pricing to induce greater load shifting. (See Allcott, 2011.) Newer devices for
3
S. Harrell, 2016, “Continuous Saving,” May, http://www.HFmagazine.com.
4
Codes and standards created by various scientific and engineering organizations with fairly broad applicability
include
 The American Society of Heating, Refrigeration and Air Conditioning Engineers (ASHRAE) Standard
90.1-2019 is the current code. Many states delay adaptation from 3 to 6 years and often modify the code
that they adopt.
 The Illuminating Engineering Society of North America (IESNA) publishes standards for the lighting
industry standards. These standards guide lighting professionals and others via recommendations for
product design, output and best practice. The International Code Council (ICC) publishes residential,
energy efficiency and building code documents for code officials. The IECC (International Energy
Conservation Code) uses substantial portions of ASHRAE 90.1. https://www.iccsafe.org/wp-
content/uploads/Code_Adoption_Maps.pdf.
 Air Conditioning Contractors of American (ACCA) publishes standards for residential air conditioners.
 The United States Green Building Council (USGBC) developed the Leadership in Energy and
Environmental Design (LEED) program, which has become the most widely used green building rating
system in the world.
Title 24 California Building Standards Code is perhaps the most complete set of requirements for “energy
conservation, green design, construction and maintenance, fire and life safety, and accessibility,” and apply to the
“structural, mechanical, electrical, and plumbing systems” in a building in California.
5
Further information available at U.S. Department of Energy, “Recovery Act: Consumer Behavior Studies,”
https://www.smartgrid.gov/recovery_act/overview/consumer_behavior_studies.html.
100

managing demand such as learning thermostats afford the end-user a greater degree of control (Mieth et
al., 2021).
This section describes one possible combination of alternatives involving demand-side management.
Analogous opportunities exist on the supply side: current RIAs consider manufacturer tax credits and
bulk government purchases as alternatives to minimum efficiency standards; however, an extensive
literature details how such policies, in conjunction with standards and public R&D programs contribute to
energy efficiency. (See, e.g., Mazzucatto, 2013; NRC, 2001.)
Demand-Side Management
Demand-Side Management programs have been proven to be more effective when complementary
elements are combined. Popular examples include combining alternative pricing with equipment leasing
or consumer education with time-of-use rates. A programmatic approach would study the optimal
combination of demand-side management alternatives, a portfolio that would provide the best results
instead of or in combination with regulatory action.
During the 1970s, demand-side management (DSM) gained increasing traction among policymakers,
regulators, and electric utilities around the world and encouraged widespread adoption of energy-efficient
appliances.6 DSM modifies consumer consumption behavior through education and financial incentives
from electric utilities. DSM allows utilities a lower cost alternative to reducing consumption than building
new assets or acquiring generation capacity in wholesale electricity markets (Gellings, 1984). Recently,
Demand Response has emerged as a related concept (Peak Load Management Alliance, 2017). Demand
Response matches consumers’ demand with supply and embraces peak clipping, load shifting, and energy
conservation.
DSM tries to change the consumer’s behavior in one of two ways: (1) by encouraging the adoption of
a more energy-efficient technology option; and (2) by encouraging the consumer to optimize a
technology’s operations, referred to as consumer response (Gellings, 1985). The RIA already considers
several of the programmatic attributes that could be part of a demand-side management program.
However, the most effective way to use these individual attributes is in combinations deliberately crafted
to leverage their synergistic effects. DSM is offered here as a necessary action to consider in evaluating
the overall effectiveness of a proposed appliance efficiency standard—not as an alternative to appliance
efficiency standards.
Customer characteristics influence both consumer acceptance and response. Characteristics include
income, motivations, attitude, knowledge, and awareness of the technologies and programs available and
decision criteria such as cash flow or perceived benefits and costs. External factors, such as economic
conditions, energy prices, technology characteristics, regulation, and tax credits also influence consumer
acceptance and response. DOE relies on the characteristics of available technology choices and overall
energy use to implement appliance and equipment efficiency standards.
Consumer Preferences and Behavior
Data on Consumer Preference and Behavior

To improve the existing National Standards Program’s effectiveness or to evaluate alternative
approaches, energy efficiency planners need to understand which consumers’ needs are most critical in
the energy purchase decision and how to meet those needs. There are data available on behavior related to
6
Prior to the 1970s, utilities tried to change load curves through so-called “Load Management” programs that
included direct load control and time of use pricing, but, notwithstanding the pricing strategy, were not thought of as
attempting to modify consumer behavior. See Gellings (1981).
101

consumer behavior in both residential and commercial buildings. Such data are important in tailoring a
portfolio of Demand-Side Management programs and activities to each appliance.
The sources of data needed to tailor such a portfolio include the U.S. Energy Information
Administration’s (EIA’s) Residential Energy Consumption Survey (RECS) and Commercial Building
Energy Consumption Survey (CBECS). These surveys, which are two of the data collections of energy
consuming end-use sectors managed by EIA, provide useful data for understanding which stimulus
elements to use for each device. Other organizations and academics have validated and refined these
needs in numerous studies beginning in the 1970s (Taylor, 1975). The Electric Power Research Institute
(EPRI, 1994), for one, identified basic residential customer needs, the importance of which varies
depending on the individual consumer. These include7
 Low Energy Cost: Consumers prioritize saving money on energy bills. They are less interested in
unit prices (e.g., cents/kWh or $/therm8), and more in the total energy bill. Alternative tariffs, bill
credits, or incentives that lower the bill entice these customers. Incentives like pricing options
allow planners to target consumers in promoting new appliances.
 Maintaining Comfort: Many consumers use heating, cooling, and dehumidification to stay
comfortable at home. In promoting a new standard, emphasizing this attribute may enhance
success.
 Safety: Many consumers worry about the safety of appliances and electric and gas systems in and
around their homes. Concerns include electrical surges, gas leaks, carbon monoxide (CO)
poisoning, and fires set by electric distribution systems. New standards that improve real or
perceived safety could use this attribute in encouraging consumers to adopt more efficient
appliances.
 Convenience: Convenience to consumers means not only buying time-saving appliances but also
engaging with providers hassle-free. Consumers do not want to waste time researching appliances
or being able to choose tariff options online. There are many current examples, such as the
investor utilities in the state of Texas where residential electricity consumers can choose the tariff
they wish to be billed under.
 Environmental Impact: Consumers are increasingly aware of resource conservation needs,
impacts of energy consumption on the environment, and the movement of reducing or eliminating
fossil fuel use.
 Security: Consumers worry about their security and that of their homes and properties. Certain
new standards may include features, such as remote monitoring, that make standards desirable.
 Control: Consumers value control. They want to control their appliances. They distrust utility
programs that restrict their energy use unless those programs offer a sound value proposition.
 Appearance: Consumers want their appliances to be attractive, including their electric and gas
service connections. Standards that impact installing auxiliary devices on buildings, such as
added meters and switches, can impact appearance.
 High-Tech: Some consumers are increasingly interested in high-tech appliances with easy-to-use
or customizable features.
The above can modify consumer purchasing behavior, and the alternatives considered in the RIA will
be able to factor these into the analysis using the available data.
7
Adapted from EPRI (1994).
8
10,000 British thermal units.
102

Measures Affecting Consumers9
Consumer Education
Many energy suppliers and governments rely on consumer education to promote public awareness of
energy efficiency. These actors may use brochures, utility bill inserts, information packets,
clearinghouses, educational curricula, or direct mailings. Consumer education is the most basic of the
DSM methods available, and can:
 Inform customers about products/services offered and their benefits, and influence customer
decisions in purchasing and operating these appliances;
 Increase the perceived value of energy-efficient technology and services;
 Inform consumers of the eligibility requirements for participation in programs offered by electric
utilities, government agencies or others;
 Increase the consumer’s knowledge of factors influencing energy purchase decisions, leading to a
better understanding of energy efficiency elements. This could include knowledge of various
electric utility tariffs, awareness of trade ally groups, and a broader understanding of
technologies;
 Provide consumers other information of general interest; and
 Improve relations between consumers, energy suppliers, trade allies, and government entities.
Direct Consumer Contact
Direct consumer contact techniques are face-to-face communications between the consumer and an
energy supplier, technology vendor, trade ally, or government representative to encourage greater
consumer acceptance of energy-efficient appliances and their effective operation. Energy suppliers
employ marketing and customer service representatives to advise on appliance and equipment choice and
operation, sizing of systems, for example, HVAC systems, lighting design, and even home economics.
Direct consumer contact can take many forms, including the following:
 Direct personal (customer) contact. Direct personal contact involves circumstances where utility
representatives come face-to-face with consumers either deliberately or in casual encounters.
Examples of direct contact include canvasing consumers door-to-door, engaging a customer while
providing a service or reading meters, or setting up a booth at the county fair or at the local
market. Direct contact is effective in initiating contact with consumers and underscores that every
employee represents their company. The best sales representatives are often those who already
have a foot-in-the-door such as meter readers and service technicians.
 Energy audits. Energy audits identify qualified candidates for renewable energy systems and
improvements for heating and air conditioning systems, building envelopes, and water heating
systems. They allow suppliers to interact with consumers and sell demand-side options. Energy
audits also aid in obtaining customer feedback and responding to customer concerns.
 Program services. Program services involve energy suppliers’ activities such as those of electric
utilities to support specific demand-side measures like installation and maintenance of heat
pumps, building weatherization, and renewable energy systems such as photovoltaics. Examples
of such programs include equipment installation, servicing, and analyses of consumer options.
Multiple sources provide program services, including government employees, energy suppliers,
contractors, or trade allies.
 Store fronts. A business area where vendors display and inform the public (existing and
prospective consumers) on energy and appliances.
9
This section adapted from Gellings (2009).
103

 Workshops and energy clinics. Energy suppliers, equipment vendors, and utilities host
workshops, clinics, and courses. They range from a few hours in duration to one- or two-day
sessions and cover various topics, including home energy conservation, third-party financing,
energy-efficient appliances, and other demand-side management technologies or activities.
 Exhibits and displays. Exhibits or displays are useful for large public events, including
conferences, fairs, or large showrooms. Vendors use exhibits to promote greater customer
awareness of energy-efficient technologies, appliances, and devices through direct contact.
Vendors also use mobile displays and designed “showcase” buildings.
 Inspection. Inspections typically involve an on-site review of the materials and installation quality
of demand-side measures. These inspections are frequently related to compliance with safety or
code requirements and manufacturer specifications. Inspections offer the implementer additional
opportunities to promote demand-side management options.
Direct Incentives
Direct incentives increase short-term market penetration of a cost control/customer option by

reducing the net cash outlay required for equipment purchase or reducing the payback period to make the
investment more attractive (i.e., increasing the rate of return). Incentives also reduce customer resistance
to options without proven performance histories or options that extensively modify a building or
customer’s lifestyle.
Direct incentives offer financial incentives that may influence the consumer’s purchase decision
regarding appliances and devices and include
 Cash grants. These are payments, typically one-time sums, made to consumers who adopt a
certain appliance or one or more cost control options.
 Rebates. Like cash grants, rebates are normally single payments made to consumers who install a
specific option, either as original equipment or as a replacement for an existing device. The
supplier or implementer generally sets rebate levels in proportion to the relative benefits of the
option.
 Buyback programs. These are special incentives that reflect supplier cost savings resulting from
implementing a mix of cost control options.
 Billing credits. These are credits applied to a customer’s energy bill for installing a particular
option.
 Low-interest or no-interest loans. These are loans offered at below-market interest rates, or
without interest, for the purchase and installation of specific high-efficiency options.
 Direct and indirect incentives. Direct incentives are used in numerous demand-side management
programs to encourage customer participation. Indirect incentives include tax credits from taxing
municipalities and those from state or federal agencies.
While this list is not all-inclusive (variations and combinations of these incentives are often
employed), it presents common, direct incentives for large-scale customer adoption. One additional type
of direct incentive is the exchange of free, or heavily subsidized, equipment installation or maintenance
for participation. Such incentives may cost the supplier more than the direct benefits from the energy or
demand impact but can expedite customer recruitment and collect valuable empirical performance data.
Both customer acceptance and response to direct incentive programs and potential benefits and costs
of these programs depend on the incentive, the technology, and the customer’ characteristics.
104

Advertising and Promotion
Energy suppliers and government energy entities use various advertising and promotional techniques
to influence consumer behavior relative to energy efficiency. Advertising uses various media to
communicate a message to customers to inform or persuade them. Advertising media applicable to DSM
programs include the range of all media: the Internet; radio; television; magazines; newspapers; outdoor
advertising; and point-of-purchase advertising. Promotion supports advertising through press releases,
personal selling, displays, demonstrations, coupons, and contest/awards. Like customer education
methods, advertising and promotion have widespread applicability.
FINDING: Demand-side management offers a wide range of methods which, used in combination,
influence consumer acceptance and use of efficient-energy appliances and hasten their adoption and
market penetration. In this sense it could be seen as a complementary effort to the Appliance and
Equipment Standards Program.
APPLIANCE AND EQUIPMENT STANDARDS AND ELECTRIC POWER GENERATION
Changing the time, pattern, and amount of energy demand on the electric power generation and
delivery system has numerous impacts. Increasing electric demand potentially strains the power delivery
system and adversely affects the environment. When old appliances are replaced with new, energy-
efficient ones, it reduces energy consumption. Consumption decreases result in reduced power generation,
conserving natural resources and decreasing power losses.
As is discussed in Chapter 4, the energy system savings, including reduced environmental impacts,
from efficient appliances are economic benefits of the standards, and are properly included in the
economic assessment of costs and benefits (see Recommendation 4-10). However, regulatory and policy
changes to the grid, including the nature and mix of renewable generation and policies over electricity
pricing both constitute alternative means to increase efficiency of appliances and, like DSM, may
complement the standards program.
System Effects of Coefficient of Performance
Since standards have a 7- to 10-year gestation period from concept to implementation, DOE should
consider the future power generation sources. Today’s heating systems can be either gas furnaces or
electric heat pumps. Coefficients of performance (COPs), that is, energy output versus energy input, for
gas furnaces typically range from 0.8 to 0.96. Electric heat pumps have COPs of 3 and above in most
climates, but can be as low as 1.0 in cold weather applications.
When electricity is generated with wind, solar, or hydroelectric power, the effective COP of electric
heating is at a minimum 3.0 and can reach 6.0 or higher. The United States and the rest of the world are
dramatically increasing the use of wind and solar. Colorado has set a goal to be all renewable electricity)
by 2040.10 Standards need to consider the future generation source since standards take a decade to be put
into use and power sources can rapidly evolve during that time. The replacement market also needs to be
considered.
10
Colorado Governor Jared Polis, 2019, “Governor Polis Releases Roadmap to 100 Percent Renewable Energy
and Bold Climate Action,” May 30, https://www.colorado.gov/governor/news/governor-polis-releases-roadmap-
100-percent-renewable-energy-and-bold-climate-action.
105

Site Energy and Source Energy: An Alternative Way of Measuring Energy End-Use in Building
Energy planners are typically concerned with both so-called “site energy”—energy consumed at the
site or building, and source energy—total energy consumed to produce the energy needed to supply site
energy and that consumed by end-use devices. Source energy is the energy used in the production and
delivery of electricity or natural gas and would include energy expended in mining, transportation, and
losses in electric and gas transmission and distribution. Another way of measurement put forward for
energy consumption by buildings, including that of the appliances and equipment within, is by the source
energy used, therefore accounting for the net efficiency of natural gas and electricity production and
delivery. As the use of renewable resources grows, however, this measure is becoming less meaningful.
Consumer product (i.e., appliance) efficiency standards appropriately are counted toward site energy use
consumed. More generally, in homes, buildings, and industry, site energy generally falls into three facets:
 Converted input energy used for comfort, convenience, or service appliances like lighting or
motive power. Examples include converting natural gas, electricity, or fuel oil into heated water
or tempered air to cool or warm a building;
 Energy consumed by the space or associated devices of the appliance. Examples include building
insulation, insulated fenestrations or the pipes and ducts that are part of the heating system; and
 Energy consumed because of operation and maintenance controls. Examples include temperature
set points or frequency of filtration cleaning.
EXOGENOUS FACTORS THAT COMPLEMENT THE NATIONAL STANDARDS PROGRAM
Other external factors significantly interact with the national standards program to influence
consumers, their contractors, and their suppliers. Addressing these factors through government
intervention is outside the scope of the Buildings Technology Office; however, their impact can be
analyzed in the RIA.
Trade Ally Cooperation11
Trade ally cooperation and support can contribute significantly to the adoption of energy efficient
technology and the success of many energy efficient programs. A trade ally is any organization that can
influence the transactions between the supplier and its customers or between implementers and
consumers. Key trade ally groups in energy efficiency include home builders and contractors, local and
national chapters of professional societies (e.g., the U.S. American Society of Heating, Refrigeration and
Air Conditioning Engineers, the U.S. American Institute of Architects, the Illuminating Engineering
Society, and the Institute of Electrical and Electronic Engineers), technology/product trade groups (e.g.,
local chapters of the Air Conditioning and Refrigeration Institute and the Association of Home Appliance
Manufacturers), trade associations (e.g., local plumbing and electrical contractor associations), and
associations representing appliance retailers.
Depending on the group, trade allies can perform a wide range of services, including
 Development of installation guidelines and procedures;

 Technology transfer;
 Training;
 Certification of installers and service personnel;
 Marketing and sales of energy efficiency activities; and
11
106

 Installation, maintenance, and repair.
In performing these diverse services, trade allies significantly influence appliance and equipment
efficiency and the consumer’s choice.
Alternative Pricing of Energy Efficiency12
The cost of ownership of appliances and equipment is a function of the relative price (first cost) of the
energy consuming device or appliance (the “product” in DOE appliance efficiency terms) and the cost of
the energy consumed. To enhance their energy efficiency, a consumer can (1) buy a more expensive,
higher first cost, but more efficient product or (2) buy less energy or buy the same energy needed at a
lower price. Alternative pricing refers to different schemes of presenting first cost and different terms
under which to purchase the energy desired. Both of these can influence the appliance’s adoption,
potentially encouraging the consumer to choose one alternative over another. With energy-efficient
appliances, the first cost of an energy efficient device may often be higher, but the cost to operate the
device be lower. Pricing as a market-influencing factor performs three functions:
 Informs producers and consumers of the cost or value of products and services being provided;
 Provides financial incentives to use the most efficient production and consumption methods; and
 Determines who can afford how much of a product.
These three functions are closely interrelated. Alternative pricing through innovative schemes that
differ from the norm can help utilities promote demand-side management options. Electricity is purchased
under certain agreed upon conditions called a tariff or rate schedule. There are various tariffs that often
specify the charge per unit energy (e.g., cents/kWh). Sometimes there are also charges for the maximum
demand in a time period (e.g., $/kW for the maximum kW in any 15-minutes in a month). These tariffs
typically vary by season (summer and winter). Tariffs called time-of-day or time-of-use, which vary by
time-of-day of the week, are becoming very popular. In a few cases, dynamic pricing is being explored,
allowing prices to vary in real-time. A group of alternative tariffs has been offered as an effective means
to encourage optimal electricity utilization patterns. Such tariffs can be combined with other strategies
(such as direct incentives) to achieve electric utility DSM goals like adopting energy efficient appliances.
Various pricing structures, such as alternative tariffs or rate designs, are well-suited for different types
of demand-side management options. For utilities, time-of-use rates are offered or tied to specific
technologies (e.g., storage heating and cooling, off-peak water heating). They can be useful for electric
energy storage (e.g., batteries), thermal storage heating or cooling, energy and demand control, and
efficient equipment operation.
Some of the most popular rate designs in use today include
 Demand rates that include charges for both energy (kWh) and rate of energy use or demand (kW)
can be arranged to influence peak demand;
 Time-of-use or time-of-day rates can influence when consumers use appliances;
 Off-peak rates can encourage use during off-peak periods instead of other periods;
 Seasonal rates can reduce demand during high use seasons or encourage consumers to shift use
from one season to another;
 Inverted rates charge an increased cost per unit as consumption increases furthering encouraging
conservation of energy;
 Variable levels of service charge more for higher reliability;
12
107

 Promotional rates can encourage use of certain appliance like storage water heaters or electric
vehicles; and
 Conservation rates can force the installation of higher efficiency devices like air conditioners.
These alternative rates are rapidly entering the electricity market, together with the associated
metering and communication equipment. Chapter 6 discusses the opportunities created by these new
technologies and the Internet of Things.
A national standards program can have an impact on the electric power system, and this impact can
be understood quantitatively. The objective is to estimate a system load shape (purchase pattern of
demand) with and without an action such as the national standards program discussed in the RIA. Using
the load shape, a further estimate can be made through a capacity expansion and production costing
model to understand the impact on the need for capacity and the cost of electricity. While there is no one
integrated model, end use models for efficient appliances and distributed energy resources are available
from NREL, a number of electric utilities have employed elements of modeling done by EPRI (REEPS
for residential; COMMEND for Commercial; IN-DEPTH for industrial), and various consultants such as
NERA.
RECOMMENDATION 5-1: The RIA should be grounded in an appreciation of consumer

needs beyond reduced energy costs. The RIA should analyze not only the impact of appliance
and equipment standards but also the additional impact of measures within a framework that
includes
 The portfolio of voluntary elements of demand-side management; and
 Impact of standards on the overall power system external and internal to the building
and of the natural gas supply and transmission and distribution.
Pigouvian Taxes
Pollution or emissions taxes (Pigouvian taxes13) are perhaps considered the most comprehensive
policy alternative to a standards program. If the market failure associated with a product is externalities
associated with energy use, then a pollution tax or cap-and-trade program can internalize that market
failure by providing consumers with efficient price signals. These market-based approaches exist in the
United States and other countries, both for specific pollutants, such as the sulfur dioxide trading program
for major emitters in the United States, and carbon emissions more broadly. Such programs, particularly
for greenhouse gas emissions, could well become a key component of energy and environmental policy.
Both theoretical and empirical work suggests that Pigouvian taxes and standards are complementary
for efficient environmental and energy policies (Hanemann, 2010; Jaffe et al., 2005). With respect to
appliances, the discussion in Chapter 4 indicates that the market failures associated with appliances are
not limited to externalities from energy use; rather, informational, behavioral and structural features may
also be important. But as with the DSM programs discussed in this chapter, the juxtaposition of market-
based pollution programs and DOE’s appliance program raises issues important to the evaluation of the
standards themselves: does the setting of efficiency standards interact with other public policy measures
currently in place to address these externalities (e.g., carbon trading and markets in California, the
Northeast Carbon market, etc.)? Will regional policy differences mean that energy efficiency standards
might be different in different locations?
13
Taxes on activities that impose unpriced costs on others (externalities) are called Pigouvian taxes in honor of
Arthur Pigou, a 20th century British economist who showed that, unlike taxes on capital and labor, whose net
benefits depend on using the tax revenue for some purpose that offsets the economic efficiency loss necessarily
imposed by the tax, these taxes may confer an economy wide economic efficiency benefit. If the revenues are used
for something useful, all the better.
108

The appliance program provides a useful mechanism to start considering how standards and a market-
based emission program—perhaps a national cap-and-trade program for greenhouse gases—can
complement each other to enhance social benefits.
RECOMMENDATION 5-2: DOE should consider how to incorporate a greenhouse gas

cap-and-trade or tax program into its evaluations of the costs and benefits of energy
efficiency appliance standards.
CONCLUSION
The RIA could include a more complete survey of the alternatives to the National Standards Program.
The RIAs reviewed by the committee already include some of the more prominent examples of
alternatives already in deployment. For example, consumer rebates are one of the alternatives currently in
the RIA. To expand on this example, there are additional elements of demand-side management that
might be included. This would make the set of alternatives more indicative of measures that are currently
deployed for energy and water conservation.
A more robust RIA could prove to be a more influential tool in the standards-setting process. As
discussed in Chapter 2, and in particular Recommendation 2-1, DOE would benefit from organizing its
analyses per the RIA process whereas current the RIA is merely last stage in a series of analyses. The RIA
comes after the analyses DOE has evolved pursuant to its EPCA responsibilities. The analytical
framework of the RIA nonetheless is the step during which important summative attributes of the
National Standards Program such as cost-benefit ratio are invoked. These attributes will be important to a
welfare-maximizing standards-setting process.
REFERENCES
Allcott, H. 2011. “Rethinking real-time electricity pricing.” Resource and Energy Economics 33: 820-
842.
0005.
EPRI (Electric Power Research Institute). 1994. CLASSIFY Profiles: Volume 1: Residential Customer
Needs and Energy Decision Making. Report TR-104567-V1s. Palo Alto, CA, December.
Gellings, C.W. 1981. “Power/Energy: Demand-Side Load Management: The Rising Cost of Peak-
Demand Power Means that Utilities Must Encourage Customers to Manage Power Usage.” IEEE
Spectrum 18(12): 49-52. December.
Gellings, C.W. 1984. Demand-Side Management: Planning, Evaluation, Implementation and Monitoring.
Tenth Annual Rate Symposium. Washington, DC: Institute for the Study of Regulation.
Gellings, C.W. 1985. “The Concept of Demand-Side Management for Electric Utilities,” Proceedings of
the IEEE 73(10): 1468-1470. October.
Gellings, C.W. 2009. The Smart Grid: Enabling Energy Efficiency and Demand Response. Lilburn, GA:
Fairmont Press.
109

Hanemann, M. 2010. “Cap-and-Trade: A Sufficient or Necessary Condition for Emission Reduction?”

Oxford Review of Economic Policy 26(2): 225-252.
Jaffe, A.B., R.G. Newell, and R.N. Stavins. 2005. “A Tale of Two Market Failures.” Ecological
Economics 54(2-3): 164-174.
Mazzucato, M. 2013. The Entrepreneurial State: Debunking Public Versus Private Sector Myths. New
York: Anthem Press.
Mieth, R., S. Acharya, A. Hassan, and Y. Dvorkin. 2021. “Learning-Enabled Residential Demand
Response.” IEEE Electrification Magazine 9(1): 36-44. March.
NRC (National Research Council). 2001. Energy Research at DOE: Was it Worth It? Energy Efficiency
Peak Load Management Alliance. 2017. Demand Response Acronyms and Glossary. May.
https://www.peakload.org/assets/PLMADR%20_AcronymsGlossary_053117.pdf.
Taylor, L.D. 1975. “The Demand for Electricity: A Survey.” Bell Journal of Economics 6(1): 74-110.
Toole, C., and D.E. Claridge. 2006. Review on Persistence of Commissioning Benefits in New and
Existing Buildings. Energy Systems Laboratory. College Station, TX: Texas A&M University.
Ürge-Vorsatz, D., N. Eyre, P. Graham, D. Harvey, E. Hertwich, Y. Jiang, C. Kornevall, M. Majumdar,
J.E. McMahon, S. Mirasgedis, S. Murakami, and A. Novikova. 2012. “Energy End-Use:
Building.” Chapter 10, pp. 649-760, in Global Energy Assessment—Toward a Sustainable
Future. Cambridge, UK: Cambridge University Press. Laxenburg, Austria: International Institute
for Applied Systems Analysis.
110

Addressing the Needs of a Changing World
INTRODUCTION
Energy consumption by end-use appliances and equipment is a function of the following: the
efficiency of the device itself (addressed by the U.S. Department of Energy’s [DOE’s] Appliance and
Equipment Standards Program); the overall system of which it is a part (e.g., a building’s thermal
integrity with heating and cooling devices); and the choices consumers make in controlling the appliance.
The chapter will discuss how the second and third elements may become more important if certain trends
continue in technology, regulation, and business models. Among these changes, the most potentially
relevant for DOE’s Appliance and Equipment Standards Program are advances in Information and
Communications Technology (ICT) enabling the Internet of Things (IoT); the evolution of the Integrated
Grid accompanying increased use of distributed resources; electrification and fuel switching more
generally; and regulatory changes involving electricity markets. There are other changes in the U.S.
electric power system under way that have a less direct impact on the Appliance and Equipment
Standards Program such as a decrease in coal-powered generation, a decrease in nuclear power
generation, and an increase in using central station, utility-scale wind and solar power generation. The key
question for DOE is how these trends and the changing future intersect with the standards program.
While the Appliance and Equipment Standards Program addresses energy and water consumption, the
Program might find an ideal target in economizing value-weighted or price-weighted energy resources in
the overall economy. This approach deviates from DOE’s traditional target in several ways. Until
recently, it was impractical to measure the “first-best” target precisely. This chapter discusses several
reasons the ideal target is now closer in reach than ever, why these changes could help define the program
in the future, and outlines steps to help the program incrementally move in that direction within statutory
limits.
THE INTERNET OF THINGS
The first trend of relevance to the Program relates to the rapid growth in the use of sensors, electronic
communication, and computational ability, coupled with an explosion of digitally-based entertainment
and other data streams can transform the role of appliance and equipment standards in achieving energy
and water conservation. Any appliance or electric energy consuming device can, in principle, be
“connected” through electric or communications infrastructure or both. Nearly everything electric,
biological and mechanical, including lighting systems, parking meters, refrigerators, fork-lift trucks, and
running shoes, is can be connected to the Internet. This IoT that is now made possible affects: appliances,
their installation, controls, and the built environment. In this way the IoT, if it becomes established, might
imply modifications for DOE’s standards program.
111

Size, Scope, and Scale
The size, scope, and scale of the future IoT has been projected by the Organisation for Economic Co-
operation and Development (OECD) to include the next 50 billion machines and devices that will go
online in the coming two decades. OECD projects the number of connected devices in households in
OECD countries to be 14 billion by 2022—up from around 1.4 billion in 2012, or from 10 connected
devices in a household with two teenagers to 50 in 10 years.1 Approximately 46% of the global
population is connected to the Internet today, up from 1% in 1995.2 One example of the proliferation of
internet-connected end-use devices is the adoption of electric vehicles, which exceeded one million
globally in 2016.3
The use of sensors is enabling of an oft-cited trend known as big data. This is discussed later in the
chapter in the section, “Further Sources of Information.”
IoT is Modernizing the Grid
The IoT can foreseeably have an impact on energy end-use. This is mediated through use of smart
meters, smart homes, and smart buildings, providing increased visibility into real-time electricity demand.
Utilities, consumers, and electricity suppliers can use the information from the IoT to match generation
resources accordingly. Vice versa, smart meters and mobile apps will provide consumers with real-time
pricing of electricity. Through the latter, consumers can match their use of electricity to reduce their cost,
the effect of which is to reduce peak demand and level out the consumption profile, providing both more
efficient generation and use of electricity.
The extensive networks of sensors, communications, and computational ability that could be
envisaged include the following:
 Smart meters. U.S. Energy Information Administration (EIA) data shows that in 2019 there were
95 million smart meter installations in the United States.4 EIA data includes meters that measure
and record electricity use at a minimum of hourly intervals and provide that data to both the
relevant utility and its customers daily.5 Smart meters may offer a pathway for real-time data on
end-use of electrical appliances and equipment.
 Smart power system equipment. Most equipment installed on the power system is increasingly
smart because it is computer- or microprocessor-based, including controllers, remote terminal
units (RTUs), and intelligent electronic devices (IEDs). It affects the actual power equipment,
such as switches, capacitor banks, or breakers and the equipment inside homes, buildings, and
industrial facilities. This embedded computing equipment must be robust to handle future
applications for many years without being replaced.
 Blockchain. The public ledgers maintained on the web known as blockchain have applications to
distributed energy resources and energy trading platforms (EFI, 2019b). Blockchain is software in
a digital record-keeping system that can be used for many things in the electric power system
from tracking transactions, like renewable energy certificates, to data about devices as they travel
1
Organisation for Economic Co-operation and Development (OECD), December 11, 2014, “OECD
Technology Foresight Forum 2014—The Internet of Things,” https://www.oecd.org/sti/ieconomy/technology-
foresight-forum-2014.htm.
2
A running tally is kept by the website, internet live stats, http://www.internetlivestats.com.
3
International Energy Agency, “Global EV Outlook 2016: Beyond One Million Electric Cars,” Clean Energy
Ministerial, http://www.iea.org/publications.
4
Energy Information Administration, “How Many Smart Meters Are Installed in the United States, and Who
Has Them?,” https://www.eia.gov/tools/faqs, accessed December 21, 2020.
5
Ibid.
112

through the supply chain. The idea of blockchain is that digital blocks are “chained” together. For
example, blockchain can facilitate payments at an electric vehicle charging station by showing
drivers real-time prices of nearby charging stations.
 Electronic communication systems. Communication systems refer to the media (e.g., TV, radio,
Internet, etc.), the channel (physical route over which the signal is transmitted), and to developing
communication protocols that encode the information into the signal (e.g., the digital format of
the data). The digital format issues are in various stages of development. The appliances and
equipment, including the power system, must be robust enough to accommodate new media as
they emerge from the communications industries while preserving interoperable, secured systems.
 Cybersecurity. Cybersecurity addresses preventing damage to, unauthorized use of, exploitation,
and, if needed, restoration of electronic information and communications systems and services
(and the information contained therein). This ensures confidentiality, integrity, and availability
and applies to end-use appliances and devices and any standards imposed on them.
 Data privacy. Tomorrow’s grid, represented by a widely interconnected system of systems,
presents significant concerns about privacy protection and stewardship. Non-intrusive load
monitoring may be possible (Paris et al., 2018). Care must be taken by DOE and other
stakeholders to ensure that access to information is not an all-or-nothing-at-all choice since
various stakeholders will have differing rights to information from the grid.
FINDING: In the modernizing electricity grid, energy-consuming devices will increasingly have
sensors and controls that are demand-responsive. Likewise, distributed energy resources such as
small-scale renewable generation or energy storage devices can provide real-time information to (and
in turn be controlled by) the independent system operators (ISOs) who operate the electricity grid in
deregulated markets. This additional embedded equipment will have an indirect role in delivering the
energy service currently ascribed solely to the end-use appliance.
THE INTEGRATED GRID
Increasing use of Distributed Energy Resources
A further trend that may affect the Appliance and Equipment Standards Program is the change in the
electric power generation and delivery. The electric power system has evolved into a topology of large,
central power plants interconnected via grids of transmission lines and distribution networks that feed
customers’ power. The system is changing—rapidly in some areas—with the rise of distributed energy
resources (DERs). These DERs include distributed photovoltaic (PV) power generation, small-scale wind
power generation, small hydroelectric power generation; small natural gas-fueled generators, combined
heat and power plants (CHPs), and energy storage such as batteries and flywheels. Energy efficiency
programs and technologies have often been valuable cost-effective alternative to the other resources.6 In
this construct, demand response programs and installations are a further type of DER.
In many places DER have already affected the operation of the electric power grid. For example, in
2016, the California Independent System Operator (CAISO) stated that there were 4,900 megawatts
(MW) of DER in its Balancing Authority.7 Through a combination of technological improvements, policy
incentives, and consumer choices in technology and service, DER’s role is likely to become more
important. Regarding the relationship between DER and energy efficiency standards, DER effectiveness
depends on how devices respond to both automatic and manual control. This includes simple controls
which energize and de-energize or modulate performance based on the desired output of end use devices
6
See, for example, Gellings et al. (1990).
7
North American Electric Reliability Corporation, 2017, “Distributed Energy Resources: Connection Modeling
and Reliability Considerations,” February, http://www.nerc.com.
113

that deliver energy services such as space conditioning (i.e., control of temperature and humidity), motor
torque for various purposes (e.g., ventilation), and hygienic services (hot water).
Implications for Appliance and Equipment Standards
The applicability of appliance and equipment standards, currently in effect at the point of energy end-
use, may become outdated due to the emergence of integrated grids. In the integrated grid concept of
operations, appliances and equipment are flexible and participate in optimizing the value of a combination
of energy efficiency and local generation8 and energy storage integrated with central generation and
storage. In this new integrated grid, energy efficiency includes both appliances and the buildings or
systems that house them as distributed resources. Increasingly these new resources are monitored and
sometimes controlled as part of the grid. These resources also work in tandem with the grid, as small
independent electrical systems cannot always easily support the starting current of certain appliances and
devices like motors without operating in parallel with the grid.
From a total welfare perspective, consumers’ needs can often best be met through deploying energy
efficient resources; it is often lower in cost than supplying more electricity from any conventional sources
(i.e., electricity cost for generation, transmission, and distribution supplied by coal, oil, natural gas, or
nuclear power generation) and has minimal environmental costs. Distributed generation is being installed
at a cost that is competitive with the abatement cost of decreasing electricity consumption.
FINDING: Lower energy costs brought about by increased use of renewable energy generation may
lessen the value of energy efficiency depending on the daily pattern of use the coincident time of
renewable generation.
FINDING: Estimates of energy savings at different times of day and seasons, and the value of those
savings under different assumptions on future penetration of renewable energy, will help describe the
potential benefit of DERs.
FINDING: In a grid with significant penetration of DERs, the flexibility of appliances to postpone
and modulate consumption may be more important than energy efficiency in terms of potential effects
on localized and system-wide costs, emissions, and reliability.
FINDING: Appliance and equipment standards can, if misapplied, be a factor constraining or

delaying the deployment of technologies that add flexibility to electrical loads.
FINDING: A robust and ever-present grid is essential for consumers that partially meet their
electricity requirements from on-site generation (such as solar). The grid is also a back-up to
“hardened” small distributed electrical systems for those that seek to self-generate all of their needs.
In these cases, efficiency is still of prime importance.
FUEL SWITCHING
The next trend of relevance to the Appliance and Equipment Standards Program is the move to
change between energy carriers—natural gas, electricity, hydrogen and so forth—that are used in
providing a given energy service. Some energy services such as hygienic services (e.g., from hot water)
8
Local generation is often referred to as distributed generation (DG). Photovoltaic (PV) power generation
installed by customers is a form of DG. Most practitioners also include DG in a basket of technologies called
distributed energy resources or DERs, which include DG, responsive loads, and electric energy storage.
114

and the appliances that provide them can be accomplished with different energy carriers. The term “fuel
switching” is often used to describe this trend, but it carries with it various connotations, some of which
are misleading. Using this term often relegates electricity to the category of a commodity instead of a
high-valued energy form—a premium fuel. The most popular definitions of fuel switching include (1) the
substitution of one fossil energy source for another, for instance, in steam or combustion turbine power
plants or industrial boilers designed to operate on either coal, natural gas or fuel oil; and (2) programs
created to encourage consumers to change out or buy electric appliances for ones using natural gas or fuel
oil. In the 1990s, fuel switching was used to promote the idea that society should be aggressively
abandoning electric appliances in favor of natural gas appliances (Farnsworth, 2018). Sentiment has since
reversed because of three factors:
 The relative efficiency of electric devices has increased more significantly as compared to
equivalent natural gas counterparts. This is due to the ability of such devices to harness
electricity’s attributes and use, for example, motors instead of engines, microwave heat sources
instead of flames, and vapor compression instead of evaporation;
 Environmental priorities have moved the reduction of carbon dioxide and related pollutants
emitted by natural gas devices forward as a national priority, and
 Electric end-use appliances increasingly have a lower carbon footprint—they are more efficient,
able to be powered by carbon-free sources such as nuclear, solar and wind and, as a result, cause
far lower emissions of carbon dioxide (CO2) and emit smaller volumes of criteria pollutants than
carbon-based fuels.
While natural gas continues to be a popular fuel for building heating and domestic water heating
across the United States, a growing number of states and localities have embraced aggressive policies that
encourage “fuel switching” from intended new gas uses to electric. The California cities of San Jose,
Menlo Park, San Mateo, West Hollywood, Santa Monica, and Marin County will ban or limit the use of
natural gas in new buildings (Groom, 2010; Moench, 2019). Some energy policy experts envisage an
electric economy as the viable strategy for decarbonizing the economy (EFI, 2019a).
California has taken an aggressive posture toward the use of building codes to enable carbon
reduction through electrification by enabling local governments to enact. So-called “Reach Codes” under
The California Codes and Standards (C&S) Reach Codes Program. These codes allow cities and counties
to adopt restrictive codes that require one or more energy measures. In the Santa Clara Valley, sometimes
referred to as Silicon Valley, thirteen communities have created a Reach Code, which mandates the use of
electricity in place of natural gas whenever practical.
CHANGES IN ELECTRICTY REGULATION AND BUSINESS MODELS
The final trend that will affect the Appliance and Equipment Standards Program is one that began in
the 1990’s in which governments worldwide have restructured, privatized, and deregulated the electricity
industry. With restructuring has come a reexamination of the role that energy efficiency plays in the
energy market and a fostering of increased energy efficiency. With market transformation of energy
efficiency, DOE will need to reexamine the need for and character of appliance and equipment standards.
The term, Market Transformation, has been used to describe changes in energy efficiency.9 This new
paradigm for energy efficiency is not yet widely used, nor has it been uniformly defined. More than a
dozen states have restricted their electricity markets to some degree. Markets now allow for energy
efficiency to be remunerated in multiple ways: via time of use rates, via capacity payments, and so forth.
Whether or not DOE’s Appliance and Equipment Standards Program is consonant with this trend—that
9
See, for example, Harris (2019).
115

is, whether its standards would slow down or even preclude investments—could have an impact on the
emergence of new methods and structures that reduce (or mitigate increases in) energy consumption.
FINDING: As DOE currently implements it, the Appliance and Equipment Standards Program
remains unaffected in restructured markets. However, a continuation of business as usual could
dampen incentives as markets are restructured. Markets now allow energy efficiency remuneration in
multiple ways, for example, time-of-use rates or capacity payments.
Market transformation is a coordinated program of activities that fosters private, competitive markets
in which consumers seek and firms provide energy efficient products and services. Incentives provided by
market changes may reduce “market failures” and obviate the need for standards in many cases and thus
may become even more important in supporting consumer choice of efficient appliances.
It could also be the case that governments can design programs to transform markets by reducing
market barriers so that energy-efficient products and services, including energy efficient appliances and
equipment, become widely available and adopted by consumers without external incentives beyond
implementation phases. In such a case, changes in markets resulting from these programs become
permanent and self-sustaining, potentially eliminating the need for appliance efficiency standards.
(Chapter 5 discusses the various alternatives to the National Standards Program that should be considered
in the Regulatory Impact Analysis.)
PATTERN AND AMOUNT OF CONSUMPTION
The changes brought about by digital technologies enabling sensors, communications and
computational ability, explained in the previous sections, provide the framework for developing a
foundation that may allow DOE to begin to shift its focus from the efficiency of an individual appliance
in ideal conditions to a broader focus. As noted above, the efficiency of the device itself; the overall
system of which it is a part (e.g., a building’s thermal integrity with heating and cooling devices); and the
choices consumers make in controlling the appliance are all determinants of the value of the energy that is
supplied. The point of compliance, currently at the point of end-use, (i.e., the consumer product or
industrial equipment), might be changed to be at the system itself. Doing so would include such aspects as
installing and operating the appliance and the demand and pattern of usage—both of which are governed
by consumer behavior. Shifting the point of compliance in this way would emphasize overall system
efficiency and may capture untapped opportunities for reducing energy consumption.
This notion seems fanciful today, but digital technologies may make it feasible tomorrow.
Nonetheless, any standards or guidelines issued to effect this change might need to specify criteria for
installing and controlling appliances and equipment and in doing so anticipate a range of circumstances.
A number of variables govern the in-situ energy consumption of an appliance beyond the operating
conditions assumed in the testing procedures used when creating an efficiency standard. These include the
quality of the installation, as well as the control and operation of the appliance.
For example, in the case of heating, ventilating, and air conditioning appliances that are an integral
part of a building, regulations could include elements that define preferred installation procedures,
controls, and preferred operating parameters in addition to requirements related to the appliance itself.
These could prescribe the need for electrical connections that facilitate real-time operation, enable remote
control for participation in demand control programs, and allow prescribed sensors for data collection.
FINDING: The actual energy consumption of consumer products and industrial equipment is a
function of several elements beyond the device’s efficiency. Information regarding device-level
energy consumption is necessary to enable a program of standards regulation to expand beyond
appliances themselves. There are organizations in civil society, including trade associations and
utilities, that collect these data. These complement the data collected by Energy Information
116

Administration in the Residential Energy Consumption Survey (RECS) and Commercial Building
Energy Consumption Survey (CBECS) programs.
FINDING: Many manufacturers already include sensors, communications (including access ports),
and electronic controls in today’s appliances. These allow manufacturers to add additional features at
little-to-no added cost.
Directly metered, whole-house energy consumption data are increasingly available, facilitated by
smart meters, which have been deployed in more than half of U.S. buildings for revenue metering
purposes. Electric utilities often install these meters to collect data as part of regulatory requirements or
capacity expansion planning, rate setting, and rate design. Dissecting whole-building data affords a better
window on the impact of its efficiency standards as well on the choice of efficiency levels. Disaggregated
data allows energy system planners, regulators and policymakers to best judge equitable policies,
adjudicate tariffs for regulated entities, and plan critical community infrastructure. The list below
indicates the kind of additional granularity in end-use consumption data that are currently available.
 Directly metered individual appliance data are frequently available. Utilities may collect this as
part of load research studies or equipment developers’ research and development activities.
Appliance level data can provide reliable engineering analysis to aid in developing standards and
verifying assumptions. Added detail will provide clarity to the determination of efficiency level.
Utilities and researchers often publish these data online. State public utility commissions and
similar regulatory bodies generally insist these data be in the public domain.
 Whole-house data can be disaggregated. Analysts can disaggregate whole-house data to tease out
appliance-level data through engineering analysis. This makes it possible to correlate this data
with micro-scale analysis of electrical demand coupled with knowledge of the building’s devices.
For heating, ventilation, and air conditioning systems, the change in consumption in various time
horizons compared to the weighted temperature-humidity index in those time horizons can yield
accurate information on disaggregated energy use and performance over time. Alternatively,
analysts can marry whole-house metered data obtained in small increments of time with appliance
saturation survey data and pattern recognition. Simple models can also yield reliable results by
correlating a population of whole-house metered data sets with appliance saturation. Whole-house
data would reflect the synergistic impact of some appliances that DOE does not capture in the
current program. Examples include buildings that use dehumidifiers and air conditioners;
buildings that use outside air for partial cooling; buildings that have multi-stage air conditioners;
buildings where heat pump water heaters and dehumidifiers both operate; installations of water
heaters and dishwashers simultaneously; and buildings with heating and cooling devices
operating at the same time.
 Online monitoring of individual appliances is increasingly available. These data sources are often
part of smart grid demonstrations, obtainable from sensors embedded by manufacturers in end-
use appliances and devices. In a few cases, utilities or providers install them to enable regulatory
justification for expenditures in demonstration projects.
 Use cases for sensors. DOE already encourages the dissemination of smart grid technologies and
has made some “Use Cases” available. Use cases are the blueprints that determine what digital
information and in what form a digital device sends and receives. Use cases are essential for
integrating digital devices in the power system, including the distribution interface with end-use
devices and appliances. By creating use cases for all relevant sensor applications in appliances
and devices, DOE would become a catalyst for their adoption. This would hasten the possibility
of a sensor network that could replace the need for an Appliance and Equipment Standards
Program. There may be opportunities for DOE to partner with developers and energy suppliers to
assist these efforts.
117

 Analyses of demand response data can provide a fairly reliable measure of demand during peak
periods. Metering data will allow analysts to understand the actual appliance-level behavior of the
consumer.
 Data from buildings that are continuously monitored. Such buildings use sensors to provide
ongoing observations of energy consumption and environmental conditions to continually “tune”
the building’s energy systems. Analyzing data from these buildings alongside data from appliance
saturation could reveal the impact of various standards. Tracking the deployment of appliances
and devices combined with continuous monitoring data could improve the development of
standards.
FURTHER SOURCES OF INFORMATION
The sources of data on energy consumption were discussed above in the section “Pattern and Amount
of Consumption.” There are in addition data contributing to the understanding of consumer behavior in
purchasing appliances and equipment, which are summarized below. The phenomenon of big data is
discussed as it applies to energy consumption. Lastly, the importance of data libraries maintained by
utilities is described.
Sales Data
There are many data sources providing information that can indicate the success of the sales of
appliances or the saturation of those devices or both, often characterizing the device’s performance. The
Internet enhances the availability of these data. Initially, efforts may be challenging but it could ultimately
provide a reliable estimate of the success of appliance and equipment performance standards. Some data
that could be gathered by DOE and others, subject to applicable restrictions on collection and use of
data,10 would include the following:
 Manufacturer sales data

 Trade association’s summaries of sales (American Home Appliance Manufacturers, National
Electrical Manufacturers Association, American Refrigeration Institute, etc.)
 Energy efficiency advocacy groups (Alliance to Save Energy, American Council for an Energy
Efficient Economy, etc.)
 Laboratories (National Renewable Energy Laboratory [NREL], Oak Ridge National Laboratory
[ORNL], Pacific Northwest National Laboratory [PNNL], and Electric Power Research Institute
[EPRI], etc.)
 Trade press and the Internet
 Electric utility appliance surveys
 Commercial Building Energy Consumption Survey (CBECS) and Residential Energy
Consumption Survey (RECS)
 Consumer product registration data (i.e., product registration submitted voluntarily to
manufacturers by consumers)
10
Including, for example, the Paperwork Reduction Act.
118

Big Data
Size, Scope, and Scale
The sheer volume of public and private data available regarding energy consumption, consumer
demographics, consumer behavior, and related data such as weather, geography, and economic activity
has caused a new industry to spring up almost overnight.
Several key trends will continue to accelerate the amount of big data relevant to appliance efficiency
standards. These are as follows: (1) the increasing use of sensors, connectivity, and computational ability
of appliances and devices; (2) an increasing need for power system planners and operators to manage the
added stress on the power system resulting from load growth and the increased severity of outages from
natural events (e.g., wildfires and hurricanes); (3) the evolution of the smart grid along with various
alternative pricing techniques (e.g., time-of-use and dynamic pricing) implemented with smart meters
offering more granularity on consumer electricity purchase behavior (Faruqui, 2020); and (4) increasing
interest in externalities from regulators and utilities resulting in increased monitoring, measurement,
rigorous analysis and data storage adding to the data library.
There is a mass of data that building and appliance sensors generate. When paired with metered data
from utility billing and other sources, these data sets can provide key information on need and
effectiveness of standards. The energy industry is spending a great deal of effort to develop analytical
techniques to unpack the information that a data library of energy demand may reveal. For example,
Exelon Utilities’ Utility Analytics lead Brian Hurst reported in Utility Dive that they have “developed 120
use cases for system-integrated analytics and the pipeline is growing” (Trabish, 2019).
There is also a growing interest in continuous monitoring of building energy performance.
Continuous monitoring is ongoing observation of a building’s energy consumption and environmental
conditions to regularly or continually “tune” its energy systems. By tracking the deployment of appliances
and devices combined with continuous monitoring, DOE may be able to simplify the complicated process
of developing standards.
FINDING: Appliance manufacturers are increasingly embedding information and communications

technology such as sensors, digital displays, communications capability and other functionality in
their appliances.
These manufacturers are responding to opportunities in the marketplace. Many consumers indicate
strong preferences for appliances and devices that have greater functionality and are more “high-tech”
with easy-to-read displays and controls. As alternative electricity pricing and demand response become
more widespread, appliance owners and users will have the information and incentives to optimize their
energy purchases and use. These ICT enhancements come at almost no cost. Today’s appliance buyers are
hard-pressed to buy appliances that still use mechanical controls.
Utilities are interested in these developments because customer analytics help them increase customer
satisfaction, enhance demand-side management programs involving energy efficiency and demand
response, reduce costs, and improve service quality. There are numerous uses for data analytics, including
improving customer service, designing demand-side management programs, and marketing. Conducting
parallel analysis of customer satisfaction survey data with automated metering data using artificial
intelligence techniques can reveal detailed information on the use of energy efficient equipment. This
analysis includes insights into individual appliance purchase and use, the presence and use of home
automation, remote auditing and diagnostics, and benchmarking and reporting.
The electric and gas utility industry is increasingly exploring using analytics to mine value from
customer data. These data, combined with EIA’s data from RECS and CBECS, and other publicly
available data sets could form the basis for analytics focused on customer demanded services, including
actual energy consumption, load shape, tariff selections, program participation, information, and special
119

needs and wants. Ultimately, these may lead to the ability to modify methods for setting and evaluating
appliance and equipment standards.
Among the more innovative applications involve using utility billing data along with business data
such as the North American Industry Classification System (NAICS) data, geographic data, weather data,
ZIP Codes, end-use equipment data, building codes, and other local statistics. DOE could easily combine
data from RECS and CBECS in these analyses to create a powerful new method for setting appliance and
equipment performance standards or to obviate the need for prescriptive standards entirely.
Data Analytic Techniques
Big data analytics is the process of examining big data to uncover information—such as hidden
patterns, correlations, market trends and customer preferences. Big data analytics are being used today as
a means by which utilities, consultants and others can leverage the inherent value in all of the data they
already have or can easily obtain. Applying big data analytic techniques to appliance and equipment
standard monitoring and development could fundamentally change the standard setting process and
provide valuable information to manufacturers, consumers, and energy suppliers to further reduce overall
energy consumption.
By enabling DOE to anonymously monitor the operation of both whole building and individual
appliances and analyze that data with information about buildings, appliances, economics and consumer
demographics, energy planners and policymakers could develop detailed models of the end-use
dimension of the energy system. DOE could develop integrated end-use engineering econometric models
with behavioral elements. They could also develop separate models for select residential market
segments, commercial building types, and alternative ownership and operating strategies for those
commercial businesses. These models would enable DOE to:
 Monitor the purchase behavior of appliances and devices and assess the impact of equipment
standards and other programs such as Energy Star on that behavior;
 Monitor the in situ energy use of end-use appliances, devices, and the buildings and systems
deployed in the United States;
 Precisely evaluate the impact of demand response, alternative pricing and demand-side
management programs absent bias such as non-response, self-selection, take-back, free riders,
and long-term participation; and
 Enable a significant improvement in the accuracy of the amount and pattern of forecasts for
energy needs.
This phenomenon of big data revived old data analysis methods and caused development of several
new ones (Bhatia, 2018). Analysts typically draw from techniques based in statistics, computer science,
math, and economics, to process petabytes of records that are collected at a fast pace from trade
transactions, smart meters, or smartphones (McKinsey Analytics, 2018; Ruderman et al., 1989).
DOE developed and has at its disposal EnergyPlus,11 a simulation engine. The BEopt (Building
Energy Optimization Tool) software which runs on this simulation engine is an optimization tool that
finds least-cost building designs for a given energy savings target. It further can identify near-optimal
designs to accommodate builder or contractor preferences.12
11
EnergyPlus is a whole building simulation tool developed by National Renewable Energy Laboratory
(NREL); see https://energyplus.net.
12
Further information is available at NREL, “BEopt: Building Energy Optimization Tool,”
https://beopt.nrel.gov.
120

FINDING: Improved data would allow for better modeling of building energy efficiency at the
whole-house level. There are data available and platforms for analyzing this data currently in use, for
example, BEOpt. By calculating changes over time, uses can compare a measure of a building’s
energy system’s relative effectiveness to time periods before and after adoption of a particular
standard.
FINDING: Analysis of smart meter and power system data can relate energy demand during
specified time periods and from specific appliances to variables such as temperature, relative
humidity, and the efficiency of a building’s energy system.
FINDING: DOE could gain important insights by adding qualitative data and using qualitative
analysis techniques.
Data Libraries
Fundamental to the analyses described above is the availability of data. In addition to DOE’s own
data from RECS, BECS, and other activities, there are relevant data available from other entities. Several
organizations have attempted to establish data repositories or data libraries where energy planners and
policy makers can store and access consumer energy consumption data collected at the building or
appliance level. These libraries, discussed further below, function under the assumption that such data is
transferable so long as the conditions under which it was obtained and cataloged are known. For example,
residential end use data is generally thought to be transferable if the climate conditions, housing
characteristics, and economic activity of the areas are similar. Among the most extensive of these efforts
are those of EPRI, the California Energy Commission (CEC), NREL, and the Association of Edison
Illuminating Companies (AEIC).
EPRI’s effort is called Load Shape Library 7.0.13 The objective of the Load Shape Library is “to
facilitate the collection, use and functionality of a library of representative electric load shapes by climate
zone, geography or by utility.” According to EPRI, the Load Shape Library databases include electric
end-use data aggregated over North American Electric Reliability Corporation (NERC) regions, whole
premise electric data by U.S city, and residential efficient electric technology measures end-use load data.
Data also includes EPRI’s field pilots, results of efforts from a national campaign of the Northwest
Energy Efficiency Alliance (NEEA) to collect end-use data, BPA’s Pacific Northwest RBSA (Residential
Building Stock Assessment), and EPRI CEED (Center for End-Use Energy Data) PowerShape™ data.
ADM Associates, Inc. publishes the CEC work, labeled California Energy Commission: Electric End-
Use Load Shapes.14 CEC has captured the extensive load research data provided by California’s Investor-
Owned Utilities in tools such as R-studio and Energy Plus and the CA System Advisor Model.
NREL has developed end-use load profiles for the U.S. building stock.
AEIC, the Association of Edison Illuminating Companies’ National Load Research Committee has
announced the creation of a National Load Research Data Repository.15
REFERENCES
Bhatia, M. 2018. “Your Guide to Qualitative and Quantitative Data Analysis Methods.” Humans of Data.
September 4. https://humansofdata.atlan.com/author/manu-bhatia.
13
Electric Power Research Institute Load Shape Library 7.0, https://loadshape.epri.com, 2020.
14
ADM Energy Research and Evaluation, https://www.admenergy.com/experience/cec-load-shapes-data-
science.
15
Personal communication between Clark Gellings and AEIC National Office.
121

EFI (Energy Futures Initiative). 2019a. Moniz Presents New EFI Report on Decarbonizing California’s
Economy. https://energyfuturesinitiative.org/news/2019/4/11/moniz-presents-new-efi-report-on-
decarbonizing-californias-economy.
EFI. 2019b. Promising Blockchain Applications for Energy: Separating the Signal from the Noise.
Washington, DC.
Farnsworth, D. 2018. Fuel-Switching: We Just Did This in 1990, So, Why Are We Doing It Again?
Regulatory Assistance Project (RAP) Blog Post. June 11. https://www.raponline.org/blog/fuel-
switching-we-just-did-this-in-1990-so-why-are-we-doing-it-again.
Faruqui, A. 2020. 6 Reasons Why California Needs to Deploy Dynamic Pricing by 2030. Utility Dive.
May 19. https://www.utilitydive.com/news/6-reasons-why-california-needs-to-deploy-dynamic-
pricing-by-2030/578156.
Gellings, C.W., A.P. Fickett, and A.B. Lovins. 1990. “Efficient Use of Electricity.” Scientific American
262: 65-74. September.
Groom, N. 2010. San Jose Moves to Ban Natural Gas in New Buildings in New Residential Buildings.
Reuters. September 17. https://www.reuters.com/article/us-usa-naturalgas-sanjose/san-jose-
moves-to-ban-natural-gas-in-new-residential-buildings-idUSKBN1W302J.
Harris, J. 2019. Market Transformation: Moving Beyond Traditional Energy Efficiency Programs to
Cement Change. Utility Dive. July 3. http://www.utilitydive.com.
McKinsey Analytics. 2018. Analytics Comes of Age. https://www.mckinsey.com/business-
functions/mckinsey-analytics/our-insights/analytics-comes-of-age.
Moench, M. 2019. California Regulators Clear Way for Natural Gas Bans to Take Effect. San Francisco
Chronicle. December 11. https://www.sfchronicle.com/business/article/California-regulators-
clear-way-for-natural-gas-14900008.php.
Paris, J., J.S. Donnal, and S.B. Leeb. 2018. “NilmDB: The Non-Intrusive Load Monitor Database.” IEEE
Transactions Smart Grid 5(5): 2459-2467. September.
Ruderman, H., J.H. Eto, K. Heinemeier. 1989. Residential End-Use Load Shape Data Analysis: Final
Report. LBL-27114. Berkeley, CA: Lawrence Berkeley National Laboratory. April.
Trabish, H.K. 2019. The Biggest Numbers Game in the Power Sector: Data Analytics and the Utility
Community of the Future—Software and Data Are Transforming the Utility Industry and
Connecting Energy Users. Utility Dive. March 25. https://www.utilitydive.com/news.
122

Appendixes


Committee Biographies
LINDA R. COHEN is professor emerita of economics and law at the University of California, Irvine.
Professor Cohen’s research lies at the intersection of economics, law, and political economy. She has
published extensively on the economics of energy policy and innovation policy. Her current work focuses
on the relationship between regulatory policy and innovation policy in addressing climate change and on
the feasibility and effectiveness of alternative innovation policies in the absence of strong markets. She is
a fellow and former council member of the California Council for Science and Technology, and was a
member of the advisory panel for the Public Interest Energy Research Program for the California Energy
Commission. She served on the National Academies of Sciences, Engineering, and Medicine’s committee
assessing the potential for energy efficiency as part of the National Academies project “America’s Energy
Future” and on the American Physical Society’s Panel on Public Affairs for the Committee on Energy and
Environment. She has been a Gilbert White Visiting Fellow at Resources for the Future and a visiting
fellow at the Smith School of Enterprise and the Environment at Oxford University. Professor Cohen
received a Ph.D. in social sciences from the California Institute of Technology and a B.A. in mathematics
from the University of California, Berkeley.
CHARLES CULP is a professor in the Department of Architecture at Texas A&M University, where he is
also the associate director of the Energy Systems Laboratory (ESL). At the ESL, he is responsible for the
development and implementation of Continuous Commissioning® tools used to improve energy
efficiency in hundreds of large buildings including higher education facilities, medical facilities, office
buildings, and airports. He has proposed and secured over $14 million in funding from federal and state
governments and industry while at the Energy Systems Lab. Dr. Culp has conducted sponsor-supported
research in low-energy-use commercial and residential building design; software control algorithms;
energy studies; low velocity air movement studies; air movement in ducts; measurement and verification
technology; serious gaming technology; and energy codes. He has had more than 30 years of experience
in the heating, ventilation, air conditioning, and refrigeration (HVAC&R) field, both in the private and
public sectors. During his employment at Interand, Dr. Culp designed interactive video systems. At
Honeywell, he was a Honeywell fellow and designed and led the development on two major energy
management and control system designs: Honeywell’s Delta 2500 and Delta 21. Previously, he has been
the deputy director of Emerson’s Advanced Material Center, the director of Emerson’s Advanced
Development Center, the director of engineering for Alco Controls, and the director of research for Fisher
Controls. Dr. Culp is active in the American Society of Heating, Refrigerating, and Air-Conditioning
Engineers (ASHRAE); a past president of ASHRAE’s College of Fellows; past chair of the Technical
Activities Committee at ASHRAE; past member of the ASHRAE board of directors; and currently a
voting member of ASHRAE’s Technology Council and Residential Building Committee. Dr. Culp
received a B.S. in physics from the New Mexico Institute of Mining and Technology and a Ph.D. in solid
state physics from Iowa State University.
SUSAN E. DUDLEY is the director of the George Washington University Regulatory Studies Center and
a distinguished professor of practice in the Trachtenberg School of Public Policy and Public
125

Administration. She is the past president of the Society for Benefit Cost Analysis, a senior fellow with the
Administrative Conference of the United States, a fellow of the National Academy of Public
Administration, a member of the boards of the National Federation of Independent Businesses Legal
Center and Economists Incorporated, and the chair of the Federalist Society Regulatory Transparency
Project Regulatory Process Committee. Professor Dudley served as the presidentially appointed
administrator of the Office of Information and Regulatory Affairs (OIRA); directed the Regulatory
Studies Program at the Mercatus Center; taught courses on regulation at the George Mason University
School of Law; served as a staff economist at OIRA, the U.S. Environmental Protection Agency, and the
Commodity Futures Trading Commission; and was a consultant to government and private clients at
Economists Incorporated. She holds an M.S. from the Sloan School of Management at the Massachusetts
Institute of Technology and a B.S. (summa cum laude) in resource economics from the University of
Massachusetts Amherst.
CLARK W. GELLINGS (NAE), an independent energy consultant, recently retired as a fellow at the
Electric Power Research Institute (EPRI), where he was responsible for technology strategy in areas
concerning energy efficiency, demand response, renewable energy sources, and other clean technologies.
He was named EPRI fellow in 2009, in recognition of his more than 28 years of technical innovation and
leadership. Mr. Gellings has made significant contributions to the development of demand-side
management (DSM) and smart-grid research, among other technical areas. He pioneered smart-grid
research when EPRI established its IntelliGrid research program in 1999. He has also conducted research
in energy utilization, electrotechnologies, power quality, electric transportation, thermal and electrical
energy storage, and renewables. From 1982 to 2009, Mr. Gellings served in seven vice president positions
at EPRI. He has received a number of distinguished awards from various organizations. A licensed
professional engineer and a member of the National Academy of Engineering, he has served on the
National Academies’ Panel on Redesigning the Commercial Building and Residential Energy
Consumption Surveys of the Energy Information Administration and on the Committee on Enhancing the
Robustness and Resilience of Future Electrical Transmission and Distribution in the United States to
Terrorist Attack. Mr. Gellings earned his B.S. in electrical engineering from the Newark College of
Engineering, then earned an M.S. in mechanical engineering from the New Jersey Institute of Technology
and an M.S. in management science from the Stevens Institute of Technology.
W. MICHAEL HANEMANN (NAS) joined the Arizona State University Department of Economics and
the Center for Environmental Economics and Sustainability Policy in 2011, where he is a Wrigley Chair
in Sustainability. Dr. Hanemann came to Arizona State University from the University of California,
Berkeley, where he was a Chancellor’s Professor in the Department of Agricultural and Resources
Economics and the Goldman School of Public Policy. His research interests include nonmarket valuation,
the economics of water and of climate change, environmental policy, adaptive management, and demand
modeling for market research. Dr. Hanemann has served on many committees of the National Academies
and was elected to the National Academy of Sciences in 2011. He was a lead author and a contributing
lead author for Working Group III of the Intergovernmental Panel on Climate Change (IPCC) Fifth
Assessment Report on Climate Change. Dr. Hanemann received his B.A. degree from Oxford University
in philosophy, politics, and economics; his M.S. from the London School of Economics in development
economics; and his M.A. and Ph.D. from Harvard University in public finance and decision theory and
economics. He received an honorary Ph.D. from the Swedish University of Agricultural Sciences and the
Lifetime Award for Outstanding Achievement from the European Association of Environmental and
Resource Economists. He is an inaugural fellow of the Association of Environmental and Resource
Economists and a fellow of the American Association of Agricultural Economics.
DALIA PATINO-ECHEVERRI is the Gendell Family Associate Professor at Duke University in the
Nicholas School of the Environment. Her research explores, assesses, and proposes technological, policy,
and market approaches to pursue environmental sustainability, affordability, reliability, and resiliency in
126

the energy sector. Most of her work focuses on the decisions that regulators and private actors must make
regarding capital investment and operations in the electricity industry and quantifying the value of
flexibility along multiple dimensions. Her work uses operations research tools to account for path
dependencies, uncertainty, and risk’s tradeoffs, ubiquitous in the energy system. Professor Patino-
Echeverri received a Ph.D. in engineering and public policy from Carnegie Mellon University and a B.Sc.
and an M.Sc. in industrial engineering from the University of The Andes (Bogota, Colombia).
ANAND PATWARDHAN is currently a professor in the School of Public Policy at the University of
Maryland, College Park. He earlier served as the head of the School of Management from 2003-2004, and
then moved to New Delhi as executive director of the Technology Information, Forecasting and
Assessment Council (TIFAC) in the Ministry of Science and Technology, Government of India, from
2004-2008. Dr. Patwardhan works in the broad area of environment climate studies, focusing on the
assessment of vulnerability and adaptation to climate change, and on the diffusion and adoption of clean
technology. He has been a member of the Scientific and Technical Advisory Panel of the Global
Environment Facility; and a coordinating lead author for the Fourth Assessment Report of the IPCC, and
for the Millennium Ecosystem Assessment. Dr. Patwardhan is the co-chair of the Scientific Steering
Committee for the Global Carbon Project, and is a member of a core consultative group on climate
change for the Indian government. Dr. Patwardhan obtained his B.Tech. in electrical engineering from
IIT-Bombay and later obtained his M.S. in environmental science and engineering and Ph.D. in
engineering and public policy, both from Carnegie Mellon University.
JAMES M. SALLEE is an associate professor in the Department of Agricultural and Resource Economics
at the University of California, Berkeley, and a faculty research fellow of the National Bureau of
Economic Research. He is a research associate of the Energy Institute at the Haas School of Business,
University of California, Berkeley. Previously, from 2008 to 2015, he was an assistant professor in the
Harris School of Public Policy Studies at the University of Chicago. In 2010 he was visiting researcher at
the University of California Energy Institute. He is a public economist who studies topics related to
energy, the environment and taxation. Much of his work evaluates policies aimed at mitigating
greenhouse gas emissions related to the use of automobiles. He completed his Ph.D. in economics at the
University of Michigan in 2008. He also holds a B.A. in economics and political science from Macalester
College.
127

Committee Activities
MEETING #1: NOVEMBER 19-20, 2019

NATIONAL ACADEMY OF SCIENCES BUILDING, WASHINGTON, DC
Day One
Appliance and Equipment Standards: Analysis Methodology Discussion; John Cymbalsky, Director;
Appliance and Equipment Standards Program, Office of Buildings Technology
Commercial Refrigeration Equipment; Lawrence Berkeley National Laboratory, Navigant Consulting and
Cosmos Energy Consulting
Residential Dishwashers; Lawrence Berkeley National Laboratory, Navigant Consulting and Cosmos
Energy Consulting
Residential Furnaces; Lawrence Berkeley National Laboratory, Navigant Consulting and Cosmos Energy
Consulting
Day Two
Question and Answer Session with DOE
MEETING #2: MARCH 2-3, 2020

KECK CENTER OF THE NATIONAL ACADEMIES, WASHINGTON, DC
Day One
Overview of the Building Technologies Office; John Cymbalsky, Director; Appliance and Equipment
Standards Program, Office of Buildings Technology
Responses to Committee’s Questions for DOE; Lawrence Berkeley National Laboratory, Navigant
Consulting and Cosmos Energy Consulting
Engineering and Screening Analysis; Lawrence Berkeley National Laboratory, Navigant Consulting and
Cosmos Energy Consulting
128

Day Two
MEETING #3: MAY 4-5, 2020

VIRTUAL MEETING
Day One
Closed Session of the Committee
Day Two
129


Review of Methods of US DOE 2021

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Review of Methods of US DOE 2021

Uploaded by

Copyright:

Available Formats

THE NATIONAL ACADEMIES PRESS

This PDF is available at http://nap.edu/25992 SHARE

Review of Methods Used by the U.S. Department of Energy in

142 pages | 8.5 x 11 | PAPERBACK

National Academies of Sciences, Engineering, and Medicine 2021. Review of

– Access to free PDF downloads of thousands of scientiﬁc reports

Copyright © National Academy of Sciences. All rights reserved.

REVIEW OF METHODS USED BY THE U.S.

Board on Infrastructure and the Constructed Environment

Division on Engineering and Physical Sciences

A Consensus Study Report of

Copyright National Academy of Sciences. All rights reserved.

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001

International Standard Book Number-13: 978-0-309-68545-0

Copyright 2021 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Proceedings published by the National Academies of Sciences, Engineering, and

Copyright National Academy of Sciences. All rights reserved.

COMMITTEE ON REVIEW OF METHODS FOR SETTING BUILDING AND EQUIPMENT

LINDA COHEN, University of California, Irvine, Chair

Copyright National Academy of Sciences. All rights reserved.

BOARD ON INFRASTRUCTURE AND THE CONSTRUCTED ENVIRONMENT

THOMAS P. BOSTICK, NAE,1 Bostick Global Strategies, Chair

CAMERON OSKVIG, Director

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Linda Cohen, Chair

Copyright National Academy of Sciences. All rights reserved.

Jennifer Baxter, Industrial Economics,

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

2 FRAMEWORK FOR DEVELOPING REGULATIONS 13

3 ASSESSING CURRENT MODELS: ENGINEERING AND TECHNOLOGY

4 THE ECONOMIC ANALYSIS OF STANDARDS 52

Copyright National Academy of Sciences. All rights reserved.

5 CONSIDERATION OF ALTERNATIVES TO A NATIONAL STANDARDS PROGRAM 98

6 ADDRESSING THE NEEDS OF A CHANGING WORLD 111

A Committee Biographies 125

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

SCREENING AND ENGINEERING ANALYSES

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

WELFARE ANALYSIS OF CONSUMERS AND PRODUCERS

Regarding additional data, the challenge is in providing detailed, forward-looking cost-benefit

Copyright National Academy of Sciences. All rights reserved.

CONTEXT OF ENERGY CONSERVATION STANDARDS

Copyright National Academy of Sciences. All rights reserved.

RECOMMENDATION 5-1: The RIA should be grounded in an appreciation of consumer

FUTURE STANDARDS REGULATIONS

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

CONTEXT: ENERGY END-USE

Copyright National Academy of Sciences. All rights reserved.

ROLE OF THE STANDARDS PROGRAM

Copyright National Academy of Sciences. All rights reserved.

CHARGE TO THE COMMITTEE

Per the contract, the committee is charged with the following:

COMMITTEE’S APPROACH TO THE STATEMENT OF TASK

STRUCTURE OF THE REPORT