Data Ma

DATA MANAGEMENT AND
ANALYTICS FOR UTILITIES

In-depth Briefing
DATA MANAGEMENT FOR UTILITIES
DATA MANAGEMENT AND

ANALYTICS FOR UTILITIES
In-depth Briefing
About Smart Grid Update

Smart Grid Update is a research-driven news, market analysis, online networking portal and conference producer.
Our work focuses on three core areas of the smart energy technology sector:
Providing business intelligence in all areas of smart energy.
Building smart energy communities for all key players to enable the exchange and sharing of ideas.
Producing conferences that assemble senior management, decision makers, and innovators to produce results
for the smart energy initiative.
Disclaimer
The information and opinions in this document were prepared by Smart Grid Update (FC
Business Intelligence) and its partners. FC Business Intelligence has no obligation to tell you
when opinions or information in this document change. Smart Grid Update makes every
eort to use reliable, comprehensive information, but we make no representation that it is
accurate or complete. In no event shall Smart Grid Update (FC Business Intelligence) and its
partners be liable for any damages, losses, expenses, loss of data, loss of opportunity or prot
caused by the use of the material or contents of this document.
No part of this document may be distributed, resold, copied or adapted without
Smart Grid Update's prior written permission.
Author
Jason Deign
Jason Deign Associates
Carlos Mrquez Salazar,
Research manager
FC Business Intelligence Ltd 2013
www.smartgridupdate.com
DATA MANAGEMENT AND ANALYTICS FOR UTILITIES
| 2
WELCOME
Welcome
Dear Colleague
If knowledge really is power then utilities in 2013 have more power at their disposal than ever
before. Or at least, they could.
But the truth is that while the amount of data being collected by utility companies has increased
vastly with the advent of smart grids, not all that information is being converted to meaningful intelligence.
Having invested in the means to gather masses of data from their grids and networks, utilities are
now grappling with the issue of how best to use it.
There is no clear-cut answer to this question, but a growing body of experience is at least helping to
map out areas where it would be useful to focus attention. This briefing aims to provide a snapshot
of the latest thinking on this issue.
We hope it will help inform your strategies for data management and ultimately add more power
to your business.
Smart Grid Update

Ashley Daugherty
Vice President
Smart Grid Update
201 204 1674
adaugherty@smartgridupdate.com
Join the conversation!

SmartGrid Update | LinkedIn
| 3
ACKNOWLEDGMENTS
Acknowledgments
Smart Grid Update wishes to thank the following people and organizations for their help in compiling this briefing:
Ben Bixby, Chief Executive Officer, MyEnergy | Co-founded MyEnergy in 2007 and is responsible for its overall vision
and strategy. Helps guide the development of MyEnergy's core technology and platform.
Giri Iyer, Product Line Leader for Grid IQ Insight, GE Digital Energy | Has a business team developing and commercializing a world class, real-time analytical system that combines a Big Data platform and 4D visualization technologies.
Brian Rich, Vice President of Business Technology, PG&E | Accountable for providing the strategic direction and
oversight for technology planning, project delivery and critical system operations for customer care, smart meter and
demand-side management programs.
Gib Sorebo, Vice President and Chief Cyber-Security Technologist, Science Applications International
Corporation | Assists government and private sector organizations in complying with legal and regulatory requirements
and address ongoing risks to their infrastructure.
Usman Syed, Senior Policy Adviser, Ontario Ministry of Energy | Leading policy development on smart meter
implementation, time-of-use rollout and various smart grid files including the original design and launch of Ontarios
USD$50 million smart grid fund.
| 4
CONTENTS
Contents
About Smart Grid Update 2
Welcome 3
Acknowledgments 4
Index of tables 5
Introduction 6
1.
Utility data management: market overview
7
1.1
Why are utilities turning to Big Data solutions?
7
1.2
Options for data management
8
1.3
Data management vendors and offerings
10
1.4
Market size and evolution 11
2.
Barriers and drivers 13
2.1 Drivers 13
2.2 Barriers 13
3.
Case study: PG&E 17
4.
Concluding remarks 19
Industry ecosystem 19
Abbreviations 22
References 23
Figures
1
Data analytics options based on IT resources and investment
11
Key challenges in data management for utilities
14
Tables
1
Sources of Big Data in utilities
Data center components and vendors
Utility data analytics options
IT vendors and products for the utility sector
10
Pros and cons of different data analytics options
11
Security considerations
16
Utility data analytics industry ecosystem
20
| 5
INTRODUCTION
Introduction
ata has not traditionally been a major concern

for utilities. Until recently the most data-intensive
elements within the utility business tended to be in
customer-related fields such as billing.
And in these, the level of complexity was average-tolow compared to that seen in other sectors, such as
financial services or telecommunications.
This made it possible to handle the requirements of the
business with modest doses of information technology
(IT) investment and skill.
The process of generating and distributing electricity,
gas and water was essentially dumb, with the utility only
collecting very rudimentary data relating to the levels of
consumption on customer premises. Smart grids have
changed things.
The deployment of advanced metering infrastructure
(AMI) and intelligent supervisory control and data acquisition (SCADA) systems is essentially all about improving
the amount and quality of data that utilities have on
supply and distribution.
Now this equipment is to a growing extent in place and
starting to deliver data back to the utility, the question
is: what should be done with this data to meet the
objectives of the smart grid deployments and, presumably, generate a return on the investment?
To help answer this question, this briefing will seek
to show why utilities are increasingly relying on data
management, which companies can provide the necessary solutions, how the market is evolving, and what
drivers and barriers it has.
| 6
MARKET OVERVIEW
1. Utility data management: market overview
1.1 Why are utilities turning to Big Data

solutions?
Before smart grids, the data that a utility would collect
from its customers frequently amounted to little more
than a monthly meter reading: one data point a month
per customer.
The advent of AMI has increased the level of data
collection dramatically. The Columbia Water and Light
Department, for example, has a pilot where meters
provide data on around half a dozen household circuits
every minute. That is more than 43,000 data points per
customer per month.
Even if, as is often the case, readings are taken at more

infrequent intervals, say every five, 10 or 15 minutes, the
increase is very significant. And this is just one element
of what is currently termed Big Data, data sets that are
terabytes to exabytes in size and come from a range of
sources (Nie, 2011), which puts their management and
analysis beyond the scope of traditional IT tools (see
table 1).
Table 1: Sources of Big Data in utilities
Data type
Technology involved
Notes
AMI
Smart meters
Increased sampling frequency leads to 1,000 to 10,000fold increase in data levels
Grid equipment
Real-time monitoring and control requires much more

granular readings than those taken by smart meters. The
GridSim simulation package described by Anderson et
al (2011), for example, uses a default sample rate of 30
samples per second, per sensor
Third-party
Off-grid data sets
Utilities are increasingly also having to integrate and

handle highly granular data from other sources, such as
pricing details for demand response or forecasting information for renewable energy
Asset management
Firmware for all smart

devices and associated
operating systems
Maintaining smart grid technology once it has been

rolled out requires frequent firmware upgrades and the
like, which equates to a considerable amount of asset
management data
Distribution
automation
| 7
MARKET OVERVIEW
Table 2: Data center components and vendors
Component
Function/notes
Main vendors
Data center
facilities
Dedicated, custom-built housing

incorporating redundant power and cooling
systems, high-speed network connections
and physical security to guard against
unauthorized access
Usually customer-owned or leased

from telecommunications or IT
services provider
Storage arrays
Provide data storage functionality
EMC Corporation, NetApp and IBM
Server platforms
Computing hardware needed for data

handling
HP, IBM and Dell
Storage area
network
equipment
Connects server, storage and external network

resources
Brocade, Cisco and QLogic
Database systems
Software systems for data management and

analysis
Oracle, IBM and Microsoft
Virtualization
systems
Allow more efficient use of discrete storage

and computing resources
VMware, Citrix and Microsoft
1.2 Options for data management

Big Data poses a major problem for utilities: namely,
where do they put the data and what do they do with
it?
The answer to the first part of the question is increasingly in the creation of dedicated data centers, the key
components of which are summarized in table 2.
When considering data center deployment, significant
consideration needs to be given to the following factors:
Because of the value of the information, data centers
typically have some form of contingency for disaster
recovery. This increases the level of capital expenditure, which may be problematic for utilities carrying
out smart grid rollouts.
Virtualization allows all available resources to be
pooled and reallocated, which can improve efficiency and return on investment (ROI), but requires
additional technology and complexity.
Cloud computing provides access to virtual
resources located anywhere, which can further
improve efficiency and ROI but can lead to concerns
over data security and integrity (since data could
end up in foreign jurisdictions).
Cloud-based delivery of IT infrastructure, platforms

and software on an as a service basis means that
instead of having to invest upfront in a data center,
a utility could instead just hire the capacity it needs
from a service provider.
The economics of cloud computing and as-a-service
delivery are hotly disputed (Cohen, 2012) and in the
long run it may be more economical for a utility to build
its own data center facilities, particularly for very large
operations.
However, as-a-service delivery is nevertheless of value
in that it can help overcome capital expenditure
constraints and thus potentially give utilities a way of
gaining value from data more quickly and easily.
Similarly, it could provide an inspiration for smaller,
geographically linked utilities to pool resources via a
shared private cloud, and thus benefit from economies
of scale that no single player could get on their own.
One remaining concern is whether the resources available on a commercial as-a-service basis can be optimized for utility applications. On this point, utilities will
need to seek assurances and service-level agreements
from the vendors concerned.
| 8
MARKET OVERVIEW
Table 3: Utility data analytics options
Data analytics
option
Notes
Vendors/providers
Allows utility to address

the specific challenges and
requirements it faces, and thus
end up with a platform that is
fully made to measure
Very few utilities are likely to have the development

skills and resources needed to create a fully-fledged
data analytics system from scratch. A more realistic
scenario is that a utility with a large in-house IT
capability might be able to carry out a certain
amount of development and integration of existing
systems, thus reducing the need to buy in skills and
products from outside
Rely on OT
systems
The benefit of relying on OT

vendors for data analytics is that
these companies are well versed
in the issues facing utilities and
can therefore provide products
that are highly relevant to the
industry
Existing in-house systems for functions such as

distribution, energy, transmission, outage, or
demand response management are classically
the domain of vendors like GE Energy, which is
beginning to tackle data integration through
products such as Grid IQ. Another OT vendor active
in this area is ABB, which provides data analytics
systems through its subsidiary Ventyx and last year
invested in a water utility software-as-a-service
startup called TaKaDu
Rely on IT
systems
Traditionally within utilities

IT companies have centered
on delivering enterprise
applications and specific
systems such as customer
relationship management
(CRM) databases. Such systems
are frequently part of the mix
required for full smart grid data
analytics and, as a consequence,
mainstream IT vendors are well
placed to provide the remaining
pieces of the puzzle
Mainstream IT vendors such as IBM, Microsoft and

Oracle, or system integrators such as Accenture,
Capgemini or Infosys, have deep and long-standing
relationships with most enterprise players, including
of course those in the energy sector. Most large
IT providers now have significant energy industry
practices, offering complex solutions for the sector
(see below)
Point products
and pure-play
providers
Different components of
the smart grid infrastructure
usually come with their own
management systems and
dashboards and these may on
occasion be sufficient for the
requirements of the utility
Meter makers such as eMeter, Itron and Telvent offer

sophisticated management tools for their products
and for the utility market in general. Itron, to take
one example, has separate software tools for smart
grid analytics, conservation and demand response,
customer care and billing, distribution design and
asset management, revenue assurance, plus energy
forecasting and load research
Third-party data
analytics service
providers
Third-party data analytics service

providers, best exemplified by
MyEnergy, take data feeds from
utilities and analyze them in
order to provide value-added
services to energy customers
It is possible to envisage an extension of the

MyEnergy concept that covers other areas of utility
data management, from asset automation to asset
tracking. While the MyEnergy concept works well
in the US, which is unique in having around 1,200
different power transmission and distribution
companies, it remains to be seen whether it could
be viable in other markets with much fewer players
Develop
systems in
house
| 9
MARKET OVERVIEW
Table 4: IT vendors and products for the utility sector
Vendor
Product
Notes
Capgemini
Smart analytics
platform
Helps utility companies extract insight from the complex

data generated by smart grid and advanced metering
infrastructure deployment
EMC
Smart Grid Analytics
Built to improve grid reliability, reduce operational expense of

grid management and customer experience, and ingest and
process data at petabyte scale with in-database analytics
IBM
Netezza data
warehouse appliance
Delivers high-performance analytical capabilities from smart

grid and smart meter deployments
Oracle
Utilities Meter Data

Analytics
Offers prebuilt dashboards that help utilities track metering

performance, protect revenue and improve billing efficiency
SAP
Smart Meter Analytics
Allows utilities to make crucial decisions faster thanks to

instant, in-depth customer analysis, advanced segmentation
based on energy consumption patterns and energy
efficiency benchmarking
Teradata
Data Warehouse
Appliance
Being used by Oklahoma Gas & Electric is using for its smart
grid rollout (Teradata, 2010)
1.3 Data management vendors and

offerings
Broadly speaking there are five options for dealing with
utility data analytics:
Siemens (Accenture, 2012), which could presumably

indicate a higher level of IT-OT system integration than
might be the case with IT-only vendor solutions. To
complete this discussion, the benefits and disadvantages of the different approaches discussed above are
summarized in the table below.
Develop the systems in house.

Buy systems from an established operational technology (OT) vendor.
Buy systems from an established IT vendor.
Buy and integrate point products from specialist
vendors.
Outsource data analytics to a third party.
It should be noted that these options are not mutually exclusive and in practice it may be beneficial for a
utility to use a mix within its data management strategy.
Details of each option are summarized in table 3 above.
Given the large number of IT vendors operating in the
utility sector, it is useful to take a closer look at some of
the main names and products on offer, as summarized
above in table 4.
In addition to the above, Accenture deserves special
mention for offering a solution in partnership with
| 10
MARKET OVERVIEW
Table 5: Pros and cons of different data analytics options
Data analytics option
Benefits
Disadvantages
Develop systems in house
Highly tailored to the utilitys

requirements
Lack of development skills and

resources
Rely on OT vendor system
Close alignment with operational

processes and requirements; good
integration with OT
Analytics and IT integration

capabilities might not be as good as
for IT vendors
Rely on IT vendor system
Good integration with IT systems;

robust design and support
May include unnecessary features

that increase cost; lack of alignment
with operational processes
Rely on point product or

pure-play vendor system
Close alignment with operational

processes and good integration
with OT
Potential lack of cross-product

and IT system integration; lifetime
support concerns
Rely on third-party service

provider system
Reduced cost, improved analytics
Evolving concept with limited track

record; data sharing may require
culture shift
A further consideration of these strengths and weaknesses enables a rough analysis of which options might
be of greatest interest to utilities depending on a) the IT
resources and capabilities at their disposal and b) their
investment appetite and capability.
This analysis is summarized in figure 1.
Figure 1: Data analytics options based on IT resources and investment
| 11
MARKET OVERVIEW
1.4 Market size and evolution

Supplying electricity is a near-universal business and
utilities therefore represent a major industrial sector.
In the US, which represents the worlds biggest utility
market, there are roughly 1,200 companies dedicated to
power transmission and distribution, with total annual
revenues of some USD$465 billion (First Research, 2012).
Recent estimates of the size of the global market for
utility data analytics range from USD$3.8 billion by 2020
(Leeds, 2012) to $4.2 billion by 2015 (Pike, 2010).
While a detailed analysis of the size and evolution of the
market is beyond the scope of this document, progress
towards the lower range of current estimates seems
likely given that the US market accounts for a significant
proportion of global spend and there:
Stimulus funding is coming to an end.
Pricing for Big Data infrastructures and services
is likely to be affected by ongoing competition
between vendors.
New consumption models such as cloud computing
and as-a-service offerings could potentially reduce
infrastructure costs.
It should be noted that the second two of these factors
could create something of a dilemma for utilities.
Those that delay implementing Big Data projects may
miss out on value but could find the costs of implementation are lower further down the line.
| 12
BARRIERS AND DRIVERS
2. Barriers and drivers
2.1 Drivers
The key driver for the adoption of data management
strategies is clearly the need to handle and analyze the
large amounts of information utilities are now faced
with.
Usman Syed, senior policy adviser at the Ontario
Ministry of Energy, states: All of a sudden there was this
data avalanche, with so much more data than there ever
was before".
The data from the meters is one thing, but there is also
data from substations, transformers and other elements
on the distribution system that can help distributors
monitor and manage things like device asset life, for
example.
Other processes that utilities hope to enhance through
better analysis of data include:
Business operation efficiency. Data analytics will
allow utilities to see where consumers may be stealing
electricity and will allow for better asset management as
well as better system planning.
Implementing efficiency measures. Data allows
for time-of-use rates which in turn allow customers to
monitor their consumption and save money by shifting
use away from times when rates are higher, while also
reducing the need for more expensive forms of peak
generation to be built.
management systems to help retailers and customers

manage their energy use. Better data will also allow utilities to identify and rectify problems with the grid more
quickly, in some cases even before the customer has a
chance to report the issue or outage.
Engaging customers. Having more information about
customers and their usage patterns is generally considered important in helping deliver a better, more tailored
service.
This is perhaps most evident in the case of MyEnergy,
which pools data from many utilities.
MyEnergy can draw on a much greater data set, and
therefore perform much more accurate analyses,
than any utility could on its own; the company uses
this to help US utility customers manage their energy
consumption.
We are the only ones that pull in as much data from
utilities, in the world, states co-founder Ben Bixby.
2.2 Barriers
Smart Grid Update carried out in-depth research interviews among 40-plus utility professionals, ranging from
chief executives to engineers, to find out what they felt
were the three most pressing challenges they faced
regarding data management (see figure 2).
Developing new business models. By adding intelligence to the grid operators can offer new services,
such as energy management, engineering, high-speed
Internet services, cable TV and private network links.
Improving grid resilience and load management.
Distribution companies must reduce power demand
to prevent outages, for example installing energy load
| 13
Figure 2: Key challenges in data management for utilities
25
20
15
10
5
By far the most important topic is actually a set of

concerns around the data itself, rather than allied issues
such as device interoperability, budgets and regulation.
These concerns include:
How can data be used for decision making?
What data will be needed in future?
What data should be collected (and analyzed) now?
While utilities recognize the need for data analysis, there is
still significant doubt over the best way to go about this.
As Giri Iyer of GE says: In the US, many utilities are
taking a stab at Big Data. There are several isolated use
cases, but we see few utilities yet with an Enterprise Big
Data strategy and commitment and they are looking
for early evidence of approach and beneficial business
outcomes.
A particularly pressing problem for many utilities
appears to be whether the data they are now collecting
Impact on operations
Asset management
Concerns over vendors
Cultural inertia/reluctance to change
Renewable energy integration
IT security issues
Legacy system challenges
Lack of hardware capacity
Deploying predictive analytics
Deploying data management analytics
Consumer engagement
Standards and interoperability
Lack of strategic direction
Aging workforce and finding talent
Data acquisition
Regulatory constraints
Budget constraints
Organizational silos
Integration of different systems
What data is relevant?
should mainly be used for reporting or real-time

decision-making. One expert interviewed by Smart Grid
Update summed up as follows: Many organizations just
want reporting of data as a first step, and for reporting
what they want to do is build a computer database and
build business objects that produce reports."
My position has been that you dont know what you
want to report on until you have experts taking data
and figuring out whats meaningful.
Other significant barriers to faster and wider data
management adoption, highlighted in interviews, are
discussed below.
Budget. Unsurprisingly, availability of cash is an important barrier for many utilities, with interview respondents
citing a lack of clear ROI models as a challenge for getting
budget signoffs from the board.
We have a limited budget and this is a large investment, so we want to feel comfortable going down that
road, said one respondent.
| 14
Interoperability. Faced with a bewildering array of

new hardware and software requirements, many utilities
are justly concerned about the ability of technologies to
communicate effectively with each other.
Interoperability is a wide-ranging challenge that covers
areas as diverse as software discrepancies to standard
formats for exchanging data between systems.
Regulation. Regulatory environments can have an
impact on data analysis by determining the potential
use and value that utilities can derive from data in the
first place.
A common theme in interviews was whether deregulation
was proceeding quickly enough, and in the right direction.
Data acquisition. Notwithstanding the large amounts
of data flowing in from AMI, data acquisition continues
to be a barrier for some utilities, particularly in relation to
hardware such as reclosers. Data acquisition needs
to be refined, and will be software related, believed one
interviewee.
Human resources. Utilities appear to face three challenges in human resource terms.
The first is that finding talented IT experts is hard at a time
when most bright technology innovators would rather
work in Silicon Valley, and those that can be attracted to a
utility are at a premium and may not stay long.
The second problem is that the people with the best
operational experience are of an older generation so
there is some urgency about the need to carry out
knowledge transfer.
The third is that people who may have the skills to help
deal with the new data becoming available are generally already tied up with other tasks, and may thus face a
conflict of priorities.
Management buy-in. One of the consequences
of old-school management is that many long-time
energy sector professionals may lack an insight into the
true value that proper data management can provide,
and thus require some convincing. This is all the more
difficult to do in the absence of clear ROI models as
described above.
Silo-based teams. Experts who have spoken with
Smart Grid Update have frequently mentioned
organizational silos as a reason why it is difficult to get

data management strategies off the ground. For example, says one consulting engineer with a large US utility:
We store data in multiple formats in multiple databases.
Retrieving it out of the database is a challenge. It is not
as user-friendly as it could be.
Silo-based operations. This silo effect is exacerbated by
the fact that operational teams tend to work in silos, too.
Iyer says: It can be challenging for different groups to
communicate with one another and share data. Youve
got an operations team that has outage data. Why arent
they communicating with the vegetation management
team that has information about tree trimming operations? If you could corroborate that data you could, for
example, have more efficient vegetation management
processes.
Reluctance to change. Regulatory processes and
monochronistic business models make utilities culturally
resistant to engage in the kinds of process changes that
could help break down silos. Iyer explains: Utilities often
have well-established operational processes that were
set up 20-40 years ago. Typically, with good reason, these
processes can be difficult to modify. The rewards must
exceed the risks inherent in business process change for
utility-wide Big Data management approaches. Meter
analytics and outage analytics seem to be the first areas
where we are seeing evidence of this change.
Market structure. In markets where energy distribution and retail are decoupled there likely to be much
less incentive for utilities to invest heavily in data
management technologies that, after all, might not be
entirely relevant to their business.
Access to funding. If the rates electricity distributors
can charge are regulated by state public utility commissions then winning rate increases to reflect higher costs
can be difficult, which in turn can restrict the funding
available for Big Data.
Planning uncertainty. In many markets legislation to
tackle issues such as carbon emissions are still evolving. This leads to uncertainty that can curtail a utilitys
appetite for major investments in Big Data infrastructure.
Technological complexity. Utilities implementing
smart grids have to get used to a whole new set of technologies. They may understandably want to get to grips
with this innovation before adding Big Data infrastructure and applications on top.
| 15
Finally, although it is currently not recognized as a major

issue it is likely that IT security will emerge as a significant challenge (if not a barrier) for utilities implementing
data management strategies, for reasons discussed in
table 6.
Table 6: Security considerations
No discussion of data management would be complete without at least touching on the subject of IT security. To date,
utilities have largely escaped the attention of cyber criminals, but there are signs this is changing: in 2012 one of the
star attractions at the Def Con Kids cyber-security meeting for young hackers was a talk on how to hack the grid (Deign,
2012).
Recent cyber attacks on power-related companies such as Aramco in Saudi Arabia and Ras Gas in Qatar further
highlight the increasing vulnerability of the sector (Hall and Blas, 2012). IT security has never been much of an issue for
utilities because the infrastructure concerned was hard to access and the potential rewards for hackers were low.
While there still may be few incentives to hack into a power system, however, the fact that most western homes will
at some point in the near future be directly connected to intelligent smart grids means it is much easier for hackers to
access the power network. Gib Sorebo, vice president and chief cyber-security technologist at Science Applications
International Corporation (SAIC), says: Even small changes in the data could affect the stability of the grid and even
jeopardize human safety."
Additionally, utilities will be collecting extensive data on consumer energy usage that could be exploited to identify
when people are not at home, whether they own an electric vehicle, or in some cases find out the brand of appliance
someone is using.
At the very least, he adds, privacy laws around the world dictate that utilities do all they can to protect this data. To do
so, however, requires multiple technology components. Companies like McAfee, Sorebo says, are investing millions in
developing instrumented solutions for industrial control systems and other embedded environments, to improve situational awareness and reduce threat.
He cites EMC Corporation and Oracle as long-standing leaders in data management and security. In addition, utilities would need to rely on specialist companies for features such as radio frequency security, encryption technology,
vulnerability scanning, data leak protection, and monitoring.
Finally, companies such as SAIC are needed to integrate these capabilities and deliver a comprehensive security system.
Privacy rules are currently the main driver for improved IT security in many jurisdictions, but it is clear that the sector
increasingly realizes this is not the only reason to guard data.
Cyber threat actors now have the capability and the intent to destroy hardware and data and the potential to disrupt
operations, Sorebo says. While the potential impacts of these threats are not always fully understood, utilities recognize
they need to act.
| 16
CASE STUDY: PG&E
3. Case study: PG&E
Pacific Gas and Electric Company, incorporated in

California in 1905, is one of the largest combination
natural gas and electric utilities in the United States.
Based in San Francisco, the company is a subsidiary of
PG&E Corporation.
There are approximately 20,000 employees who carry
out Pacific Gas and Electric Company's primary businessthe generation, transmission and delivery of
energy.
The company provides natural gas and electric service
to approximately 15 million people throughout a
70,000-square-mile service area in northern and central
California.
In February 2012 PG&E was named one of Americas
most intelligent utilities in the third annual "UtiliQ"
ranking by Intelligent Utility magazine and IDC Energy
Insights.
But to achieve such praise the company has had to
immerse itself in the field of process analytics. The utility
is being inundated with phenomenal amounts of data,
says Brian Rich, Vice President of Business Technology.
Process analytics means that for the first time the
company is breaking down traditional silos that
havent had the information required to integrate their
processes.
The key challenge the company has faced with the rollout of AMI is that discrete events on the demand side
of the business are now directly affecting events on the
supply and distribution side. This means decisions on
either side can no longer be taken in isolation.
Instead, both sides have to share and jointly analyze
information. PG&E is facing up to this with a number
of projects and pilot initiatives. Rich says the company
is encountering issues that broadly fall into three
categories:
Dealing with unstructured data is a challenge for
most available analytics systems, particularly at the
levels found within PG&E. We have a lot of it, says
Rich, from the maintenance of our asset records
in our field systems to customer service data to all
other sources across the board, unstructured data
creates a great challenge in analytics.
While there are many IT systems that purport to
deal with generic Big Data issues, Rich says there is
a dearth of products tailored to the utility industry.
We see a lot of solutions but not many yet which
demonstrate an understanding of the utility data
model and have productized integration into key
utility operational systems, such as SCADA and AMI,
he says.
Data volumes are a problem. PG&E has 70 terabytes of AMI interval data alone and that volume is
increasing by 3 terabytes a month.
Because data is arriving at such a rate, PG&E has found
that any attempt to perform deep analyses, for example for demand response settlements or evaluation
measurement and verification (EM&V) for energy efficiency programs, risk holding up the flow and affecting
operations.
Its solution is to replicate the data coming in from
operations. The original data set is subjected to core
operational functions such as billing and online presentation to customers.
The replica, meanwhile, is reserved for more in-depth
data mining and analytics. When dealing with the influx
of operational data from AMI and beyond, warns Rich:
Most utilities will find this the biggest challenge in
optimizing analytics: keeping up with the sheer volume
of data while also delivering on the promise of the business value associated with it.
| 17
CASE STUDY: PG&E
Despite the challenges, however, Rich is convinced

PG&E has benefited significantly from process analytics.
One of the results has been a much better knowledge
of consumption patterns, allowing the utility to help
customers drive down their costs.
Before we were flying blind, says Rich. Now we can
have much more meaningful conversations with our
customers when we want to engage them about our
portfolio of demand-side management programs.
Another area where the company is benefiting is in
outage restoration. Previously PG&E was dependent on
customer phone calls to assess the size of an outage,
but now outage notifications and progress on restoration can be tracked via smart meters.
Allied to this, says Rich: We are able to do a lot of overthe-air remediation. If we have a meter that is not giving
us readings we would previously have had to roll a
truck, but now we can often fix it by pushing a firmware
upgrade to it over the network.
Finally, for other utilities considering process analytics,
Rich has this advice: Really prioritize your use cases and
then walk back the data sources that are required for
those use cases."
Getting the utility to track firstly what you are trying to
achieve, and then drilling down to the data, is important. A lot of these deployments are really complex, and
without this it could all be an exercise in futility.
| 18
CONCLUDING REMARKS
4. Concluding remarks
With smart grids, utilities asked for more information.

More information to give customers detailed pricing
signals. More information to detect faults and nontechnical losses. More information to stabilize networks
laden with renewable power.
That wish for more information has been granted. And
now utilities are realizing that it is one thing to collect
data and quite another to manage and analyze it in
a way which can deliver the greatest possible value
to the business. As a result, few utilities can currently
claim to be getting the full benefit of their smart grid
deployments.
on investment.
On this point there is no one-size-fits-all solution, since
the return will to a large degree be dependent on the
nature of a companys business model and operating
environment.
Nevertheless there is a good general awareness of the
benefits that can be derived from better data management, and growing number of real-life examples that
demonstrate how these benefits can be achieved.
In conclusion, although utilities are still only setting
out on the road to better data management, they are
increasingly clear about the routes they need to take,
and are rapidly picking up speed.
In fact it is not unusual for power companies to still be

managing data at a level of granularity comparable to
that of pre-smart grid days, despite now having technology that can deliver much more detailed information.
Why is this so? One reasonable concern is that the
amount of data coming out of AMI and other deployments is simply too great to handle, that utilities will be
swamped in a data deluge and their IT systems will not
be able to cope.
By and large, however, this is not the case. While the
challenges associated with managing the masses of
data associated with smart grids are certainly significant,
they are not overwhelming. Indeed, utility information
management is simply one of several Big Data areas
which a growing number of IT vendors are rushing to
solve. The tools to deal with AMI data may yet need
some refining, but they certainly exist. Instead, the challenges are more to do with cultural shift and particularly
the progression from silo-based operations to a more
connected and interconnected way of working.
At the same time, there is a perhaps understandable
hesitation in prioritizing data management projects
because at this stage of the game utilities are uncertain
about what activities might provide the greatest return
| 19
CONCLUDING REMARKS
Industry ecosystem
A list of the main players in utility data management
(excepting utilities) is provided in table 7. Note this list is
not exhaustive but contains the names of organizations
referenced most frequently in interviews carried out by
Smart Grid Update.
Table 7: Utility data analytics industry ecosystem
Name
Type
Notes
ABB
OT vendor
Provides data analytics through Ventyx and

networking through Tropos
Accenture
IT vendor
Partners with Siemens for AMI solutions
Aclara
Specialist utility data systems provider
Alstom Power
OT vendor
American Energy Solutions
Consultancy
American Public Power

Association
Industry body
Booz Allen Hamilton
Consultancy
Brocade
SAN vendor
Canadian Electricity Association
Industry body
Capgemini
IT vendor
Delivers smart analytics platform
Cisco
SAN vendor
Also active in smart grid infrastructure
Citrix
Virtualization vendor
Consert
Cooper Power Systems
OT vendor
Dell
Server vendor
Edison Electric Institute
Industry body
Electric Power Supply

Association
Industry body
Electricity Distributors
Association
Industry body
EMC
IT/storage vendor
eMeter
AMI vendor
GE Energy
OT vendor
HP
Server vendor
IBM
IT vendor
Now owned by Eaton; recently acquired Eka

Systems
Represents US utilities
Offers Smart Grid Analytics
Markets Grid IQ analytics system
Also provides storage, servers and databases, plus

Netazza warehouse appliance
| 20
CONCLUDING REMARKS
Table 7: Utility data analytics industry ecosystem
Infosys
IT vendor
Integral Analytics
Itron
AMI vendor
Logica
IT vendor
McAfee
IT security vendor
McKinsey
Consultancy
Microsoft
Database vendor
Also active in virtualization
Motorola
Telecommunications
systems vendor
Involved in smart grid communications
MyEnergy
Analytics service provider
National Information Solutions

Cooperative
IT vendor
NetApp
Storage vendor
Oracle
IT/database vendor
OSIsoft
Application software vendor
Power Marketing Association
Industry body
PricewaterhouseCoopers
Consultancy
Qlogic
SAN vendor
SAIC
IT vendor
Offers system integration and security
SAP
IT vendor
Offers Smart Meter Analytics
SAS
IT vendor
Schneider Electric
OT vendor
Siemens
OT vendor
Silver Spring Networks
AMI vendor
SNC-Lavalin
OT vendor
Space-Time Insight
Sprint
Telecommunications
systems vendor
Involved in smart grid communications
TaKaDu
SaaS vendor
For water utilities; ABB is an investor
Telvent
AMI vendor
Teradata
IT vendor
The Society of Industry

Professionals
Industry body
Vmware
Virtualization vendor
Offer Utilities Meter Data Analytics
Partners with Accenture for AMI solutions
Canadian engineering firm
Offers Data Warehouse Appliance
| 21
ABBREVIATIONS
Abbreviations
AMI: advanced metering infrastructure

CRM: customer relationship management
IT: information technology
OT: operational technology
ROI: return on investment
SCADA: supervisory control and data acquisition
| 22
REFERENCES
References
Accenture, 2012. Accenture and Siemens providing smart metering solutions for utilities globally. 18 October. [Press
release] Available at: <http://newsroom.accenture.com/article_display.cfm?article_id=5552> [Accessed 14 January 2013]
Anderson, D. Zhao, C. Hauser, C. Venkatasubramanian, V. Bakken, D. and Bose, A., 2011. A virtual smart grid.
IEEE Power & Energy Magazine, 13 December, pp.49-57.
Cohen, R., 2012. Is cloud computing really cheaper? Forbes, 3 August [online] Available at: <http://www.forbes.com/
sites/reuvencohen/2012/08/03/is-cloud-computing-really-cheaper/> [Accessed 14 January 2013]
Deign, J., 2012. ENISA recommendations: does Europe measure up? Smart Grid Update, 28 November [online] Available
at <http://analysis.smartgridupdate.com/policy-regulation/enisa-recommendations-does-europe-measure> [Accessed 14
January 2013]
First Research, 2012. Electric power transmission, distribution & marketing industry profile [quarterly update] 20
August.
Hall, C., and Blas, J., 2012. Aramco cyber attack targeted production. Financial Times, 10 December [online] Available
at <http://www.ft.com/cms/s/0/5f313ab6-42da-11e2-a4e4-00144feabdc0.html#axzz2HxvxORzy> [Accessed 14 January
2013]
Leeds, D. J., 2012. The soft grid 2013-2020: Big Data & utilities analytics for smart grid. GTM Research, 13 December
[online] Available at <http://www.greentechmedia.com/research/report/the-soft-grid-2013> [Accessed 20 January 2013]
Nie, N., 2011. The rise of big data spurs a revolution in big analytics [executive briefing]. Revolution Analytics.
Pike Research, 2010. Smart grid data analytics market to reach $4.2 billion by 2015. 1 December. [Press release]
Available at: <http://www.pikeresearch.com/newsroom/smart-grid-data-analytics-market-to-reach-4-2-billion-by-2015>
[Accessed 20 January 2013]
Teradata, 2010. Teradata helps Oklahoma Gas and Electric meet tomorrows smart information needs. 28 September.
[Press release] Available at: <http://www.teradata.com/newsrelease.aspx?id=15490> [Accessed 14 January 2013]
| 23

Data Ma

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Data Ma

Uploaded by

Copyright:

Available Formats

DATA MANAGEMENT AND

ANALYTICS FOR UTILITIES

DATA MANAGEMENT FOR UTILITIES

DATA MANAGEMENT AND

About Smart Grid Update

FC Business Intelligence Ltd 2013

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES

Smart Grid Update

Join the conversation!

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES

Data analytics options based on IT resources and investment

Key challenges in data management for utilities

Sources of Big Data in utilities

Data center components and vendors

Utility data analytics options

IT vendors and products for the utility sector

Pros and cons of different data analytics options

Utility data analytics industry ecosystem

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES

ata has not traditionally been a major concern

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES

1. Utility data management: market overview

1.1 Why are utilities turning to Big Data

Even if, as is often the case, readings are taken at more

Table 1: Sources of Big Data in utilities

Increased sampling frequency leads to 1,000 to 10,000fold increase in data levels

Real-time monitoring and control requires much more

Off-grid data sets

Utilities are increasingly also having to integrate and

Firmware for all smart

Maintaining smart grid technology once it has been

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES

Table 2: Data center components and vendors

Dedicated, custom-built housing

Usually customer-owned or leased

Provide data storage functionality

EMC Corporation, NetApp and IBM

Computing hardware needed for data

HP, IBM and Dell

Connects server, storage and external network

Brocade, Cisco and QLogic

Software systems for data management and

Oracle, IBM and Microsoft

Allow more efficient use of discrete storage

VMware, Citrix and Microsoft

1.2 Options for data management

Cloud-based delivery of IT infrastructure, platforms

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES

Table 3: Utility data analytics options

Allows utility to address

Very few utilities are likely to have the development

The benefit of relying on OT

Existing in-house systems for functions such as

Traditionally within utilities

Mainstream IT vendors such as IBM, Microsoft and

Meter makers such as eMeter, Itron and Telvent offer

Third-party data analytics service

It is possible to envisage an extension of the

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES

Table 4: IT vendors and products for the utility sector

Helps utility companies extract insight from the complex

Smart Grid Analytics