How Big Data is Becoming a Bigger Deal for the Power Sector

By Tanya Bodell, Energyzt

Big data is a term used to describe massive amounts of information that frequently occurs in the form of unstructured data sets that cannot be analyzed with standard database software.

The energy industry has worked with big data for years, regularly processing significant amounts of information produced on an intra-hourly basis.

Markets settle on metered data that measures power in five-minute increments. Utilities use supervisory control and data acquisition (SCADA) systems. Investors and planners run models with full representation of each generating unit, transmission load flow and hourly dispatch. Although other industries are relatively new to big data, they are finding innovative ways to use it. Applying these innovations to the energy industry promises to be transformative.

Improving the Customer Experience

Big data can modify the ratepayer-electricity relationship into an entirely new customer experience. Amazon introduced the concept of customized customer recommendations 20 years ago, which has been followed by Google, Netflex, Pandora and others, promoting appropriate products to interested customers via the correct communications conduit. Yet, the usual level of interaction between utilities and customers historically has been limited to paper bill inserts that barely warrant a glance.

This is changing.

Utilities that have engaged in social media such as Facebook and Twitter are ready to leverage that information to improve customer interface. Companies such as EnerNoc Inc., Opower and C3 have positioned themselves as big data firms that will revolutionize customer interaction with energy. Telephone, cable, home security and Internet service providers are offering behind-the-meter analytical services directly to end users of electricity, integrating energy management services with their other core offerings. The big data revolution has begun.

Improving Supply Chain Visibility

Other retail industries are using big data to forecast demand and improve supply chain planning.

For example, Walmart’s data warehouse relies on 2.5 petabytes (i.e., 2.5 quadrillion bytes) of data to improve inventory levels and minimize the occurrences of shortages in real time. The power sector has been doing this for years with utilities’ using data to drive instantaneous dispatch decisions and planning decisions to balance supply and demand on a real-time basis.

Automatic generation control systems already work off data tied to generating unit economics, algorithms for coordination of different generation optionality, and transmission constraints.

Existing tools are increasingly limited, however, in the face of higher variability of supply created by intermittent generation resources and fluctuating load levels associated with demand response. Expect big data from newly installed smart meters, syncrophasors and real-time weather information systems to facilitate dispatch decisions on the evolving grid, increase efficiency and improve reliability.

Understanding Risk and Uncertainty

Big data can be used to decipher seemingly indecipherable complexities to inform investment decisions. Oil and gas companies are leveraging advanced geophysics modeling and simulation based on operational data collected from thousands of sensors in subsurface wells and surface facilities to make informed investment decisions in the face of great uncertainty.

As of 2013, Shell had collected 46 petabytes of data using a number of sensors; as of 2012, Chevron Corp.’s internal information technology (IT) traffic exceeded 1.5 terabytes per day. In the power sector, investors also use advanced analytics and modeling to project prices and dispatch to value assets and make investment decisions. Analytical sophistication, however, has been hindered by legacy practices established prior to 21st-century computing capabilities.

Cloud computing and parallel processing allows us to project outcomes under multiple scenarios, generating a distribution of outcomes to enable investors to understand risk and the drivers of uncertainty in their asset values.

We can incorporate big data on weather, customer response, fuel price volatility, alternative retirement and build-out scenarios, capital investment boom-bust cycles and other factors to better inform investment decisions. Big data allows investors to understand the risk profiles of their portfolios and new opportunities, resulting in more informed decisions.

Bigger is Better

Given large amounts of data used in day-to-day operations, many in the power sector might wonder why everyone else thinks big data is such a big deal. It is dangerous to be blasé.

Companies able to leverage big data into value creation opportunities will win; business as usual will falter. It is time to embrace what big data can do for your customers, your business and your investors.


Tanya Bodell is the executive director of Energyzt, a global collaboration of energy experts who create value for investors in energy through actionable insights. Visit Reach her at or 617-416-0651.

“Without big data, you are blind and deaf and in the middle of a freeway.” — Geoffrey Moore

More Electric Light & Power Articles
Past EL&P Articles
Previous articleSiemens wins order for steam power plant equipment from Poland
Next articleNRG starts up solar power project in California

No posts to display