You’re Going to Need a Smarter Crystal Ball

by Lawrence E. Jones and Kwok W. Cheung, Alstom Grid Inc.

The next generation of utility forecasting will revolutionize operations, boost reliability and turn smart grids into value engines

Stare at it all you like, but today’s crystal ball is a poor predictor of power grid dynamics. In many ways the tools barely keep up with changing dynamics in the control room. Ever-evolving complexity and lack of long-term energy policy add to the uncertainties that could destabilize power reliability. Control room operators are using a murky crystal ball to make critical decisions. As a result, forecast breakdown threatens the networks’ availability and cost structure.

But before we attempt to make changes in the control room, we must know what’s not working and why. And we must know what factors will need our attention. Now’s the time to ask: What could go wrong?

Wake up to July 2030. Your system has met a new requirement for 30 percent renewable power, with 20 percent wind power generation, 5 percent concentrated solar and other photovoltaic sources, and 5 percent of distributed power coming from customers who installed small wind turbines and solar panels. The uncertainty levels from these intermittent sources are reaching critical levels.

The year 2030 has a long summer of severe, hot weather. It’s topped 100 degrees for the past month. Power demand has escalated past your ability to call for reserves. The wind running the turbines picks up and drops unevenly, and clouds ebb and flow over your entire generation footprint. At one point in the day, strong gusts not forecasted ramp your wind generation capacity to its maximum. The winds later return to forecasted velocities, and you are left with only 50 percent of your capacity available for dispatch.

Problems aren’t confined to the generation side. After years of slow adoption, low prices and better battery life created an unexpected flood of electric vehicle (EV) sales. Original EV demand forecasts called for a relatively predictable load every night. Better battery storage technology, however, changed the game. Cars with high-capacity batteries now need more power to recharge in a session. EVs are just one element of the new smart city you serve. In that city, microgrids, load aggregators, rooftop solar panels and residential wind turbines make the balancing act of grid management even more complex.

You just envisioned a world with uncertainties like never before, a situation in which even an experienced operator will have a hard time keeping up. The control room won’t be alone in the struggle because we can predict one thing right now: The buck will stop in the C-suite.

Rewind to Now-Imagine a New Paradigm

All those things could happen. They might be happening now. But we have no forecasting infrastructure to test for them. It’s not just about building a better crystal ball. The future reliability of the grid starts with our vision-with a broader perspective on the sources of uncertainty that will affect the whole utility value chain, from control room operators to utility management.

To start, we must consider a new set of factors related to keeping the lights on, from the penetration of variable and distributed energy resources to the evolving regulatory frameworks and industry dynamics.

Then we must provide the accurate grid business analytics and economic forecasting tools that utility executives will need to make informed decisions for engaging regulators, shareholders and the general public.

Finally, we must hunt for black swans. Black swan events, according to Nassim Nicholas Taleb in his book “The Black Swan,” surprise observers and have huge impacts. They are the outlandish possibilities no one takes seriously, but when they happen everything changes.

Think about how suddenly the BP disaster in the Gulf Coast changed the pace of oil drilling, the tsunami in Japan reshaped the future of nuclear power and the August 2003 power blackout affected 50 million people in the U.S. and Canada.

We all see a certain logic to the trajectory of change today. Our crystal ball of tomorrow must look outside the logical to identify the forces of nature, the unintended events large or small that we can’t see with the bare eye, so we can prepare for an uncertain future.

From there, we can consider what it will take to manage tomorrow’s grid. Research and development must produce better reliability-based forecasting to upgrade our crystal ball. All of this exploration will help engineer the essential tools that add more predictability to power grid operations.

Take a look at crystal ball 2.0 for answers to these five major changes of the new paradigm.

“- Change No. 1: Increased penetration of variable, rooftop energy resources. Every day, increased wind and solar energy penetration into the grid bring new opportunities for reducing carbon emissions. Low-carbon renewable sources come with a challenge: increased variability and uncertainty coupled with less time to anticipate availability changes. In the 2008 report “20% Wind Energy by 2030: Increasing Wind Energy’s Contribution to U.S. Electricity Supply,” the Department of Energy (DOE) explored a scenario that wind power can contribute 20 percent to the U.S. electricity supply if challenges are addressed. That means we must contemplate what more wind power will do to grid operation and to what degree wind will continue to be a factor. Similar scenarios studied for solar energy point to potential availability issues in different U.S. regions, as well. Controlling for stable, reliable electricity generation and delivery will take more than knowing the problems. It will take better measurement and predictive tools. Wind and solar power present some of the biggest challenges to accurate forecasting and perhaps the most complex. A recent survey of grid operators representing 72 percent of the world’s installed wind capacity looked at how utilities incorporate variable energy forecast information into operating policies, strategies, processes and decision-support tools used daily. The DOE-funded study, “Strategies and Decision Support Systems for Integrating Variable Energy Resources in Control Centers for Reliable Grid Operations,” identifies best practices today and what’s still needed to increase reliability and control. (See sidebar on Page 37.)

“- Change No. 2: Here come EVs, energy storage and more. EVs will increase the variability of the system load significantly as they reach projected double-digit percentages during the next two decades. Dealing with more load from yet-to-be-realized sources won’t depend solely on anticipating demand and adding more capacity. The solutions more likely will lie in new ideas such as aggregating and managing dispatch with price signaling and other forms of automatic control. We even can envision harnessing EV battery storage charged with excess generation as it becomes available.

“- Change No. 3: Carbon footprint, green energy forecasting. Our visionary forecasting must examine how to measure and manage carbon emissions, especially if carbon trading takes hold and puts a price on carbon. In this scenario, utilities must manage how much carbon-producing power is on the grid at any time. At the same time, municipal, commercial and residential customers will need documentation to stand behind green energy contracts as they sign up for 10, 20 or even 30 percent energy from green power sources. If control rooms are to provide specified mixes of low-carbon generation, forecasting must be able to identify the percentage of low-carbon generation in hourly intervals or shorter. In a regulated world, high-carbon-emission generation might not be an option to compensate for the uncertainty of more mercurial renewable sources.

“- Change No. 4: Utility interdependence. Smart cities must consider the interdependency of all utilities and critical infrastructures, i.e., water, energy, telecommunication, transportation. For instance, few realize that most energy sources depend on large amounts of water and vice versa. Our new crystal ball must consider the impact of limits to water availability on generation capacity and related linkages. In addition, when you think of all the new forecasting data and Internet-linked systems planned for tomorrow’s control room, it doesn’t take too much imagination to picture how the whole data-driven infrastructure could crash under the weight of its own complexity. As in so many realms, the convergence of systems could become too complex for anyone to manage. The avalanche of new data, devices and information technology platforms has headed down the mountain. This exchange must take place securely and at high speed. When that happens, bandwidth also could become a limiting factor. Grid operators must predict ways to ensure data flow is secure, reliable and scalable. In the new reality of smart cities, intelligent buildings start to become their own microgrids-automated, but not controlled by the utility power grid. Consider the effect of many smart buildings connected and controlled by price signaling. Hundreds of automated building control systems across an entire metropolitan area could create huge swings in demand that create problems of their own, but they also can provide more flexibility in support of integrated variable generation.

“- Change No. 5: Knowledge exits with an aging work force. Experienced control room operators are poised to retire during the next few years and will take with them vast amounts of undocumented knowledge of how the grid works. The new, less experienced staff will inherit a half-smart grid in a hybrid state, combining old and new systems and a fleet in continuous flux. Grid managers will need a look-ahead strategy for knowledge transfer and training to ensure expertise stays in-house. Careful documentation, synthesis and historical performance archives fed through sophisticated search engines and retrieval tools will then be able to inform future decision-making and operation. The emergence of knowledge management systems will help new employees come online smoothly.

Integrating Forecasting With Decision-making

The dream of a smarter crystal ball, i.e., better forecasting, could serve only to overwhelm dispatchers without corresponding development in decision-support tools. Because wind and solar generation is here today, the industry already is calling for more intelligent ways to expand the time horizon to review wind dynamics while offering better ways to improve reliability and maximize efficiency.

The global survey on integrating variable energy resources found that control centers today and in the future must match new forecasting tools with decision-making systems. The grid operators surveyed identified several examples, e.g., integrating wind forecasts in real-time applications and processes for managing unit commitment, state estimation, regulation, load following, economic dispatch, contingency reserves and contingency analysis. (See Figure 1.)

Look-ahead Capability-It’s not Just Hocus-Pocus Anymore

Technology for operations and resource dispatch on the drawing board seeks to make use of greater look-ahead capability, delivered with more accuracy and confidence in forecast data. Some of this will come from integrating weather monitoring. Wind and solar monitoring and modeling can extend the time horizon needed for response.

Situation awareness under increased uncertainty. Higher penetration of variable energy resources and grid management represents a change in the paradigm we will live with for decades. Given the uncertainties from renewable to distributed sources, change is no longer a matter of choice. The shift requires advanced situation awareness models and tools that deliver greater predictive ability, in turn extending the time horizon for decision-making.

Improving operations-based forecasting and protocols will help in the near term. Consider the effect of bringing on frequently updated, centralized wind power forecasting from multiple providers fed to energy management systems and other control room support tools that facilitate real-time monitoring. Having the facts in hand is a huge start. The goal, however, is to have operations variables in advance, thus reducing uncertainty.

Function follows form-data visualization. The design of the user interfaces could make the difference between heightened control and failure. It’s not just about the data but the way data can be streamed and displayed for a clearer picture of real-time vs. potential options. By 2030 operators must cope with an avalanche of information, which will be useless without dynamic, personalized interfaces. The next phase will call for better control room dashboards with intelligent user interfaces that facilitate the connection between operators and the computerized tools in control rooms and the utility C-suite.

Forecast Application for Smarter Grids

Smart grid technology promises to enable price signaling to control demand, not just generation. The vision for the smart grid connects customer sensitivity to price with the tools that enable customers to control demand based on price.

Managing everything from washing machines to EV chargers, smart dispatch can inject much balance and stability. Smart grid price signaling will help smart grid technology realize its promise as a value engine. Integrating forecasting and demand signaling will let grid operators execute with reduced reserve resources, lowering overall cost.

Better forecasting alone is insufficient to balance the uncertainties of highly distributed and dynamic generation, demand resources or both. Smarter dispatch systems also must enable regional transmission organizations and independent system operators to optimally clear wholesale electricity market pricing while providing system operators with a holistic, forward-thinking view of system conditions as a tool to manage uncertainties. Smart dispatch is envisioned to revolutionize resource allocation, particularly designed for operating in the smart grid environment. The smartness of this new era of dispatch can manage highly distributed and active generation and demand resources in a direct or indirect manner.

This new approach for smarter resource allocation has great potential to use EV charging energy storage for the grid, balancing the uncertainties of wind and solar generation. If the control room can determine when to charge many EV batteries, demand can be held off until there is excess generation. The storage potential of the batteries can be used to smooth out the demand and generation peaks.

In the same way, a more organized system of aggregation can help balance the extremes of distributed generation in smart cities. Aggregators accountable for managing load will make system demand more predictable.


All of the scenarios envisioned here, from growing levels of operational uncertainty and the potential for EVs’ adding new demand to water shortages and aging control room operators, could make electric supply and delivery unmanageable. The grid needs better forecasting tools as never before. The one thing we can count on is a black swan that even the best crystal ball can’t see.

That puts new pressure on the need for a broader vision that looks into seemingly unrelated fields. This improved crystal ball then will drive improvements throughout the control room and into grid operators’ C-suites. Expanding the operational planning time horizon for real-time decision-making in the control room will be critical to managing the new utility operating paradigm. More accurate, long-term forecasting will make sure the right resources are deployed, not just to manage generation but to balance demand through smart grid technologies and price signaling.

Forecasting is not a one-and-done exercise. As long as the market changes, we must continually stay ahead five years, 10 years and even 20 years. Control center operators are experts at adapting to changing conditions; however, they have yet to face the new, different kinds of emerging uncertainties.

Wind Forecasting Leads Best Practices to Integrate Uncertain Renewables

Predictability of wind, solar and other variable generation output is the key to managing uncertainty. A recent DOE global survey identifies current best practices tools and decision-making systems used to manage the uncertainties of wind and solar energy. In the same survey, an overwhelming majority of grid operators named wind power forecasting as an important prerequisite for integrating wind energy. (See Figure 1.)

According to the North American Electricity Reliability Corp. (NERC), several forecasting products are important for integrating variable energy resources. Grid operators typically use one or several of the following lists of forecasting products:

“- Day-ahead (DA) forecast
“- Wind plant high-resolution forecast
“- Weather situational awareness forecast
“- Short-term forecast (e.g., 10 minutes)
“- Ramp risk forecast
“- Six-hour forecast
“- Nodal injection forecast
“- Ramp forecasting
“- Next-hour forecast
“- Probabilistic forecast
“- Ensemble forecast (use of multiple wind forecasts).

The grid operators in the survey also identified the importance of precise prediction tools and several forecasting products to anticipate generation needs and control mechanisms. (See Figure 2.)

Grid operators overwhelmingly agree that short-term forecast (five to 10 minutes) will be increasingly important as the penetration of wind and other variable energy resources increases.


Lawrence Jones is vice president of policy and regulatory affairs of Alstom Grid Inc. He was the principal investigator of the global survey of wind integration in control centers funded by the DOE. He is a senior member of the IEEE.

Kwok Cheung is director of research and development at Alstom Grid Inc. He is a senior member of the IEEE.

More Electric Light & Power Current Issue Articles
More Electric Light & Power Archives Issue Articles

Previous articleThe Changing Role of Utility CFOs
Next articleAutomating Customer Empowerment

No posts to display