Eventually, most electric distribution system operators (DSO) will need to be able to accommodate more distributed energy resources (PV, batteries, EVs, etc.) than the distribution systems that they manage were designed for, if those systems were designed to handle any at all.  Once the need to accommodate moderate to high penetrations of DERs is present, and especially when future DER planning is incorporated into utility investment planning, a critical but complicating piece of the scenario is the need for the architectures and information and control technologies to support a DSO’s ability to take advantage of the flexibility and resiliency that DERs offer when they’re able to communicate with each other outside of the DSO control center. 

The ability for devices on the outer reaches of a distribution system to communicate and cooperate with each other, and make decisions independently of, but while also coordinated with, the distribution control center and within defined bounds that won’t compromise the integrity of the larger system are what make this project, and Open Field Message Bus (OpenFMB), unique.

 This article addresses the application of the Laminar Coordination Framework, publish/subscribe messaging, and the OpenFMB framework to achieve flexibility and resiliency in electric distribution systems, and how these approaches are being leveraged in a project undertaken by Duke Energy and the U.S. Department of Energy to increase flexibility and resilience. 

Coordination – the means by which decentralized elements cooperate to solve a common problem – is the key to successfully implementing distributed functionality on an electric distribution system.  Organizing the electric distribution system into logical layers to coordinate local operations such that local devices work together to solve local problems, while also coordinating with the upper layers of the system to ensure integrity, enables a legacy distribution system to deliver superior flexibility and resiliency than one that solely utilizes centralized decision making. 

Drivers for Change

Distribution utilities are watching and considering the significant changes in the way some of their peers are reacting to changes that are being driven by new technologies, changing customer expectations, and new regulatory directives.  Decarbonization targets are being set by regulators, legislators, and utilities.  Non-wires based alternatives to traditional distribution project investments are being evaluated with the goals of increasing resilience and lowering capital investment costs.  Integrated Resource Planning is clearly defining what renewable resource capabilities utilities have and highlighting constraints to further adoption. 

In this time of change, utilities are also recognizing that the underlying grid architectures used when designing their distribution systems are now constraining their ability to adopt new technologies as broadly as they, their customers, and their regulators might like. Further, they see that  traditional architectures and communications protocols can hinder their operational abilities to effectively control and coordinate new technologies, especially with high penetrations of DER.

Communications and Architecture

One element that needs to change to achieve operational targets while accommodating new technologies and increasing penetrations of DER is the logical and communications protocols that distribution systems typically have used.  Traditionally, centralized architectures are based on point-to-point communications in a hub-and-spoke architecture where each grid device is connected to and individually polled by a distribution management system (DMS) via a SCADA front-end.  In this traditional architecture, distribution system devices also receive operational control and coordination instructions only from the DMS.  The information and action flow works like this: 

  1. The DMS receives information from the grid devices,
  2.  Executes coordination and control logic, and then
  3. Sends instructions to grid devices for them to perform. 

This type of client-server architecture is adequate for predominately static and predictable historical load patterns, but runs into challenges when integrating new technologies and high penetrations of DER.  For example, if a circuit has a high penetration of PV, it can experience rapid load fluctuations as weather conditions change and impact PV generation – such as clouds passing over and obscuring direct sunlight from the PV array. 

The client-server architecture can be limited in its ability to respond to changing grid conditions quickly enough because of the time needed for grid edge devices to transmit the data to the DMS, perform coordination and control functions, and then to send instructions back to the devices so that they can make the needed changes to maintain reliability.  The geographical distances between the centralized DMS and grid edge devices exacerbate the problem and if large enough, can prohibit the control and coordination instructions from the DMS from reaching the grid device quickly enough to respond to real-time changes in the network state and avoid interruptions in service.   

Figure 1: Centralized Network vs. Distributed Network with Coordination Bus

A distributed architecture accommodates near real-time decision making at the grid edge by using a coordination bus, or message bus, and publish/subscribe messaging to enable grid devices to exchange data without the data having to make the round trip to the DMS. 

In a publish/subscribe messaging pattern (Fig. 1) grid devices publish data to the coordination bus, and other grid devices who need that data to make operational decisions can access it as it is published to the bus, rather than having to wait for the DMS to receive the grid device data, use that data to execute decision making logic, and then send control and coordination instructions to other grid devices.

By enabling data to be exchanged between grid devices locally, via the message bus, rather than having the DMS act as a clearinghouse and central decision-making point from which all grid device instructions are issued, device reaction time and distribution system performance is enhanced.  Importantly, the Central Coordinator, in this case the DMS (Fig. 1), is not removed from the flow of data but also subscribes to the message bus and receives all of the data messages needed to perform its role. 

Grid coordination is the operational alignment of assets to provide electricity delivery.  A decentralized system has multiple components that operate independently, with minimal supervision and coordination between the components.  A distributed system is a system of systems where the parts coordinate to solve a common problem.  

Figure 2: Laminar Coordination Framework – Courtesy Pacific Northwest National Lab

Figure 2 demonstrates a distributed systems architecture where local nodes, or devices, exchange data via an inter-domain communication bus.  This architecture, called the Laminar Coordination Framework, enables communication between logically federated grid devices and removes the immediate need for a central node to react and respond to local grid conditions while also remaining in the flow of data and able to make decisions for the entire system if needed. 

An Application of the Concepts

Duke Energy identified an emergency staging area used during extreme events that has been classified as critical load that would benefit from the concepts discussed in this article: distributed architecture, publish/subscribe communications messaging, and the laminar coordination framework.  The primary desired outcome of the project is to increase the resiliency of the distribution system serving the critical load by enabling local decision making by grid devices, while also successfully controlling and coordinating the system that includes variable generation from locally connected DER and by taking advantage of local battery storage.    

By implementing distributed communications and control, the critical load receives increased resiliency by enabling local devices (reclosers, switches, circuit breakers) to share information via the coordination bus and then execute one or a series of programmed switching options based on operational conditions.  Sharing information via the coordination bus and a publish/subscribe communication protocol allows the devices to act independently based on the information immediately available rather than waiting to receive commands from the centralized DMS – which increases the resiliency of the system by enabling the local devices to choose from an increased number of switching configurations based on immediate operational conditions. 

By utilizing the coordination bus and publish/subscribe messaging, intentional or failure induced changes to the system’s topology are shared with all devices via the coordination bus as system conditions change and the system is able to react and reconfigure more quickly, lessening or eliminating interruptions to electric delivery.  Additionally, the publish/subscribe protocols being utilized for OpenFMB inherently support transport layer security (TLS) and topic whitelisting to enhance the cybersecurity posture of the local and distributed data exchanges.  

Flexibility and Resiliency

The Duke Energy and U.S. Department of Energy project described in this article utilizes existing distribution system devices, rather than new equipment, to provide the desired functionality, which can dramatically reduce initial investment costs for utilities designing for flexibility and resiliency by employing this approach.  Utilizing the OpenFMB framework enables in-service devices to communicate in new ways, in this case locally with other devices while also communicating with the centralized DMS, providing new options that increase the resiliency of the distribution system. 

Utilities are searching for lower cost alternatives to a rip-and-replace approach to solve for flexibility and resiliency while also being able to incorporate increased penetrations of DER and energy storage.  The approach taken by this project achieves all of those objectives and allows a utility to accommodate its asset mix while taking a lower cost approach and using existing standards, being compatible with multiple communications protocols, providing distributed local device coordination and decision making and fitting into legacy deployment architectures. 

Disclaimer: Contributions to this project were achieved through the Grid Modernization Laboratory Consortium (GMLC), a strategic partnership between the U. S.  Department of Energy and the National Laboratories. The GMLC was established as part of the U.S. Department of Energy Grid Modernization Initiative (GMI) to accelerate the modernization of the U.S. electricity infrastructure. The views expressed in the article do not necessarily represent the views of the U.S. Department of Energy or the United States Government.

Dr. Stuart Laval will be presenting at DISTRIBUTECH International, which takes place from January 28-30 in San Antonio. Learn about his presentation here.