by Tom Helmer and James Osbourne, Black & Veatch
Smart grid technology investment has given rise to the need for true system engineering skills similar to those fostered by U.S. investment in the space program. The investment being made in intelligent infrastructure should be the nation’s man on the moon for the next decade. A system engineering approach to the smart grid, along with system engineering integration methods, ensures scalability, security and future data mining opportunities.
Management of distribution smart grid systems involves many integrated software systems and intelligent devices. For example, a smart grid solution might include more than 40 systems, 500,000 intelligent electronic device (IED) data points and 100 integrations to address the payloads that have become part of the smart grid landscape. These payloads, which highlight the complexity of this problem domain, go from .004 seconds for protection and control to tens of minutes for IED oscillograph data. Given this complexity, a system engineering approach and integration methods can ensure the system is scalable, secure and provides the ability to leverage information for future data mining activities.
SYSTEM ENGINEERING APPROACH
System engineering is an approach to manage complexity. A system engineering approach is more appropriate during smart grid system design than a power systems automation or general information technology point of view and is well-matched for supporting application-to-application (A2A), substation automation and control room integration designs. A key aspect of the system engineering approach is the decomposition of the overall system into each of its software and hardware components. Because power systems have a wealth of intelligent devices with specialized characteristics, care must be taken to account for each power system component individually and as part of operating and monitoring the power system.
System engineering integration methods can be employed to plan complex systems and serve as smart grid project accelerators. These integration methods are based on using enterprise-class integration tools and information management technologies to support a highly decoupled design with scalability, maintenance, tuning and security mechanisms. Figure 2 summarizes the system engineering integration methods covered in this article.
Most utilities are implementing modern enterprise service bus (ESB) technology for their A2A integrations. Some are attempting to use their corporate ESB for all integrations.
Employ multiple ESB domains. A key smart grid integration method is the accommodation of multiple ESB domains to communicate across the operation center, enterprise and substation (see Figure 2). This approach brings with it all the native ESB advantages for A2A integrations and an in-depth security approach to supporting integrations with systems that interact with the power system. The approach has its challenges; however, extending the use of a single corporate ESB will not address adequately all needs of a utility’s smart grid environment. Messaging and orchestrating jobs (extract transform load, secure FTP, etc.) across domains enables more effective auditing and logging capabilities to support regulatory requirements and smart grid cybersecurity standards.
SUBSTATION AUTOMATION INTEGRATION
Utilities typically design their substation automation schemes to support operations within the substation local-area network (LAN) and for remote monitoring and control via supervisory control and data acquisition (SCADA). IEDs play a critical part in the life cycle asset management of power system devices based on the functionality of newer IEDs. For example, IEDs can be configured to detect potential failures of power transformers, circuit breakers and other equipment. This benefit can be significant with older equipment at critical locations on the system where failure can be costly. IEDs also can be used by software to provide useful analysis to warn the user of undesirable events and operating conditions. By integrating the appropriate information with the utility’s SCADA system and smart grid data repository, the utility can gain the most value from deploying these new IEDs inside its substation.
Support operation of power system via substation automation/SCADA and substation automation/enterprise integrations. Smart grid solution architecture should support operation of the power system via substation automation/SCADA integrations and substation automation/enterprise integrations supporting asset management and trouble shooting of missed events. Substation automation operational information should go from the substation to the control room via SCADA, and the substation automation nonoperational information should go from the substation to the enterprise via a different communication infrastructure designed to support the nonoperational payloads and security requirements.
Add a data concentrator or data gateway device for dual-path communications. It is recommended to establish a new data concentrator or data gateway device in the substation that supports current cybersecurity software for intrusion detection and supports dual paths to communicate with substation devices on the substation LAN. The gateway device should be the point of demarcation for security zones. This allows the utility to leverage all valuable nonoperational data being collected by the IEDs as they come online by storing all of their history information in the enterprise smart grid data repository that can be mined by the utility’s asset management programs. This nonoperational path to the substation data concentrator should be used to manage the login and passwords and the software patches for the substation IEDs, as well.
Use enterprise-class technologies to implement NIST recommendations. For substation to enterprise integration, use enterprise-class technologies to manage authentication and publish software patches for new IED devices in accordance with National Institute of Standards Interagency Report (NISTIR) 7628. This brings the power of enterprise ESB technology to support the auditing and logging requirements being specified by newer versions of NISTR, as well as building a natural security in-depth approach as recommended in NISTR.
Coordinate the substation automation devices with centralized DMS. A recommended integration method involves coordination of the substation automation devices with the control room distribution management system (DMS) and distribution SCADA (DSCADA). The latest substation automation devices contain advancements in processing power and logic to aid in the management and operation of the power system grid. Unfortunately, many substation automation vendors lose sight of the value of this information beyond the substation. Errors and failed operations information tend to reside in the substation automation master with limited access. Configuring the substation automation masters to report errors and failed corrective operations back to the DMS/DSCADA gives the system operator visibility into the automation schemes. This integration method ensures the DMS/DSCADA intelligent applications respect and honor the substation automation autonomous schemes before attempting to heal the grid. In addition, access to this information enables the utility to replay events of major outages and system disturbances as part of future dispatch training.
A key requirement is to support substation automation protection and control-type of operations outside of the normal substation LAN, such as monitoring and controlling distributed generation/distributed energy resources (DER). Distributed generation/DER technologies require alternative substation functionality that supports transfer/trip operations. These alternatives come with their own challenges mainly pertaining to communication and latency. Utilities are field-testing new wireless technologies–WiMAX–and private high-bandwidth (40+ Mbps), low-latency (<10ms) application-based quality-of-service (QoS) backhaul mesh grids.
The control room is a critical part of any utility’s operations center. Control room staff must have information readily available to make timely, accurate operational decisions.
Use the appropriate control room standard for the right messaging payload. A key integration method for integrating control systems is use of the appropriate control room standard for messaging payload. Efficiency is critical, so selecting a hybrid of integration technologies is warranted for smart grid architecture. Use an appropriate, well-tested integration mechanism with a proven ability to support the control room.
Pick one system of record for the current operating state of the power system. A second key integration method for the control room is to establish one system of record for the current operating state. This requires that the best-of-breed solutions push all changes from the chosen system of record–DMS recommended–to other systems that require the information, such as an outage management system (OMS). Standard control room integration mechanisms such as Inter-Control Center Communications Protocol (ICCP) support device changes and tagging. The more challenging integration involves propagating the temporary model edits such as linecuts and jumpers from the DMS to OMS automatically. A pragmatic approach to low-frequency types of temporary model edits is to send a notification automatically. By adopting this approach, the operator is required to reapply the temporary edit in the OMS to ensure accurate outage statistics. This complexity is not experienced by utilities that deploy combined DMS/OMS solutions from vendors who have a single as-operating network model.
Support zero data latency for the operational model of the power system. The final integration method for the new DMS smart grid applications is to support zero data latency for the operational model of the power system. Operational model integrity is the keystone to supporting the myriad of operational benefits that can be gleaned from new systems. This integrity is achieved through zero-latency model updating. Information about planned and current state must be available in real time and must be switched into service in concert with operational changes. Integration with the GIS and the field force automation system are keys to supporting an up-to-date DMS model of the power system.
SMART GRID DATA REPOSITORY
A smart grid data repository should be established to capture the nonoperational data continuously via a logically independent communications path between the substation and the enterprise, including the scalar and waveforms generated by the IEDs. The smart grid data repository should be used to manage at the enterprise the SCADA historian time series information. The smart grid data repository also should be used to manage all the historical AMI interval meter information and meter events.
Set up a smart grid data repository. The repository should support an enterprise copy of the SCADA historian (time series) information along with long-term repository for AMI meter interval and meter event information. Figure 4 illustrates notionally what should be collected and managed by the data repository under this integration method.
Current big data technologies support the management of AMI meters, IED time series and IED nonoperational asset management scalar information; however, it might fall short when considered for a smart grid data repository. These technologies fail to support the required protocols, data structures and metadata related to data concentrators inside the substation or collect the IED nonoperational waveforms, power quality graphs, sequence of event recorders, digital fault recorders (DFRs) and phasor measurement units (PMUs). This limitation hinders the utility’s ability to effectively perform missed event root-cause analysis, power quality studies, sequence of events mining, and–with some of the newer signal processing technologies–predicted device and cable failures.
Given the complexity in utilities’ managing distribution smart grid systems, systems engineering approach and integration methods can ensure systems are scalable, secure and provide the ability to leverage information for future data mining activities. Utilities should support an in-depth security paradigm through multiple ESB domains and use the auditing and logging functionality of the ESB. Substation automation data concentrators/data gateways are key architectural components for security and dual-access paths into the substation LAN. Centralized DMS/DSCADA applications must coordinate with and respect substation automation and DA autonomous schemes. The substation automation solution should report errors and remediation failures back to the DMS/DSCADA system. Acknowledge that the field work force is a key factor to maintaining a zero-latency power system model in your DMS. Finally, smart grid design should include a smart grid data repository that incorporates operational and asset utilization payloads from devices such as IEDs, meters, DFRs and PMUs.
Tom Helmer is an executive consultant with Black & Veatch. He has 30 years of experience in solution architecture and systems integration and specializes in smart grid and pipeline integrity. His background includes a tenure as a system/software engineer at Hughes Aircraft Co. Reach him at email@example.com.
James Osborne is a principal consultant with Black and Veatch. He has more than 20 years of experience in utility and telecommunication industries and specializes in GIS, AMS, OMS, WMS and MDM. Reach him at firstname.lastname@example.org.