By David Christophe, Nokia
Society has grown increasingly dependent on electricity for economic prosperity and safety. As a result of this dependence, electric power has become indispensable. As utilities work to make their grids more reliable and resilient, they are increasingly turning to advanced communications network technologies to support a wide array of mission-critical applications.
These mission-critical communication networks are built to the utility industry’s stringent reliability and security requirements and use the same technologies employed by telecommunications service providers worldwide. They can play a significant role in reducing the frequency and duration of power outages in distribution grids. There are steps that must be taken, however, to ensure that these communications networks are secure, have predictable performance and can quickly reroute traffic in the event of failures.
Creating a Resilient Grid Communications Infrastructure
In the past, utilities generally relied on communications networks based on time-division multiplexing (TDM) technologies such as synchronous optical networking/synchronous digital hierarchy (SONET/SDH) to support their operations. This was due to the low latency, high availability and predictability of SONET/SDH. TDM technology, however, is neither flexible nor particularly efficient in terms of bandwidth use. For these reasons, communications service providers and others that rely on communications networks have moved away from this technology to a combination of Internet protocol (IP) and multi-protocol label switching (MPLS) technologies.
The accepted communications standard for new smart grid devices and applications is IP, which is already the technology of choice for communications networks worldwide. IP in and of itself, however, cannot meet utilities’ mission-critical performance or resiliency requirements. Applications that are essential to the safe and efficient control of the electric grid such as teleprotection and SCADA have extremely stringent reliability and performance requirements.
The key challenge is that IP alone cannot deliver the levels of predictability and reliability required by emerging smart grid applications, nor can it adequately support legacy utility services. IP coupled with MPLS technology, however, can address these requirements. The combination of IP and MPLS can support newer applications natively designed for IP networks, as well as legacy TDM-oriented applications-all on a single network. This stands in stark contrast with the approach of each application having its own dedicated network, which has historically been the case.
Of course, one of the biggest benefits that IP/MPLS brings to utilities is resiliency, most notably in two dimensions-predictability and security.
Predictability is King
On its own, IP is a non-deterministic technology, meaning the network cannot determine a route that a specific packet of data will take when it is sent across the network. Nor can it determine how long it will take the data packet to reach its destination. This is actually one of the strengths of IP, because this approach ensures the most efficient use of the available network bandwidth.
For mission-critical utility operations, however, this approach is not sufficient. MPLS, in contrast, places IP data packets into MPLS packets, which in turn, are forwarded along predetermined paths. By forwarding packets along predetermined paths, it is possible to ensure that all packets are delivered within a known delay budget. This functionality allows low-latency applications such as teleprotection to be supported over IP/MPLS networks. IP on its own can’t meet the latency requirements for these applications, but IP over MPLS can.
Security is Queen
As mentioned at the beginning of this article, modern societies are extremely dependent on electrical grids. As a result, they are more likely than other public services to be targeted for cyberattacks. Smart grids rely on real-time information and data analysis, so disruption of that information stream can be devastating. As important, grids are widely distributed, geographically speaking (more than ever with the introduction of small-scale, renewable generation sources), and so both cyber and physical threats are an unfortunate reality.
To address these challenges, the latest communications technologies geared toward the smart grid-namely IP/MPLS infrastructure and fiber optic and microwave transport networks-have built-in protections against cyberattacks. Cybersecurity capabilities such as encryption, firewalls and access control lists (ACL) are high-level approaches that utilities can use today to protect their operations and their customers.
Physical security, which remains a critical issue, is a challenge that communications networks can’t necessarily address directly. A reliable and resilient communications network, which is already being used to support operations at remote substations and other locations on the grid, however, can support capabilities such as sensors, alarms and CCTV, all of which can serve as important elements of a physical security strategy.
It’s important to keep in mind that a smart grid communications network is not a single entity; it will typically incorporate a wide area network (WAN), multiple field area networks (FAN) and neighborhood area networks. It also will, almost inevitably, involve a variety of technologies, applications and end-points (far more of these than ever before). As a result, it will be important to fully assess the myriad of vulnerabilities with an eye toward avoiding them where possible, and mitigating them quickly where not.
Embracing Risk Mitigation
A key to ensuring resiliency is analyzing the potential risks to the smart grid. Utilities need to understand the implications if those risks are exploited. Put simply, any smart grid architecture should ensure end-to-end protection of services. In practice multiple layers of protection are employed.
As important, it is essential to implement an intrusion-detection system of checkpoints including:
“- Centralized authentication and logging
“- Security policies for each service through access control lists, MAC-pinning, IP and bandwidth filters
“- Centrally managed and monitored firewall at every substation
“- Per-service firewall policy for nodes bringing services into substations via a Layer 2 virtual private network (VPN)
“- Comprehensive password protection at different levels, which allows users to be quickly isolated and locked out, if necessary.
The native capabilities of IP/MPLS support the network intelligence, flexibility and control needed to support these and other end-to-end protection features.
Security features such as these should be embedded throughout the network. Yes, it’s important to strike the right balance between ensuring security and delivering on other business objectives, including overall network performance. Encryption features that negatively impact latency, for instance, could be counter-productive.
The biggest expectation placed on utilities is that their services be available. Citizens, businesses and governments count on electricity being available for their use at all times. More important, communications for mission-critical applications need to be always on and trouble-free.
Therefore, the networks must be designed so there is no single point of failure, with sufficient redundancy that multiple points of failure can be mitigated or worked around quickly and seamlessly.
When converging a variety of traffic types over a common communications infrastructure, it is still important to logically separate different types in order to manage security, availability and reliability for each application, as each type has different requirements. MPLS supports VPNs of various types, making it easy to separate different kinds of traffic. Using this functionality, one VPN could be carrying SCADA to a control center, another supporting video cameras for security at remote locations or substations and a third for routine business services. Such logical separation also makes a network easier to operate, mitigating the risk of human error.
Perhaps one of the most critical elements when looking to improve the resiliency of the network is to know what’s happening on the network at all times. This “visibility” is supplied by thousands or even millions of devices placed between generating facilities, control rooms, residences and businesses. Sensors and intelligent electronc devices (IEDs) throughout the network can provide real-time intelligence and support automation in both substations and the distribution network, enhancing the utility’s ability to respond quickly-ideally automatically-in case of a fault, accident or attack.
Managing the Increasingly Distributed Grid
With the increasing introduction of renewable and other more intermittent sources of electricity onto energy distribution networks, real-time monitoring and control of the grid becomes increasingly important. All of these energy sources, often at the far edges of the distribution network, must be connected, monitored and managed.
To address this challenge, utilities are using FANs to extend communications to remote locations. FANs incorporate a mix of technologies, from fixed access options such as DSL or fiber, but also with a growing reliance on wireless broadband. Options are now available for operators to build their own wireless broadband networks using commercially available LTE technology operating in leased spectrum. This is an alternative to proprietary technologies in unlicensed spectrum that can be subject to disruption due to interference. It also provides the opportunity to shift traffic onto commercial mobile networks as needed, due to the commonality of private and commercial networks.
These FANs can then connect into the IP/MPLS-based WAN, which is highly reliable, fast and secure, to ensure seamless monitoring and control of smart grid devices and sensors throughout the utility’s service footprint. This enables utilities to rapidly transport the data required to manage the more complicated and variable electricity flows they are increasingly seeing. These fast and reliable communications networks are able to support the automation, applications and links to state-of-the-art storage facilities required to smoothly manage an irregular power supply and help ensure the quality of the power supply.
Resilience Depends on Communications
To ensure that their grids, which are increasingly distributed and complex, are both reliable and resilient, utility operators need a communications network that can gather data from potentially millions of sensors and other devices, analyze the resulting data streams in real-time and have systems in place to automatically address problems that occur. They also need solutions in place to address both cyber and physical security threats. Advanced communications networks- incorporating IP/MPLS, commercial LTE mobile broadband technology and optical and microwave data transport-can address all of these requirements.
David Christophe is director of Utility Solutions Marketing at Nokia, where he focuses on helping distribution utilities modernize the communications networks that support their grid operations. David has a 20 plus year background in IP/packet networking, and has spent more than a decade working with utilities to enhance their mission-critical communications networks.