Ethernet in Substations Helps SRP Build for Today and Plan for Tomorrow

By Rick Hyde, SRP Communications Engineering and Frank Madren, GarrettCom Inc.

The Salt River Project (SRP), based in Phoenix, Ariz., has a reputation as an industry leader in power reliability, system performance, power restoration and customer satisfaction. One reason for SRP’s leadership is its willingness to plan ahead. To meet the growing power needs of metropolitan Phoenix, SRP is adding generating facilities and transmission lines that will protect SRP customers from market fluctuations and power shortages. In addition to adding new facilities, SRP is currently upgrading its nearly 200 existing substations to feature Ethernet in the control system over a six-year rollout period.

Ethernet Becomes Attractive

SRP is the nation’s third-largest public power utility, providing power to more than 800,000 customers throughout a 2,900-square-mile service territory in central Arizona. Like many power utility companies, SRP has used expensive, proprietary point-to-point circuits for different access requirements at its substations: the traditional 4-wire serial lines for SCADA and telemetry, and 2-wire RJ-11 cabling for voice or dial-up modems. With the growing popularity of the Ethernet standard and Internet access, demand for access was increasing to the point where the old systems were becoming impractical.

Click here to enlarge image

As IP grew in popularity, SRP’s customers (both internal and large commercial end users) began demanding Ethernet access. SRP began deploying a SONET-based wide-area network in 1996. This network utilized about 500 miles of company-owned fiber optic cable that connected to about 180 electric substations. The manufacturer of the SONET equipment, GE Multilin/Lentronics, provided 10BaseT Ethernet cards that could be installed at each location. Vendors of various substation relay and measurement devices began to provide more and more products that were Ethernet-ready.

In the year 2000, SRP began the six-year upgrade program to install Ethernet into its 180 distribution substations to support equipment interrogation and video monitoring. All that was left for SRP to do was identify dependable Ethernet vendors who had equipment that would stand up to the utility’s reliability standards in the demanding substation environment—and develop a rollout plan.

Versions of Ethernet switches that provide extended operating temperatures, high availability and resistance to power line electromagnetic interference allow Ethernet to be extended into substations. In this environment, hardened Ethernet products offer highly reliable remote monitoring functionality, reducing truck rolls for servicing while making it possible to maintain high power availability to the public.

Rudd Substation

SRP is currently installing Ethernet in its newer transmission substations, such as the 500-kV Rudd substation on the west side of Phoenix. The Rudd Substation demonstrates that a modern power substation requires high LAN flexibility. The communication demands on a power company such as SRP require telco-quality systems to interface between the “edge” networks in the substations and the central office. Additionally, the networking equipment must be able to operate in this hostile environment.

Click here to enlarge image

While distribution substation retrofits often bundle a number of serial inputs through a port multiplexer for inexpensive access to the Ethernet switch newer applications tend to have more direct input to switches. As utilities install equipment such as digital recorders and closed-circuit TV, which make remote site management easier and less expensive, the demand for cost-effective fiber ports and modular upgrades will continue to grow.

There is another highly practical reason why a lot of devices ordered for larger substations have fiber optic interfaces. Concerns about line faults and ground potential rise make fiber optics a better cabling choice than copper, which is more susceptible to build-up of electrical charges.

The Rudd 500-kV substation is fed from a major generating station in Arizona. To support the step-down function from 500-kV to 230-kV to feed smaller substations, Rudd occupies about 80 acres of land and ultimately will support 10 bays of 230-kV load. Rudd construction plans called for a bulletproof monitoring and management system because of the vast amount of subscribers this substation supports.

The amount of fiber optic cabling in the Rudd station offered new opportunities—and posed unique challenges. Older substations had few relays equipped with fiber optic cabling. General Electric, the winning contractor for the station specified GE-UR relays, each with a fiber optic Ethernet interface. This provided SRP with a challenge and an opportunity. Suddenly, relays that had never been considered for remote interrogation were available. In addition, Ethernet connectivity was installed for security equipment such as camera and gate control. This meant a significant jump in the number of IP addresses that needed to be supported, and could have meant disaster had it not been for the control house design.

The Rudd control house is built with a raised floor for computer access. Unlike traditional control houses where cabling is dressed as best as possible within the building, at Rudd, all cables could be run under the flooring. Not only is this a much cleaner appearance, but it also makes it easy to run additional cabling or change the cabling.

The Rudd experience has given SRP valuable information on cable management and connectivity. Fiber optics has great benefit, but within the same room, an installation that is mainly fiber can be overkill. In future construction, SRP will look to a higher apportionment of 10BaseT copper for intramural connectivity, saving fiber optic for longer haul and noisier, more hostile or highly secure installations. One thing that will not go away is the increasing number of Ethernet-ready products. An effective strategy for managing cabling and connectivity is necessary to avoid chaos.

Finding an appropriate and cost-effective switch for this type of environment was critical. There were space, power, cost and reliability considerations. SRP chose GarrettCom’s Magnum 6K25 Ethernet Switch, which is hardened for hostile environments and supports modular configuration of ports. Flexibility was important in a mixed-cable environment; the substation did not have to waste unneeded ports, but could have the factory configure the correct mix of fiber optic and copper.

GarrettCom Inc., located in Fremont, Calif., designs, manufactures and markets Ethernet LAN products for industrial and telecommunications applications. It is one of the few Ethernet manufacturers to provide the dual-fed 48V DC floating power supply necessary for the high-voltage environment. Thus far, SRP has not sustained a failure of this product due to a power surge.

All control and communications cabling at the Rudd substation is routed under the raised access floor of the control house for a clean appearance and easy access.
Click here to enlarge image

All control and communications cabling at the Rudd substation is routed under the raised access floor of the control house for a clean appearance and easy access.

In addition to the dual-fed DC power sources and flexibility in mixing media all in the same unit, GarrettCom offered a product profile less than half the size of competitive products—requiring only one rack unit space to support up to 25 ports. This was an important consideration due to the spatial limitations of substations.

SRP found that the NEBS certification of the Magnum products provided an extra level of security for its high-availability application. NEBS is an acronym for “network equipment building system,” and describes the criteria deemed necessary by the telecommunications industry to support mission-critical applications that demand high reliability and no electromagnetic radiation or radio frequency interference that can interfere with other sensitive systems.

Distribution Substation Retrofit

At the same time SRP is building new 500-kV substations, it is moving forward with retrofitting its distribution substations. Installing Ethernet in the distribution substations made it possible for SRP to replace dial-up modems, which were slow, inefficient and costly. The increasing demand for modem access meant a growing number of dial-up lines were needed in each substation.

To optimize Ethernet’s benefits at the lowest possible cost, a number of serial inputs are bundled through a port multiplexer connected to an Ethernet switch. This strategy preserves the investment in older serial-equipped products, while at the same time realizing the benefits of Ethernet networking capability and making it possible to install newer Ethernet-enabled products in the future.

With more than 40 substations upgraded to date, SRP has seen significant benefits. Customers, such as internal relay, metering, and power quality monitoring personnel can sit at their desks in the central office and telnet to interrogate remote sites over high-speed lines. Ethernet benefits SRP operations by increasing communications speed and accessibility for in-house customers. It also reduces costs: One Ethernet circuit can replace many dial-up lines.

Internal customers have less need to place service queries and are relieved of the frustrations of busy modem lines, as well as their slower speed. An added benefit is that field service forces can connect their laptops into a switch at a remote substation and immediately connect to company LAN as if they had never left the office.

At this time, high-volume end customers that wish to check meter reading are still restricted to modem access for security reasons, however, as Ethernet security continues to improve, SRP expects to see this change.

Converting from serial lines to Ethernet required identifying Ethernet switches, which would be the workhorse of each substation, that offered high reliability, a dual power supply for redundancy, and configuration flexibility. The cabling of choice in the established distribution substations was copper (RJ-45). Copper is less expensive than fiber optic cable and is adequate to support the noise immunity requirements within equipment panels of the lower-voltage substations.

SRP engineers chose GarrettCom’s Magnum 4K24 Switches for the Ethernet rollout to its existing distribution substations. Like the Magnum 6K25, it offers dual-fed 48V DC floating. Because the older substations have numerous relays, PLCs, and other devices that require copper cabling, the 4K24 switches, which offer up to two built-in fiber port for extramural connectivity and up to 24 ports total (22 to 24 copper ports), are an economic choice to meet this need.

Rack configuration in a typical SRP distribution substation. Note that the compact Magnum 4K24 Switch occupies a single rack unit, which is an advantage due to the spatial constraints of the substation.
Click here to enlarge image


Next Steps

Events of the last few years have demonstrated the power industry’s need to provide plentiful, reliable electric power under increasingly demanding conditions. Power utilities are turning to networking solutions using industrial Ethernet equipment because of its high reliability and emphasis on standards, which translates into lower costs and more flexibility than earlier proprietary networking strategies.

At this time, SRP uses Ethernet to monitor equipment. SRP’s relays continue to “talk to” each other over traditional data circuits, but newer relay purchases come equipped with an Ethernet interface that allows the relay shop to remotely download data, change parameters, test and pave the way for extended Ethernet connectivity.

Ethernet’s next step in the power community—live data transmission, which is currently still in the province of expensive, proprietary, but secure four-wire serial cabling—will require the cooperation of manufacturers of high-end Ethernet equipment. Security and reliability are two major hurdles that must be managed. To have a bullet-proof communications system, which is impermeable to both hacker attack and environmental excess, the industry must supply routers than support virtual LANs and high-security firewalls. Because these routers will be installed in high-voltage substations, DC power supplies are critical, and the routers need to be software-compatible with the routers in the IT departments since the proliferation of IP addresses requires central assignment and management.

SRP anticipates Ethernet routers and security software—as well as power industry comfort level with this “new” technology’s security and reliability—to converge in two to three years. At that time one can expect to see Ethernet replacing the primary communications links for live data transfer. In the meantime, power monitoring and management upgrades using hardened Ethernet equipment are yielding major cost and convenience gains today, while simultaneously providing the groundwork for full Ethernet connectivity in power utilities tomorrow.

Rick Hyde, principal engineer for SRP Communications Engineering, received his MSEE degree from the University of Arizona. He has worked for SRP for 25 years, designing and supervising the installation of SONET wide-area networks, fiber optic cable, microwave radio and telephone systems. Rick is a member of IEEE and the WECC Telecommunications working group.

Frank Madren, president of GarrettCom, Inc., is an innovator in Ethernet solutions for office, telecommunications, and industrial applications. Under his leadership GarrettCom has developed a line of more than 100 off-the-shelf Ethernet products that address practical applications that previously required custom solutions.

Previous articlePOWERGRID_INTERNATIONAL Volume 9 Issue 3
Next articleDuke Energy announces close of 25-percent undivided interest sale in Indiana facility

No posts to display