IT Benchmarking Study Seeks Metrics

Although IT implementation and integration projects are as varied as the utilities applying them, the goals described by NSTAR and BG&E reveal the overall objectives are the same–utilities want to boost net revenues while also improving service to customers. Such common expectations indicate that performance metrics can be developed and applied across the industry.

KEMA Consulting is devising a core set of performance metrics based on its project experiences and research it is conducting with participating utilities. These metrics will serve as the baseline criteria in a multi-utility benchmarking study of IT performance and returns on investment (ROI). Experience indicates these and other performance metrics can be identified at three levels–technological, operational and financial.

Technology is a purely internal measurement of performance and efficiency. It directly impacts how well the distribution system operates and how efficiently personnel do their jobs. In many cases, it dictates how many personnel are needed to complete certain tasks. Technological benefits can be measured by asking these questions:

  • Variance between planned technology implementation cost and schedule, and actual cost and schedule.
  • By what percentage has system availability improved?
  • By how much has the cost per transaction been reduced?
  • What percentage of applicable transactions are run through the system (i.e., has the system been adopted by employees)?
  • What is the cycle time for a transaction run through the new system as compared with the old, paper-based system?

Operational benefits relate to service provided to the customer. Some aspects of operations are simple to quantify, such as outage minutes, outage frequency and restoration time. But putting a dollar figure on good or bad customer service may be impossible. One can begin to gauge operational success by asking these questions:

  • Can the utility tell the customer an outage cause and provide an accurate estimated restoration time?
  • By how many minutes have outage durations dropped and by what percentage has outage frequency decreased?
  • What percentage of the time is the utility meeting customer commitment dates?
  • What decrease has been seen in customer transactions per employee?

Financial payouts are the most difficult to assess because, as mentioned earlier, so many factors ultimately impact the bottom line that it is challenging to identify a simple correlation between specific benefits and their source. However, these questions uncover the aggregate financial benefits needed to justify utility IT projects:

  • By how much has the cost per line mile been reduced for both construction and maintenance?
  • How has the return on assets increased? What is the revenue generated per kVA capacity?
  • By how much has revenue per employee improved?
  • How many customers are served per employee?

Obviously, these questions provide only basic insight into return on investment, but they are the starting point in determining just how well–or how poorly–an IT project is performing. In its benchmarking study, KEMA Consulting will discuss these questions individually with participating utilities, examining their IT investments and comparing them using common performance metrics.

As a first step, KEMA will evaluate the return on investment measurements used by each utility. KEMA will then assist the study participants in developing the metrics that most accurately portray IT investment payoffs. Results for each utility will then be compared in confidence with averages for all participating utilities to determine how well IT investments are performing.

For more information on the KEMA Consulting ROI study, contact David Roos or Jennifer Krabbenhoeft at KEMA Consulting. Roos can be reached at (303) 324-5855 or at, and Krabbenhoeft is available at (303) 708-9355 or


Previous articlePOWERGRID_INTERNATIONAL Volume 7 Issue 2
Next articleLogica releases new enterprise application integration product for utility market

No posts to display