Manitoba Hydro has submitted a long, long, long report by their hired guns, KPMG, to the Public Utilities Board in their latest effort to undermine the New York consultant who blew the whistle on the shaky, scary way that Hydro is really run.
Which certainly wasn't "Facts Vindicate Whistleblower", the headline that anyone who actually read the report would agree is an accurate reflection of the contents.
Manitoba Hydro is not being mismanaged, said KPMG, to wild huzzahs from Manitoba Hydro managers. KPMG had been hired to repudiate the whistleblower's allegations -- and they did the job, to Hydro's satisfaction.
The New York Consultant went to the Manitoba government 19 months ago with a variety of concerns about the way Hydro was being run. The corporation, said the whistleblower, was making multi-multi-billion dollar decisions based on data spun out of outdated computers using outmoded software analyzed by a tiny team of good ol' boys. Their "expertise" had already cost Manitoba $1 billion in unnecessary expenses, declared the consultant, and Hydro was risking bankruptcy---after plunging Manitobans into unnecessary blackouts and brownouts. Scary stuff indeed.
KPMG spent more than 100 pages discussing the computer modelling that went into decisions to spend $18 billion on a series of dams to provide power to sell into the U.S. over the next 30 years.
HERMES is the main tool used to support operations scheduling. Modules within HERMES represent the MH system in a significant amount of detail. These modules have been developed and regularly updated over many years and reflect extensive work to calibrate model outputs to actual system performance and thereby continually validate the model.
SPLASH is also used to support financial forecasting.
That's like asking your best friend if the movie he saw last night was any good. "Ehh. It's okay," he says. Okay? Hardly a resounding endorsement. Sort of like... reasonable.
We didn't have to dig deep to find out why KMPG was so lukewarm.
- "We did not undertake an audit of the models or verify their computational accuracy."
- In fact, KPMG suggested that Manitoba Hydro, maybe, might think about, if they have time, backtesting their models.
"In addition to the current validation procedures used for HERMES and SPLASH, MH should consider incorporating backtesting practices to validate its models."
Backtesting---that's when you put in known data and see if the computers spit out something close to the already known answer.
To you, that might seem obvious, but to the geniuses running Manitoba Hydro, not so much. Although somebody did suggest it before.
"Backtesting is a means by which errors in the inputs can be removed in order to verify the appropriateness of the model logic. The Consultant asserts that MH does not back test its HERMES or SPLASH models. Accordingly, the Consultant argues that management decisions and reports based on the outputs of these two models may be flawed." (KPMG Report)
- KPMG also alerted the PUB to the fact that Manitoba Hydro is one good flu away from chaos.
"As HERMES and SPLASH are in-house models and operated by a small group of highly skilled modelers, Manitoba Hydro should provide more formal documentation of the models to preserve their proprietary information and assist new modelers. This will require dedicated additional resources to develop the documentation, but doing so will help mitigate risk in the event of staff turnover."
• "MH would benefit from more formal documentation and oversight of the modeling process. However, MH must also be aware that such documentation and oversight will require additional resources without necessarily producing an immediate financial return in terms of improved forecasting performance."
Good luck finding the answer by mucking through 162 pages of gobbledygook intersperced with long sections of numbers censored by Hydro and commentary censored by the New York consultant.
KPMG says Hydro is so consumed with the fear of drought that it's extremely conservative in its decisions on how much power to sell the Americans. Since taking a bath in the last big drought in the 2004, they include "curtailment" provisions in sales contracts which allow Hydro to stop sending power to the U.S. if it's needed at home. Hydro even has contracts to buy power from the U.S. which is sold to American customers (if we've deciphered this properly):
"Based on the information presented above, we see no evidence that MH is over committing its firm dependable energy production through the proposed export contracts is and thereby unnecessarily exposing MH to volume risk."
"13. For example. PUB order 32/09 states on page 27 (of 48) that “Dependable hydraulic generation for the year 2003/04 was 18500 GWh that being a level significantly below the 21 000 GWh on which MH bases its potential for firm export contracts (after fullfilling the domestic requirement).
Yet MH has not lowered the dependable resource level to 18,500 GWh; rather MH now defines the dependable resource as a multi year historical event (not a one year, event) This effectively rneans that once every fifteen years (the deemed frequency of the 2003/04 drought event) MH will be faced with dependable energy shortfalls comparable to 2003-04 though perhaps in an environment of much higher import prices.
MH has not adequately demonstrated that the Corporation's mean energy forecast adequately reflects this self-imposed additional risk”
In a nutshell, when caught saying it had almost 14 percent more power to sell than it actually produced, Hydro simply redefined how much it needed to set aside for customers in Manitoba.
There, doesn't that leave you reassured?