Control in motion4 September 2002
From the earliest mechanical governors to today's microprocessor-controlled systems, much has changed in the way hydroelectric power plants are controlled, yet much remains the same
AMOS Woodward did not invent the water wheel governor, but he had an inventor's heart and saw much that needed to be improved in the governors in use in the 1860s. He received a patent in 1870 for Improvement in Water-Governor that marked the beginning of a small company that would have a profound effect on hydro turbine controls over the next century. Thirty years later, Woodward Governor Company had become one of the pre-eminent suppliers of governors to the emerging hydroelectric industry in the US. The industry itself was undergoing profound change. Old-fashioned water wheels driving mechanical loads gave way to full-fledged hydro turbines coupled to a new device produced by Edison and Westinghouse: the AC generator. By 1911, Amos' son Elmer had become chief engineer and rolled out his masterwork: the oil pressure governor, which evolved into what is now known as the Gateshaft governor.
Perhaps the most common hydro governor in the world today the Woodward Gateshaft governor is now as it was then: 'a simple, strong and durable' device capable of 'accuracy and smoothness in the control of load changes'. Designed to last 100 years, thousands of Gateshaft governors remain in use today, still functioning and largely unchanged (aside from normal wearing components) after nearly a century of service.
Gateshaft governors - and the larger mechanical and analog governors that followed - share the same fundamental features found in the most advanced computerised control systems today. They sense speed quickly and precisely, control gate position exactly via a closed-loop feedback system, and provide a means (droop) for operating multiple generators in parallel with stability. Although many advances in hydro control theory were made in later years, these fundamental qualities that separate a governor from a simple speed control remain today.
The boom years
The demand for electricity in the US during the first half of the twentieth century grew explosively. Since hydro provided not only power generation but flood control and irrigation as well, Congress approved and funded the construction of many large dams throughout the Western US: Hoover, Bonneville, and the largest undertaking of all, Grand Coulee. As hydro turbines grew larger, so did the size of the hydraulic servomotors controlling the opening and closing of the massive turbine wicket gates. This required greater amounts of governor force and oil flow, which required larger governors and oil pressure systems. Whereas early mechanical governors could fit in the trunk of a car, governors powerful enough to control these new turbines required large cabinet actuators the size of small garages, delivered on rail cars, with large pressure vessels parked nearby.
Plants had also grown considerably in the number of units per power house. Where once there were only a few units, now there were five, ten or as many as 27 in a single power house. Remote control mechanisms were added to the governors, enabling them to be controlled from a central control room. Likewise, the central control room concept required auxiliary systems, such as cooling water and turbine lubricating oil, to be automated. The use of relay logic was implemented, permitting semi-automatic starting of units.
Governor manufacturers were tasked to support plant automation, and they responded in creative ways. DC motors were added to mechanical governors to allow the speed adjust and gate limit controls to be driven by remote raise and lower signals. Multiple speed and gate position switches, driven by mechanical cams connected to governor restoring shafts, provided discrete signalling of turbine status during starting and stopping. Generator air brake valves became standard equipment on large cabinet actuators. Selsyn transmitters and receivers were provided to relay vital unit operational data up to the control room. By the 1950s it was no longer good enough to precisely control speed/frequency; automation required direct megawatt control to simplify remote setting of power output.
By the 1970s, even this wasn't good enough. To effectively control power flow in their transmission systems, utilities needed remote control of entire power plants from dispatch centres located hundreds of kilometres away. Thus, the supervisory control and data acquisition (SCADA) industry was born. Governor manufacturers responded in kind with the introduction of the analog electronic governor. Endowed with multiple fail-safe operating modes and capable of receiving direct setpoints instead of (slowly) responding to the rotation of a gear element, the analog electric governor quickly gained favour.
Nevertheless, most new advances were layered upon the old. Typically, a SCADA master station sent signals to a slave remote terminal unit in the plant, which interacted with the existing plant relay logic that sent raise/lower pulses to the governor (there were still thousands of mechanical governors still in service, as now). Fully-automatic plants from this period used more than 40 discrete relays to control governor, exciter and turbine/generator auxiliary equipment. Impressive when it worked, difficult to troubleshoot when it didn't. At the heart of it all remained, for most plants, a mechanical governor, which was seen as a benefit to most plant personnel. They knew that if automation failed, they could go down to the governors and operate them manually.
The digital age
In the early 1980s, nearly all of the original governor manufacturers were long gone: Leffel, Pelton, Lombard, I.P. Morris. Even Allis-Chalmers, a large US turbine manufacturer who had made a fair amount of its own governors was out of business, having been sold to Voith. It was in this period that a totally new form of control blazed on the scene and turned the governor market on its ear: digital computers. This was truly a paradigm shift but Woodward was slow to notice it. Small upstart companies, using their own microprocessor designs or buying off-the-shelf industrial controllers, leaped into the market and quickly gained a foothold in the burgeoning small hydro market.
Developers of small hydro projects had small budgets and couldn't afford to pay the salaries of the large operations and maintenance (O&M) staff found at large hydro power plants. This financial reality coincided nicely with the digital controller's virtues. With an integrated digital control system, you could not only govern a unit but control its generator voltage, automatically synchronise it, and then control pond level while on-line (most small plants were strictly run-of-river). All for half the cost of a cabinet actuator. Digital controls monitored the myriad alarms and safety systems in the plant, and communicated with headquarters in real-time, allowing one operator to remotely operate multiple plants safely. Embracing this new technology, consulting engineers began designing unattended hydro plants.
At first, utilities scoffed at the notion of replacing their tried-and-true governors with a small computer. Computers at that time were better known for billing foul-ups, frequent operating system crashes, and simple games. Why trust them in a mission-critical application like controlling a massive hydroelectric turbine? As in many fields, though, early adopters paved the way for what is now commonplace. Now, all new power plants use digital controllers for everything from governing to river management, linked directly to utility SCADA systems, information technology systems, and even the internet.
Changing O&M concepts
The US had been accustomed to utility regulation, which turned out to be a double-edged sword. Utilities had operated their power plants with an almost fiduciary duty. Their job was to keep the lights on for the nation, and they took preventive maintenance very seriously. Minor overhauls were performed annually, with major overhauls occurring every five years (or sooner, if the situation warranted it).
Deregulation - even the threat of it - changed everything in the utility world. When proponents dangled the chimerical concept that deregulation would reduce everyone's electricity rates, customers demanded the break-up of the utility monopolies. The utilities, fearing younger and nimbler competitors, began slashing O&M staff to reduce their cost of generation. Some utilities switched overnight from preventative maintenance to a new concept called reliability-centered maintenance, which, loosely corresponds to the old adage: 'If it's not broke, don't fix it'. Trouble was, nobody knew how to predict when it would break.
What does any of this have to do with the governor? In hundreds of plants throughout the US, mechanical and analog governors had been operating with few problems for decades. Unlike digital controls, these governors can still limp along even when they are extremely worn or out of adjustment. Even if a governor is sloppy and can't control speed, the operator can always just walk up to it, operate it manually to get the unit on-line, and then just 'block load' the unit.
Digital governors, on the other hand, do not come with a mechanical manual control. They require the CPU, the I/O modules and most of the sensors functional in order to start and run a unit, even in manual. If specified, they can be provided with redundant sensors and I/O modules, and enhanced logic to automatically detect failed or failing sensors. All these features must be specified clearly or chances are they will not be included in the delivered system. Without these, when some part of the digital control system has a serious problem, somebody's going to have to go up to the plant - quickly - to replace parts and attempt to restart the system. In the deregulated world, a missed start when a unit is needed (e.g. its power has been committed) can have dire financial consequences.
Likewise, a missed start at a remote power plant that has mechanical or analog governors is a big problem, too, but the solution is different. The main culprits of missed starts are more humble: worn governor components, dirty oil, lack of calibration, or a combination of all three over time. These can be easily addressed by performing routine maintenance. The problem is: whereas governors used to get annual maintenance, now they are required to run for many years without significant (or any) maintenance.
Pacific Gas and Electric (PG&E) has a large number of mechanical and analog governors, spread throughout the Sierra Nevada mountains, and struggled with this problem. Many of their plants were unattended, and access to them was difficult and time-consuming, even in good weather. Due to retirements and downsizing, even finding qualified governor mechanics to send up was a challenge. Yet, upgrading to digital was expensive.
PG&E, in conjunction with Stevens Point Consulting, developed a creative solution. They implemented an annual calibration and on-Line tune-up programme that allowed PG&E to test governor health and analyse performance on-line, without taking an outage. If a governor passed the test, it was 'good to go' for another year. If it failed the test, procedures were employed to isolate the cause of the problem while the unit remained on-line.
Despite the trend toward digital control, mechanical and analog governors will continue to be used for many years. Customers with these types of governors need to develop plans for how they will service their old governors. In spite of pressure from vendors to upgrade to digital, many plant mechanics understand these governors and want to work on them. Yet, getting parts from large OEM vendors can be frustrating: parts take too long to get (when you can get them) and cost too much. Luckily, small companies comprised of former OEM employees are rising to the occasion, providing new and used governor parts, field service, training and calibration services.
Digital controls, on the other hand, have support problems of their own. The 'shelf life' of a digital controller is typically between five and ten years. Computer technology is simply moving too fast for any vendor to support a particular platform longer than this. When a control platform is retired, getting replacement parts for these units can rapidly become impossible. Unless you want to replace the 'digital brain' every five to ten years, you will need to invest in a healthy stock of spare parts right from the beginning.