Introduction

This manual aims to provide a guide for water utilities on why and how to benchmark. And while presenting the key concepts and clarifying some of the industry's misconceptions is inevitable, a great effort has been made to provide a practical reference with examples and applicable knowledge obtained first hand from some of the key benchmarking initiatives in the water industry around the world.

The text reflects the work of the IWA benchmarking task group, which was created with the purpose of continuing on previous works on performance indicators and benchmarking, and producing this manual to facilitate the understanding of all those benchmarking worldwide, and provide valuable knowledge to those wanting to benchmark.

1.1 WHAT IS BENCHMARKING?

Most benchmarking books written in the past twenty years have tried to answer this question on their first pages. After all benchmarking is a complex concept and a good book should try to interest the reader from the beginning laying out the topic clearly and concisely.

This has resulted in a handful of different ways to describe benchmarking. However, the truth is that with very few exceptions, those books had an easier task than this one you are holding right now, for they were rarely addressed to the water industry. This should not be taken to mean that benchmarking is any different when applied to a water utility than to a manufacturing company. It is not. But after a decade and a half of sometimes confusing terminology, water professionals may have a harder time with benchmarking concepts than the average reader (Cabrera et al., 2009a).

One of the main aims of this manual is to try to provide clear definitions and a single benchmarking language for the water industry. A task posing a challenge even greater than describing benchmarking in a single phrase:

The careful choice of words aims to capture in a single sentence a broad concept. And key to that concept is the idea that benchmarking is simply a tool. A powerful tool that is especially suited for the water industry, but in no way an end in itself. Benchmarking without a clear goal will often lead to disappointment and waste of resources.

Another cornerstone of the benchmarking concept is captured by the word systematic. Benchmarking techniques should always be aimed at continuous improvement. As a matter of fact, benchmarking fits especially well in the PDCA (Plan, Do, Check, Act) concept and should always be approached with those four stages in mind. As a natural consequence, benchmarking should derive in any organization in a natural tendency towards continuous improvement.

However benchmarking is anything but a "do it yourself" concept. The search for leading practices implies that lessons need to be learnt from others, hopefully from the best in class. But as much as benchmarking is about looking outside, it is also an exercise of looking within and learning how things are done internally. It is only from that inner knowledge and the understanding of how others (the best) do things, that improvement can be achieved.

An obvious factor to that success is understanding the concepts of best practices and best in class. Those words are often quoted in the literature pointing to iconic case studies and world-class companies as references in certain processes. While it is true that those organizations constitute a clear reference, benchmarking is not limited to industry leaders and large organizations. And as a matter of fact, some of the benchmarking projects undertaken lately by water utilities show that lessons can be learnt from almost anyone, and that being able to identify the best practices and those who have developed them is a key factor to success.

The truth is that to find out whom the best in class is, measurements need to be made. However, and despite the very extended notion that benchmarking, or some type of benchmarking, is achieved by comparing metrics, it should be made very clear that creating a bar graph with a numerical comparison of several utilities is not benchmarking. Even the original definition of "metric benchmarking" (which will be reviewed later in this chapter) considered it to be much more than the comparison of a few indicators. The benchmarking concept has always encompassed a systematic process and the will for continuous improvement.

The consequences of continuous benchmarking efforts in a utility are often reflected in a more mature organization which is more transparent and understands better how things are done, which things need to and could be improved and what it takes to improve them.

1.2 BENCHMARKING: METRIC, PROCESS OR NONE?

Even from the early days, the water industry was quite fast in reacting to a new management tool called benchmarking. It was the early 90s, and the echoes from Harvard had already been made available to the general public in what many considered the first mainstream publication on the topic (Camp, 1989). From the very beginning, benchmarking was seen as a tool that, in the absence of efficiency stimulating incentives, could help water companies in their search for excellence.

The aim of benchmarking was to identify those who were best in their class, and once the best practices had been identified, they were to be adapted in search of better performance. Robert Camp had been working at Xerox in difficult times when the only option for the company was to learn from the best (their Japanese colleagues at Fuji-Xerox) and improve their practices.

However, those were also the days of the dawn of the Office of Water Services (OFWAT). Regulation was introduced in England and Wales in 1989 following the privatization of the water sector. Yardstick competition (the use of metrics to compare the performance of the different water companies) was soon chosen as a key regulatory tool. Such artificial competition was to compensate for the lack of a real market tension between water companies, and therefore drive them to improve performance.

It is hardly surprising, that when the AWWA Research Foundation issued a report on benchmarking (Kingdom and Knapp, 1996) the universe described for water utilities was ruled by Xerox benchmarking and the yardstick competition created by OFWAT. However, the language used to describe that universe proved to be far more controversial than the tool itself.

The AwwaRF report, well written and easily understandable, coined the terms metric and process to qualify what had simply been benchmarking until that date. However, while the text had an overwhelming success in the dissemination of these terms, the underlying concepts did not travel as far.

Metric benchmarking made reference to the comparison of key performance indicators. This was (and still is) the basis for the system used by OFWAT and by most of the water industry regulators in the world. However, metric benchmarking soon became appealing to a completely different type of user: utilities who wished to determine their competitive level by screening their performance areas for strengths and weaknesses and comparing them to the performance of other utilities.

Process was used to identify "Xerox benchmarking" as described by Camp. This method identifies an organization that is best in class in a particular process or business area; and after determining which factors are key to the success, adapts the best practices to improve performance.

The two techniques shared some similarities. They both needed the participation of several utilities. They both involved some sort of comparison and, above all, metrics were a crucial first step for Xerox benchmarking (although they should never become the main aim of the project).

However, this picture that clearly separated the concepts and techniques (as presented by Kingdom and Knapp) never made it to the general public. Both concepts have often been confused since then, and the actual words (especially "process") have led to a good number of misunderstandings as people tried to interpret what the terms implied.

And so, for a whole decade, we witnessed the appearance of dozens of papers and technical references in which the terms "metric benchmarking" and "process benchmarking" were used with different meanings, representing all shades of grey. This lack of consistency in the definitions was even present in technical working groups and international projects, and their name no longer helped to assess the nature of an effort without a thorough explanation of the work. It could be argued that no one was wrong in this story and that words are just words. But the truth is that benchmarking specialists need a common language to be able to forget about words and start working on the techniques.

1.3 A NEW BENCHMARKING FRAMEWORK

One of the main objectives of this manual from the very beginning was to clarify the difference between metric benchmarking and process benchmarking, an issue that the industry needs to leave behind. Endless hours have been devoted in the past decade to discuss whether a project was a metric, a process or even a benchmarking project at all.

The bottom line to all those discussions is that the value of a certain initiative or project does not really lie in its name, and whether it can be called a benchmarking project. Once again it must be stressed that benchmarking is only one of the many available tools, and results are much more important than the tool itself.

Unfortunately, even we soon found out that the line between metric and process is just too difficult to draw. At the time of writing this manual, too many years had spanned with many uses of both terms to allow reconstruction of their meaning. A similar situation was faced by the IWA Water Loss taskforce a decade ago (Lambert and Hirner, 2000) when they decided to discontinue the use of the term unaccounted for water. That was a successful initiative, and despite the fact that the term is still used today, the terminology proposed by both IWA and AWWA regarding water loss has become a common language shared by the water industry worldwide.

With that model in mind, we decided that this manual should be the ideal vehicle to present a new benchmarking framework.

In the following pages, you will be provided with a complete picture of benchmarking in the water industry in the world without the use of the terms metric and process.

As a matter of fact, let us clearly stress that point in the following sentence:

To better illustrate the concepts of performance assessment and improvement, Figure 1.1 maps most of the practices qualified as benchmarking in the water industry. Any current benchmarking project can be placed in the figure according to the objectives pursued, the techniques used and also taking into account the level of detail (in other words, the power of the magnifying glass or how deeply in detail is the utility being studied).

This new framework can also accomodate the regulatory activities in the industry, which are usually focused on comparative performance assessment, but strictly speaking do not constitute benchmarking. Therefore, it would be advisable to use terms like yardstick competition or comparative performance assessment for those regulatory efforts.

As a matter of fact, one of the main problems encountered to date when defining benchmarking was that different projects focused on a different level of detail within the utility (vertical axis). While some of the projects studied the utility as a whole (lower level in the figure) other projects went into greater detail and focused on the main functions that conform the core business of water utilities, the processes that compose those functions or even the elementary tasks in which those processes can be broken down.

Good examples of these different approaches can be found in current projects of both performance assessment and performance improvement. For instance, most regulators (like OFWAT in England and Wales or ERSAR in Portugal) assess water utilities at the utility level, and although there may be a specific focus in some key strategic areas, the assessment is rarely done at the function or process level. On the other hand, it is quite common for industry driven projects to focus on functions (customer service, asset management, etc.) and to go down to the process and task level in order to find answers to the "how to improve" question, for instance the benchmarking projects promoted in Australia by the suppliers' association, WSAA.

The assessment and improvement of performance at different levels of detail within a utility is perfectly logical depending on the needs and objectives of each project, and numerous successful examples can be found in the literature.

In each one of the levels of detail described in Figure 1.1 is possible to carry out a performance assessment. For instance, a funding institution will try to assess the overall utility performance or at least its overall financial or service performance. However, a plant engineer may be quite interested in both the assessment and the improvement of the performance for a single task within a very specific process (membrane cleaning in a microfiltration plant).

Additionally, a first and necessary step in benchmarking is the assessment of how efficiently or to which standard a certain utility operates, or a function, task or process is carried out. Such assessment is usually done by means of performance indicators which need to be compared to a reference to obtain the assessment (the reference can be a fixed one - e.g. a standard or target or the performance of a peer - e.g. another water utility -). We call this the comparative performance assessment phase.

Once performance has been assessed, a logical follow through is to undertake actions to improve it. And hence, by means of identifying and adapting the best practices of those who are better (or the best) at a certain function or process, it is possible to improve performance. We call this the performance improvement phase, and it requires the participation of several utilities or benchmarking partners (even from other industries) in order to gather additional information that will lead to the identification and adaptation of best practices.

These two clearly differentiated phases (performance assessment and performance improvement) are the necessary parts of benchmarking. And therefore, benchmarking can be defined as "a tool for performance improvement through systematic search and adaptation of leading practices".

This new framework can easily accommodate any project carried up to date within the water industry. As a matter of fact, a very interesting exercise consists in identifying the scope of existing projects in Figure 1.1.

For instance, in Figure 1.1, metric benchmarking projects (as defined by Kingdom and Knapp in 1996) would occupy the left half of the figure, while process benchmarking would be represented by the right half (Figure 1.2).

This division, as seen in the figure, was made irrespectively of the level of detail of the project, and therefore a vertical axis would not be necessary in order to define the technique used. In the case of the definitions created by Kingdom and Knapp, metric benchmarking would fall completely under what we call now the performance assessment phase, while process benchmarking would belong to the performance improvement stage.

Some of the confusion in the past originated in the understanding of the term "process benchmarking". The comparison of performance at process level was sometimes referred to as "process benchmarking" (Figure 1.3). This understanding of the definition was not an isolated case. However, in some of these projects, the objectives did not include performance improvement (a great difference from the original definition). The comparison of Figure 1.2 and Figure 1.3 clearly shows that those two concepts of "process benchmarking" were quite far from each other and certainly needed clarification.

Once projects are represented in this scheme, the origins of misleading terms can be easily traced and their true nature understood. For instance, most of the former metric benchmarking projects (which collected performance indicators and compared their values) were focused on performance assessment at the utility level. They would occupy the left side of the figure (performance assessment) and only the lowest bar (utility).