January 29, 2016

Don’t be fooled by vanity metrics

“It is better to look good than to feel good, and, darling, you look marvellous!” ~ Billy Crystal as ”Fernando” on Saturday Night Live

Looking good is something that is important to most; whether it is one’s personal appearance, one’s property and possessions, or the work one does, a good appearance provides a sense of quality and competence. If it looks good, it must be good, right? Well, we all know that this isn’t always the case. What lurks beneath a shiny exterior might be troubling, even disastrous. Looking good is not, then, necessarily better than feeling good.

In business, looking good often means having good statistics. In operations and maintenance, the statistics you might be reporting would be uptime, availability, efficiency, budget, or any number of other measures. While these numbers are generally based on actual performance results, it may be that the performance you are reporting is not the performance that you are seeking; that you are looking at numbers that make your operation look good while disregarding those that don’t. In the online world, this phenomenon is referred to as Vanity Metrics. The thing is, vanity metrics are not constrained to the World Wide Web.

“People with targets and jobs dependent upon meeting them will probably meet the targets – even if they have to destroy the enterprise to do it.” ~ W. Edwards Deming

Consider, if you will, the metric of maintenance cost. Maintenance cost is defined as the sum total of maintenance labor, maintenance materials, contractors used to perform maintenance work, capital maintenance, and the cost of all projects to replace worn out assets. Reducing maintenance cost would appear, on the surface, to be a worthwhile goal, and any efforts that would demonstrate a reduction in maintenance cost would appear to “look good.” If this were the measure upon which your performance review was based, and on which your compensation might depend, you would work hard to reduce maintenance costs wherever you could.

This approach, however, might not be right for the overall health of the organization. A simple way to reduce maintenance costs, for example, would be to shut equipment down. If it doesn’t run, it doesn’t need much maintenance. Such an approach, however, wouldn’t be good for the organization as a whole as the reason the company has the equipment in the first place is that it is used to produce something of worth. Shutting down the equipment may save on maintenance, but it costs the company in revenue. Looking at maintenance costs as a percentage of revenue generated provides a more useful measure, and one that can be compared year over year.

“A stitch in time saves nine.” ~ Proverb

Another approach may be to delay maintenance until the next reporting period. Certainly, the maintenance avoided this year would make this year’s metrics look good, but the following year there would be additional work to catch up on. Indeed, if equipment were allowed to deteriorate (through delayed maintenance), more serious repairs might be needed. True, these repairs would show up in the following year’s costs rather than the current year’s, but by focusing solely on saving maintenance money this year would ultimately be worse for the company in the long run.

“What gets measured gets managed.” ~ Peter Drucker

This is not to say that metrics are bad; to the contrary. Metrics are absolutely necessary in order to maintain and improve; to identify what works and what doesn’t; to ensure that changes result in positive outcomes and not negative.

THE IMPORTANT THING TO UNDERSTAND IS HOW THE ASPECT OF PERFORMANCE BEING MEASURED CONTRIBUTES TO AND REFLECTS THE OVERALL SUCCESS OF THE ORGANIZATION.

With respect to industrial reliability, there has been an evolution of metrics. Some terms, like Mean Time Between Failures (MTBF) and Mean Time To Repair (MTTR), have been in use for years and are important measures to know and understand. But as reliability has evolved, so have the metrics that we use. In the early days of reliability prior to World War II, the main effort was to decrease MTTR. Equipment ran until it broke, and getting it back up and running quickly was paramount. The worker who could get a machine back into service the fastest was a hero. What often resulted is equipment that would fail frequently but would be repaired and put back in service in short order. MTTR was what was being measured, so MTTR was what improved.

With the advent of organized reliability thinking after World War II, efforts shifted to increasing MTBF by doing preventative maintenance; replacing parts on a fixed scheduled before they wore out. This was done on the assumption that scheduled maintenance events would total less time and be less costly than repairing a machine after a total failure, an assumption that was (and is) true. A significant performance measure that developed was the percentage of preventative maintenance tasks completed on schedule, meaning that those who scheduled their work most efficiently were rewarded. Equipment downtime for scheduled maintenance is time that the equipment cannot produce, however, so while on-time preventative maintenance was working to increase MTBF, there was still a price to be paid with respect to equipment availability.

With ever-increasing pressures to do more with less, new maintenance philosophies have come about, along with new performance metrics. Manufacturers are no longer stockpiling huge amounts of raw materials or partially finished items; lean manufacturing is now the norm. The output from one stage of a process immediately is fed to the input of the next, with little to no standby work-in-progress inventory. For lean processes to work, however, each stage of the process must be as reliable as the next. Equipment cannot be taken down so that good parts can be replaced by new good parts; if a piece of equipment is to be taken down it should be done based on an identified need and not some arbitrary timetable. Equipment cannot be kept idle; it must be available to produce as much of the time as possible.

A more holistic approach to maintenance has resulted from these changes. While it was previously acceptable to look at individual pieces of equipment in isolation, it is now important to consider each piece in relationship to the overall process. Determining how a failure might occur and what effect that failure would have elsewhere allows the determination of what is critical to the process and what is not. Maintenance dollars can then be spent where they result in the greatest benefits. For all of this to work, however, the performance metrics used must be relevant to the goals the company wants to achieve.

Consider a trucking company wanting to maximize profits by improving the reliability of its fleet. Oil analysis has been proven to be an effective way of improving reliability, so the company embarks on a program to have 1000 oil samples analyzed annually. The company still changes its engine oils on a 250 run-time-hour cycle, and rebuilds or replaces its engines based on a 20,000 run-time-hour schedule. It sets performance measures based on the number of samples analyzed in a year, the percentage of engine oil changes done on time, and the number of engine overhauls completed on time. All noble goals, all measurable, and all achievable. With some effort and attention, the company can set targets, meet them, raise them, and meet them again, thus looking good to anyone who looks at the metrics. The problem is, none of the metrics address the primary goal of maximizing profits. If the company were to measure its maintenance expenditures as a function of revenues, however, it would not look at the number of oil samples analyzed but the decisions and savings arising from those samples. It could work to optimize oil drain intervals based on oil condition instead of a fixed cycle. It could rebuild or replace engines based on condition rather than solely on run-time hours, extending the useful life of the equipment. It could benchmark itself against other companies and adopt best practices, then see how those practices affect profitability.


Fluid Life’s Reliability Services Team can assist you in ensuring that your reliability metrics are linked to your company’s success; that you are optimizing your maintenance budget through oil analysis and not just analyzing samples. We can help you build a better, more robust and more effective program, with services that provide as much or as little assistance as you require. Contact a Fluid Life representative for more information about how to get real value from your oil analysis.