And the survey says...customer behavior can't always be predicted

Statisticians, beware. Customer service surveys are poor predictors of repeat busines. Here's how to improve the odds.

Twenty years ago a few hotels and restaurants used comment cards. Only a couple of consumer products manufacturers did surveys. Most companies did not bother to measure customer satisfaction. Other more direct measures, such as sales and repeat business, were more meaningful because they provided "hard" data on customer buying behavior.

Today it is a different story. Every organization you do business with at work or home has a customer satisfaction survey to fill out. Your car dealer, dentist, gardener, child's school, and even the dreaded DMV all want your feedback.

This frenzy of measuring customer satisfaction via surveys has led to the unprecedented growth of firms such as J.D. Power and Associates and Walker. Companies pay big money for these prestigious firms to conduct surveys and provide an analysis.

The problem is, most of this survey data does not do a good job of predicting customer loyalty and future purchasing behavior.

Gauging performance

A tachometer is a gauge found on the dashboard of most cars. The needle on the tachometer moves all over the dial as a driver accelerates or decelerates. The tachometer is a sensitive instrument designed to be in direct sight of the driver whenever he or she looks at the dashboard.

The problem is, a tachometer does not provide the data needed to be a good driver. I know people who have been driving for more than 20 years, yet have no idea what it measures. They have learned to ignore it because it does not provide the information needed to be a good driver. Even people who drive standard transmissions don't pay much attention to the tachometer-they shift based upon speed, sound, and other factors.

Customer survey data are like a tachometer for many organizations. The survey data show movement from month to month, but the data often do little to help the company better manage its performance. Customer survey data are even more dangerous than a tachometer because they sometimes tell a company they are doing a good job when they are not.

Recently I heard an executive from one of the major car companies talking about customer satisfaction data at a conference board meeting. He explained that the company had spent 15 years focusing on customer satisfaction data. They had even linked executive compensation to the data, and car dealer allocations were partially based on customer satisfaction scores.

According to the data collected by the company and J.D. Power and Associates, customer satisfaction had improved steadily over the last ten years. Indeed, levels show that the vast majority of customers rate them a six or seven on a sevenpoint scale. The problem is that owner loyalty data shows flat performance at a little better than 40 percent over ten years. This is about the same level of loyalty that the other major car companies receive from their customers. Customer satisfaction data is not a good predictive measure for this company because it is not correlated with buying behavior.

Problems with customer satisfaction surveys

Generally, only two types of people fill out customer satisfaction surveys: the bored, who are excited to get some mail, and the angry. Thus, a company gets a view of its performance based on an unrepresentative sample of data. Another problem with survey data is that customers often receive the survey days or weeks after their experience with a company's products or services. For example, a major hotel chain recently sent me a J.D. Power and Associates survey asking me about my stay five weeks earlier. I travel more than 200 days a year and often can't remember even the name of the hotel I stayed in last week. Needless to say, I didn't bother filling out the survey. The biggest problem with customer surveys is that they don't do a good job of predicting buying behavior. Opinions and perceptions often indicate future behavioral tendencies, but surveys are one of the worst ways of gathering these sorts of data.

Relying on hard measures

Some companies don't even bother much with surveys or customer opinion data. Lexus concentrates mostly on car owner loyalty data, and they have one of the highest levels of owner loyalty. Increased customer satisfaction scores do not put one dollar in your bank account unless they lead to buying behavior, or at least referrals. Measures such as repeat business, loyalty, and increased spending are the important "hard" measures of customer satisfaction. These measures are actually correlated with financial results.

The problem with only relying on these hard measures is that they are "heart attack" metrics. By the time you realize you have a problem with a measure such as owner loyalty, your company may be close to death. Leading companies also track "cholesterol" metrics that help predict future problems in output measures such as loyalty and repeat business.

Some alternative approaches

1) Phone surveys. Although all surveys have their problems, phone surveys tend to be better than those sent through the mail because you get a random sample of customers, and you can probe for details to support quantitative ratings. At Saturn, employees themselves conduct the surveys. This makes the employees more accountable for the quality of the cars they build, and helps build a stronger relationship with Saturn customers-they may prefer to talk to the person who helped build the car they drive.

2) Focus groups. IBM makes extensive use of customer focus groups or workshops, where they gather detailed data on customer perceptions about their products, services, pricing, and responsiveness. And another company gives focus group participants remove control units to get individual data on rating questions. Each participant secretly responds to each rating question. The distribution of the data is then presented immediately and discussed to add detail to the quantitative data.

3) Operational metrics. Customer opinion data can only be collected every so often or you risk driving customers nuts. FedEx, for instance, doesn't rely much on periodic surveys. One of their cholesterol metrics is their service quality index or SQI. (FedEx employees also refer to this as the OSI, or "Oh, sh-e' index.) Over the years, they have compiled a list of all the things they could do to aggravate a customer, and use customer feedback to rate these things from mild aggravation to major problem. For example, losing a package might be one of the worst things FedEx could do. The SQI is measured and reviewed every day. FedEx has found that the SQI is highly correlated with future customer buying behavior.

4) Relationship level Another way of measuring customer satisfaction is to measure the level and strength of the relationship with each account or customer. Ciba Specialty Chemicals, whose U.S. Colors Division is located in High Point, North Carolina, supplies coloring agents to the big car companies, as well as other industries. The company measures the relationship with each account on a 1-to-10 scale. Using a marriage analogy, a "1" customer is someone who is already cheating on Ciba, someone they would not mind losing to a competitor. A "10" customer is someone they have partnered with for life and has the highest degree of loyalty. The company has found that the level of loyalty is a good way of predicting future buying behavior. The metric has also taught them that it is not important to build strong loyalty from every customer-only the most important ones.

Creativity is the key

The key to coming up with a good approach to customer satisfaction is to consider unusual and creative approaches. Leading companies, such as the ones mentioned here, have implemented creative approaches to gathering data on customer satisfaction. The bottom line is that if customer satisfaction data do not predict healthy Financial results, they are useless in helping you to better manage your organization.

Even worse, customer satisfaction data may not reveal a growing problem that only shows up when customers provide feedback by buying from one of your competitors. Most satisfied customers eventually leave, so it is critical to come up with better ways of gauging their feelings than the ubiquitous survey.

Mark Graham Brown is the author of two best-selling books on the Baldrige Award criteria, as well as his most recent book, Keeping Score: Using the Right Metrics

to Drive World-Class Performance (Productivity, Inc., 1996).

You may reach Brown at 310-376-2836. Hear Brown speak at Performance 2001, AQP's 22nd annual Spring Conference and Resource Mart, March 27-29, 2000 in Orlando, Florida.

Copyright Association for Quality and Participation Mar/Apr 2000
Provided by ProQuest Information and Learning Company. All rights Reserved