Applying the top 10 rules

1. Measure importance as well as satisfaction

Since customer satisfaction is a measure of the extent to which an organisation has met its customers’ requirements, you can’t just ask customers how satisfied they are. You also have to measure the first half of the equation, the customers’ requirements and their relative importance.

Many companies ask the wrong questions because they base their survey on the ‘lens of the organisation’, filling the questionnaire with the questions they want to ask. But is satisfaction is about the extent to which you are meeting the customers’ requirements, you will produce an accurate measure only if the questionnaire is based on those precise requirements on which customers judge you. The only way to achieve this ‘lens of the customer’ approach is to conduct a preliminary exploratory research exercise to fully understand what’s important to customers, basing the survey questionnaire on customers’ most important requirements.

Many rating scales are suitable for different kinds of market research but customer satisfaction measurement is different. Due to the need to monitor over time a measure that will change only slowly, the commonly used 5-point verbal scale is not suitable. Only a 10-point numerical rating scale is adequate.

For a measure that is sufficiently accurate to track over time, a sample of at least 200 responses is necessary. The reliability of small samples will be so volatile that they cannot possibly indicate whether the organisation has made a real improvement. Companies with a relative small customer base will therefore usually conduct a census survey. This is not necessary for those with a few hundred customers or more, but to avoid bias, any sampling must be completely random.

Many organisations take important decisions on the results of a customer satisfaction survey with a response rate below 20%, even below 10%, yet just a moment’s thought would demonstrate the folly of this approach. With a 10% response rate, the small minority of customers that responded are just not the same as the 90% that didn’t take part.
Tests show that they will tend to be much closer to the extremes of the normal distribution curve, a phenomenon known as ‘non-response bias’. Many of these respondents are habitual complainers and others are often ‘bad-fit’ customers – people whose needs are not met by what your company does.
Making decisions on this information means you would be trying to make the company better for the kind of customers you would be better off without, probably making things worse for the customers you most want to have! Scary stuff.

It sounds obvious but organisations often intimidate their customers when presenting them with a customer satisfaction survey. Sometimes, as British Gas did with my 82 year old mother recently, the burley engineers stand over the customer whilst she is filling in the questionnaire. That’s a great way to get the information your employees want you to have! But any customer satisfaction survey that is not anonymous and confidential will suffer from the same problem. Most people will be reluctant to criticise anyone they will have to deal with in the future if that information will be traceable to them.

“Was the customer service representative friendly and helpful?” She may have been extremely friendly and no help whatsoever. They often are! Double questions can be impossible to answer and are often unactionable.

Customer satisfaction measurement serves two main purposes. First, it provides a benchmark for organisations to monitor to judge if they are improving. Since customer satisfaction tends to improve very gradually, this headline measure has to be sufficiently robust to detect small movements and not prone to volatile swings that don’t reflect real changes in satisfaction.
The worst option on both counts is a headline measure based on a single overall satisfaction question scored on a 5-point verbal scale. To eliminate the random measurement error that affects all survey questions, the headline measure must be a composite index produced from the 15 to 20 customer requirements that typically comprise a customer satisfaction questionnaire.
Moreover since customers base their satisfaction judgements more heavily on what’s most important to them, the index should be weighted according to the relative importance of the requirements.

The second and obvious purpose of customer satisfaction surveys is to help companies get better. To meet this objective, the survey results must clearly signpost the satisfaction improvement opportunities that would provide the company with the best return on investment. This won’t necessarily be the areas with the lowest satisfaction scores.
Since satisfaction is about the extent to which the customers’ requirements are being met, you need to focus on ‘satisfaction gaps’, requirements that are very important to the customer, but where satisfaction is low. These are identified by subtracting the satisfaction score for each requirement from its importance score. It is also well established that improving customer satisfaction is much more effectively achieved by focusing efforts and resources to make a big difference to just one large satisfaction gap than by making lots of small improvements across the board.

Customers don’t always notice the improvements made by organisations so it pays to tell them. Even better if you can link improvements to what customers told you when you did the customer survey. That shows you’re really customer-focused. It also shows that it’s worth taking part in the survey because it does make a difference.
When asking customers to take part in the survey you should therefore inform them that they will receive feedback on the results, and after the survey you must provide it. Feedback could be in the form of a specially printed leaflet, a piece in a customer newsletter or magazine, a page on the website, point of sale posters or any other suitable medium.
Whatever the format it should tell customers three things. Firstly, how the survey was done, especially demonstrating that it didn’t break any of the rules. The questions were based on customers’ requirements, the sample was random and representative, the survey was anonymous and confidential. Secondly tell them the results and be honest. Thirdly tell them the key lessons you have learned and what action you’ll be taking to improve things.