Frustrate Your Employees With This Destructive Survey Practice
The Net Promoter Score is the most egregious survey practice in most organizations. The NPS violates the foundations of psychological measurement and the misinterpreted results are devastating to employees whose salary, promotions and other rewards are based on the outcomes.
A colleague asked my opinion on the Net Promoter Score (thanks Jodi!), so here you go… The following is an excerpt from The Survey Playbook (pp. 88-90):
“…. As currently used, the NPS is flawed and the results misleading because it commits several of the errors described earlier including failing to define every anchor (Lesson #10) and creating ambiguous categories out of ordinal level data (Lesson #13).
Worse, the NPS is flawed because it violates basic principles of psychological measurement, making false assumptions as to how people interpret scales. Rule #1 in Psychology is “people are different” and respondents have different perceptions as to what the undefined anchors mean. The NPS measure flatly rejects this commonsense rule and states that all people interpret an 8 on the scale exactly the same way.
Employees impacted by the NPS know that respondents MUST choose a 9 or a 10 because an “8” is bad news. In NPS lingo, a customer that rates the performance of a teller to be a 9 or a 10 is a “promoter” (positive score) but one who rates the same teller as a 7 or an 8 is “passive” and counts as a zero towards the score. Customers who rate the teller a 6 or lower are labeled “detractors” (negative score) and all responses are lumped together as if they are equally bad.
This misinterpretation is not only false in theory but is overwhelmingly rejected by the data. Having reviewed the NPS ratings and comments given by more than 900,000 people over a five year period, I have seen thousands of examples where individuals have provided a score of 7 or 8 and then typed comments to say how pleased and satisfied they were. This frustrates the individual being rated: “gee, if you were so delighted why did you rate me an 8 instead of a 9?!” as if the respondent was supposed to understand the hidden meaning behind this inflexible scale.
3 Billion reasons to stop “satisfying” your customers: Free Video Training series
Adding to the problem, companies than assign positive and negative scores to the categories, take the difference and turn it into a percentage, then view those percentages over time as if the contrived scores are now comparable. Essentially they take their customers’ initial perceptions in the form of a number, then transpose that number multiple times at which point one cannot accurately interpret the customers’ perceptions. For those keeping score:
Input Ordinal data –> Transpose to Categorical data –> Transpose again toInterval data = A hot mess
Skilled NPS users employ various work-arounds to get interpretable results (hmm… what does that tell you about the NPS?…) One quick fix to this mess is to keep the rating as it is instead of transposing the results into meaninglessness. If someone rates your service as a 7 then it is a 7. If they rate it a 9 it is a 9. Calculate the means and this number can be compared across departments or over time. Of course it is still squishy in that we have not defined the anchors but results will be interpretable if augmented (and revised!) with comments provided by the respondents.
A superior method is to use a 4-point forced choice scale with thoughtful, clear and interpretable descriptors for each anchor:
Would you recommend this service to a friend or colleague?
- Definitely Yes
- Probably Yes
- Probably No
- Definitely No
Each category is distinct. Probably Yes is categorically different than Definitely Yes or Probably No. Respondents that choose Definitely Yes or Definitely Nomake their intent clear. Respondents can choose Probably Yes to indicate that they are not enthusiastic or do not plan to actively promote your business and that something has caused them to not choose Definitely Yes. Respondents can choose Probably No to not be a full rejection, that they have concerns or perhaps do not know enough information to recommend.
A simple count of each category will tell your organization where you stand with your customers. Now that you can interpret your customers’ perceptions, you can take meaningful action. The goal is then to get as many Definitely Yes as possible and then act on the comments provided to determine why customers lean toward or away from recommending your services.
As always, all comments are welcome. Just don’t rate me on an 11-point scale as I won’t know what you are trying to tell me! / Doc
