Weekly Thoughts


Weekly Thoughts: N.P.S. BS?

Here is something that caught our eye this week:

N.P.S. BS?

Turns out, measuring customer satisfaction on a scale of 1-10 isn’t a silver bullet  

Every time we get comfortable with a “truth”, something happens that throws us for a loop.  Fat was bad, now it’s good.  Profitability was a key driver of corporate value, now it’s irrelevant (to VC at least).  Daenerys was our Rightful Queen, now she’s batsh*t crazy.  It’s hard to keep up.

Similarly, we read this week that a core metric used to evaluate corporate performance – the Net Promoter Score, or “NPS” – may not be exactly what it seems.  The NPS is attractive because it is relatively simple to administer, asking customers a single question, “On a scale of 0 to 10, how likely are you to recommend the company’s product or service to a friend?”.  Results are divided into three groups: “Promoters” (people who answer 9 or 10), “Passives” (7 or 8), and “Detractors” (0 to 6) with the score calculated by subtracting Detractors from Promoters (Passives are ignored).

First introduced in 2003, it has become increasingly common within the corporate lexicon to use NPS as a reflection of customer satisfaction and by extension, an indicator of business health.  We’re not talking about off-hand comments here either.  The score is being reported in earnings calls, references in securities filings, and the results are being used as the basis for large decisions at large companies, like employee bonuses (Best Buy, Citigroup, and American Express), investment decisions (Target and Intuit), and to qualify overall business health (Delta and UnitedHealth).

Unfortunately, based on recent WSJ research, there’s not much basis for the score as a reliable metric as it can produce rather noisy results and does not account for cultural differences. From the WSJ:

“Some academics have questioned the whole idea, suggesting that NPS has been oversold. Two 2007 studies analyzing thousands of customer interviews said NPS doesn’t correlate with revenue or predict customer behavior any better than other survey-based metrics. A 2015 study examining data on 80,000 customers from hundreds of brands said the score doesn’t explain the way people allocate their money. ‘The science behind NPS is bad,’ said Timothy Keiningham, a marketing professor at St. John’s University in New York, and one of the co-authors of the three studies. He said the creators of NPS haven’t provided peer-reviewed research to support their original claims of a strong correlation to growth. ‘When people change their net promoter score, that has almost no relationship to how they divide their spending.’”

Peer reviews aside, even the inventor of the metric, Fred Reichheld, believes that the results have been taken out of context.  As noted to the WSJ: “…he is astonished companies are using NPS to determine bonuses and as a performance indicator. ‘That’s completely bogus… I had no idea how people would mess with the score to bend it, to make it serve their selfish objectives.'”

One of Chenmark’s core values is Keeping Score (using data to evaluate performance), so the temptation is there to lean heavily on NPS, but we are increasingly aware that there are no silver bullets when it comes to satisfaction metrics.  Despite its benefits –  it is simple to administer and communicate –  the limitations of NPS require that we utilize more of a mosaic approach to evaluate how our customers perceive the value our companies provide.  This makes life more difficult, but also more authentic and fun.  If there is anything we have learned in our small business experience thus far, it is that few things can truly be distilled down to a single number.

Have a great week,

Your Chenmark Team

Subscribe to Weekly Thoughts

Previous Post Next Post

Recent Posts

First Follower

When we started Chenmark, many thought we were a "lone nut" by quitting lucrative Wall Street jobs to "become landscapers in Maine". As we have built out our company over the last five years, however, we have been lucky enough to attract a couple of first followers.  These are the core team members who believe enough in what we are doing that they are willing to do something a bit different and join us in our pursuit of becoming the acquirer of choice for North American small businesses.

Read More

Nevermind Success

A couple of weeks ago, we wrote the about the lessons we learned from Sub Pop Records' signing of Nirvana.  This week, we wanted to highlight a related story, which has more to do with Nirvana than Sub Pop. Bruce Pavitt and Jonathan Poneman, Sub Pop's founders, saw talent in Nirvana and figured out a way to sign the band and release their first album - Bleach - in 1989. 

Read More

Still Eating Broccoli

We have been thinking a lot recently about silver bullets.  Entrepreneurial culture tends to concentrate, for good reason, on examples of massive efficiency-creating technology or truly exceptional leadership.  Uber represents a one-stop solution for local transportation just as Steve Jobs is credited with being a singular driving force behind the success of Apple. 

Read More