What should we ask a vendor about their number?

What is in the denominator, what counts as handled, and whether an abandoned conversation is included. Those three answers turn a percentage into something you can interpret, and a vendor who cannot give them quickly has told you something as well.

Why does it rise when nothing has improved?

Because it moves fastest on simple enquiries and because abandonment usually counts as success. Automating the one-line answers and losing a few frustrated customers both push it upwards, and neither is the outcome anybody bought the system for, though both look identical in the report.

What should we measure instead?

This alongside repeat contacts and what happened afterwards. Deflection on its own describes who handled an enquiry; pairing it with whether the same customer came back within a few days describes whether the enquiry was actually dealt with, which is the question anybody funding the work is asking.

Glossary

Deflection rate

The share of incoming enquiries handled without reaching a person, quoted constantly and defined differently by almost everybody quoting it.

In plain terms

How many customer enquiries the software dealt with so nobody on your team had to. It sounds like a simple fraction. The trouble is that everybody counts the top and the bottom of that fraction differently, so the same percentage can describe very different situations.

Why it matters

Because it is the number the whole purchase gets justified on, and it is the number least likely to survive contact with your own definitions. A figure agreed in a sales conversation and measured a different way afterwards is how a successful deployment becomes an argument about the reporting.

How it works

It is a fraction, and both halves are contested. What counts as an incoming enquiry and what counts as being dealt with are decisions somebody made, and different reasonable answers move the result by a wide margin without anybody being dishonest.

The denominator is where most of the movement is. Counting every visitor who opened a chat window produces one number; counting only enquiries that would otherwise have reached a person produces a much smaller and much more useful one.

A customer giving up counts as a success under most definitions. Somebody who asks twice, gets nowhere and closes the window has been deflected in the arithmetic, and the business has lost them without any record of what happened.

Simple questions inflate it. Automating the enquiries people used to answer in one line raises the rate quickly and saves almost no time, whereas the difficult cases barely move it and are where the actual cost sits.

It says nothing about whether the answer was right. The measure is about who handled the enquiry rather than about the outcome, so a system confidently answering wrongly scores exactly as well as one answering correctly.

Seasonality and product changes move it independently of the system. A rate that rose after a release which removed a confusing screen has told you about the screen, so any comparison across months needs to know what else changed, or it will credit the software with somebody else's work.

The version worth tracking is your own, defined before deployment. Fix what counts on both halves, measure the current position by hand if necessary, and treat any vendor figure as a claim about their customers rather than a forecast about yours.

Same percentage, two situations

What makes this more than pedantry is that the gap between the columns is not a rounding difference. On the same deployment, generous counting can report a headline that looks transformative while strict counting reports something a manager would describe as a modest improvement, and both numbers are honest arithmetic performed on the same conversations. That is why the useful discipline is to write your own definition down before anybody demonstrates anything, because after a deployment the definition becomes a negotiation and whoever is being measured has every incentive to prefer the generous reading, usually without ever consciously deciding to. The strict version is also more work, which is the real reason it loses: somebody has to decide what would have reached a person, and somebody has to look at abandoned conversations. The compensation is that the strict number is the only one that ever predicts anything about next quarter's staffing, which is the question the measure exists to answer.

Seen in the wild

A widget counting every opened conversation in its denominator.
Tidio Lyro
A quoted rate that turns out to exclude the enquiry types you care most about.
Sierra
Answers improving because the underlying material improved, not the model.
Guru

Common misconceptions

People assume

Two vendors quoting the same rate are offering the same thing.

In fact

They may be counting different denominators and different definitions of handled. The percentage is a conclusion drawn from choices neither of them has shown you, so comparing the figures compares nothing.

People assume

A higher rate is better.

In fact

It rises when customers give up, and it rises fastest on the easy enquiries that cost least to answer. A rate climbing while repeat contacts climb with it is describing a problem rather than a success.

Questions

What should we ask a vendor about their number?: What is in the denominator, what counts as handled, and whether an abandoned conversation is included. Those three answers turn a percentage into something you can interpret, and a vendor who cannot give them quickly has told you something as well.
Why does it rise when nothing has improved?: Because it moves fastest on simple enquiries and because abandonment usually counts as success. Automating the one-line answers and losing a few frustrated customers both push it upwards, and neither is the outcome anybody bought the system for, though both look identical in the report.
What should we measure instead?: This alongside repeat contacts and what happened afterwards. Deflection on its own describes who handled an enquiry; pairing it with whether the same customer came back within a few days describes whether the enquiry was actually dealt with, which is the question anybody funding the work is asking.