High-stakes marketplace matching
Helped fix the matching problem after major firms had struggled with it for years.
- 97.5%: Accuracy, against a 95% target
- 1/20: Of what the costs were before
- $10M+: The value of the matching problem
Before
A marketplace connecting customers with professional service providers had a matching problem worth more than $10M. For two years, well-known consulting and technology firms had tried to fix it and failed. The client gave a two-week window, ready to spend serious money but out of patience. They kept switching suppliers, each trying the same approach (tuning the model) on a problem that wasn’t really about the model.
The hard part
Nobody had asked the basic question: is the AI the problem, or is the data underneath it? Every supplier had assumed the data was sound and treated this as a model-training job. It wasn’t. The underlying data was inconsistent. The same things were recorded differently in different places, so no amount of model tuning was going to fix it.
What got decided
Looking at the real data made the answer clear, and three decisions followed:
- Fix the data first, then build the matching on top, rather than trying another model
- Let customers search in plain language instead of putting the AI directly in front of them, a confidence call the accuracy made possible
- Stop trying new suppliers, because the problem is solved
Result
97.5% accuracy, against a 95% target. Costs cut to a twentieth of what they were. The client stopped shopping around and showed the result off internally, which prompted a reshuffle at the top.
Two years of failure weren’t down to weak AI. They were down to an unchecked assumption about the data. Looking at the data before the model unlocked everything else.
Download the PDF