The industry narrative is clean: “LLMs replace support agents.”
The pitch is always the same. Lower headcount, 24/7 availability, instant responses. If you look at the SaaS landing pages, it looks like a solved problem.
But I’ve spent enough time in the internals of Shopify stores and cloud architectures to know the reality is different.
The reality is that LLMs don’t actually solve support problems. They just automate the speed at which you frustrate your customers.
I remember a retailer last year who was convinced they’d “solved” their ticket volume. They deployed a state-of-the-art agentic layer. On paper, the metrics were amazing. Deflection rates soared. The “bot” was handling 80% of queries.
Then we looked at the churn.
What was happening was a silent failure. The AI wasn’t solving problems; it was providing confident, hallucinated promises. It was telling customers their packages were arriving on Tuesday when the warehouse was actually on fire.
The customers didn’t fight the bot. They just stopped buying.
This is the “Support Debt.”
When a human agent makes a mistake, you fix the process. When an LLM makes a mistake, it does it ten thousand times a second, and it does it with a level of politeness that masks the catastrophe.
The a-ha moment comes when you realize: AI without a surgically clean data layer is just a faster way to lie to your customers.
Most people are obsessing over the model. The “GPT-4 vs Claude 3.5” debate is a distraction. The model isn’t the bottleneck.
The bottleneck is your data hygiene.
If your order status API is flaky or your inventory sync is lagging by ten minutes, your AI agent is just a high-speed megaphone for that brokenness.
I’m starting to think that the “AI Support” dream is actually a mirror. It doesn’t show us how smart the models are; it shows us exactly how messy our operational data actually is.
Maybe the goal isn’t to replace the agents.
Maybe the goal is to finally fix the data that made the agents necessary in the first place.
Nobody seems to be budgeting for that part.