At Magic Pages, we get a certain kind of support message every few days now, and it always has the same shape.
It's long. It's numbered. It quotes something specific. A package version, an error class, an internal-sounding mechanism with a name that reads like it came directly from a source file of Ghost.
It asks us to confirm or deny two, three, four possibilities, phrased like we're debriefing a high-impact incident that affected our entire infrastructure. And somewhere in the middle of it, there's a claim about how our infrastructure or the underlying software works, stated with total confidence. Just that it's entirely made up. A mechanism that doesn't exist, describing a system the sender never actually inspected.
I don't want to point fingers. It would be easy to write this as a complaint or rant. But that's not what this is. I don't think anyone sits down and writes these out of malice, or even out of laziness exactly. I think they ask an LLM what's wrong, get back something articulate and specific-sounding, and forward it to us more or less as-is.
And well, that's an increasingly normal thing to do. What I want to think about is what it's doing to actually getting a problem solved.
The asymmetry no LLM is modeling
The hypothesis embedded in these messages is almost always some version of "does your managed platform silently modify documented open-source behavior" – and that's almost never the right hypothesis. Not because we are oh-so trustworthy, but because of basic probability.
We host nearly 1,500 Ghost sites on the same stack. If we modified Ghost core behavior somewhere, every one of them would be affected. We'd have dozens, if not hundreds, of support tickets, and it would usually show up in our own testing before a single customer noticed anything.
Meanwhile, a language model confidently inventing a plausible-sounding internal mechanism it has never actually observed is something that happens constantly. It is well documented, and requires no conspiracy at all. Just a model reasoning toward whatever sounds most like a real explanation. After all, these models are trained on being "helpful".
So, an LLM will always be more willing to hallucinate a specific, wrong answer than to say "I don't actually know how this managed platform is configured, and I have no way to check.".
It has no access to our infrastructure. It usually has a light understanding of the underlying software (in this case, Ghost). It has almost never actually read the current documentation as part of forming the answer. And even when it has, it can still misread which part applies.
What it has, instead, is a very convincing and helpful tone of voice, layered over something that would take thirty seconds to falsify against the actual documentation.
In our support chats we usually falsify it quickly. Not because we like to be clever, but because we can just... read the documentation ourselves and go check the logs. The chance that we quietly changed how a well-documented open-source system behaves is much, much lower than the chance the model made something up.
What actually would have been faster
The single most useful message a customer can send to customer service anywhere follows this pattern: "this is what I did, this is what I expected, this is what happened instead".
No theory required. No numbered options for us to confirm or deny. Just the observation.
We are, genuinely, happy to do the diagnosis. After all, that's our job. Give us the symptom, not the mechanism you think produces it.
We have shell access, log access, and years of experience with similar requests. An LLM has none of that. But it will confidently sound like it does.
A huge formal letter format doesn't make things quicker either. In fact, a long, numbered, professionally-formatted message doesn't move faster through a support queue than three quick sentences. It just takes longer to read, and if the premises underneath are wrong, we now have to gently un-teach the premises before we can even get to the fix.
"I attempted X, expected Y, got Z" usually gets solved in one reply. "Here is my working theory of your infrastructure, please confirm or deny these specific claims about it" usually ends up in a back and forth. We explain why we disagree with what you sent. We get another LLM reply back. We try to invalidate that.
This isn't really about AI
However, I don't think the solution is "stop using AI to talk to support". People are going to keep doing that, and often for good reason. English isn't everyone's first language and some people find it genuinely hard to describe a technical problem clearly. I'm not against any of that.
What I'd ask for is a sensibility check before hitting send: did the tool actually look at anything real, or did it just generate something that sounds real?
If your message ends with a hallucinated technical analysis that neither you nor the model ever verified, that's the sign to strip it out before it is sent.
The old rule of good bug reports was always: describe what happened, not what you think caused it. That rule got more important, not less, now that it has become so easy to generate a very confident paragraph about the supposed root cause. But confidence was never the same thing as correctness.
On our end, we'll keep reading through it either way. We'll keep reasoning with it, checking the logs, going back to the actual docs, even when we're fairly sure where it's going to land.
Because that's the job of being the human in the loop for infrastructure someone else doesn't have to understand. So a real person can get their real problem fixed.