Where Does the Language Model Sit in Your CX Analytics?
In almost every CX meeting I have joined over the past year, large language models come up at some point. Someone has had thousands of comments summarised, someone else has had themes extracted, and there is a tidy dashboard on the screen. Each time I find myself wondering the same thing: is this model actually connected to a decision, or is it a layer that makes the dashboard look good?
Think back to classic text analytics. We weighted words with TF-IDF, pulled topic clusters out with LDA and classified comments with logistic regression. All of these methods relied on a fixed set of features; the model rarely went beyond the categories we had taught it. Language models stretch that limit. They can classify with no examples or with a handful of them (zero-shot and few-shot), turn text into a semantic vector, infer themes from context and estimate what the writer intended.
Take the sentence “Nobody called me back.” A classic model would probably call it neutral, because it contains neither a negative word nor a familiar complaint pattern. A language model can read the same sentence in context and label it as a feeling of being neglected. The difference looks small, but it changes a great deal about how well you understand what the customer is going through.
Even so, I would not use that power to predict churn directly. A language model was not built to be a deterministic prediction model, and there is no guarantee it will give the same answer to the same question every time. What it does very well is represent meaning. That is why the set-up I trust more is a hybrid one. You first turn the text into an embedding, then feed that representation, together with the customer’s behavioural data, into a classic machine learning model, and finally use a method such as SHAP to separate how much each feature contributes to the result. This arrangement is far more stable than asking the language model for a prediction.
For me, this last step is where the work meets finance. A language model writes a good summary, but a CFO is not satisfied with a summary; they want to see the impact. The sentence “AI says our customers are unhappy” immediately raises a few questions in the room: why are they unhappy, because of which issue, and what does it cost us? When the language model extracts the themes, the prediction model calculates the risk score and SHAP separates the impact, you can answer them. Leave one of the pieces out and all you have left is a feeling.
Then there is reliability. Language models can produce inconsistent labels, put the same comment into two different classes on two runs, and overreact to a single changed word in the prompt. The way to manage this is to place the model inside a data pipeline instead of leaving it to run as a free-standing generator. You version your prompts, pass the outputs through a validation layer, have people check a sample of the results regularly, and monitor whether the model drifts over time.
So where does a language model add the most value? The places I see are fairly clear: telling apart how intense an emotion is, reading what the customer intends and expects, catching abstract themes such as fairness, trust or respect, and turning layered, complicated complaints into a summary people can follow. The final decision, though, still belongs to a prediction model and a calculation of financial impact.
Language models are a fascinating technology, and I have no argument with that. Their value shows up when you decide which architecture to connect them to, and how. In your CX analytics, is the language model working as a tool that produces content, or as part of your risk engine?