A dashboard showing 100,000 chatbot queries tells a support manager how busy the system was. It does not show how many customers solved their problem, especially when a Twi-speaking customer can leave after a misunderstood exchange without the chatbot recording a failure.
In September 2016, Wells Fargo chief executive John Stumpf appeared before the US Senate Banking Committee after regulators exposed a sales system that rewarded account volume. Employees had opened deposit and credit card accounts without customers’ consent while trying to meet aggressive targets. The number on the dashboard had risen. The customer outcome underneath it had deteriorated.
The Consumer Financial Protection Bureau documented the practices when it announced enforcement action against Wells Fargo in 2016. The case became a blunt lesson in measurement: when leaders treat an activity count as proof of customer value, the count can hide the damage it appears to measure.
Query volume measures traffic, not resolution
Absa has said its chatbot fields 100,000 queries each month. That figure establishes scale. It does not, by itself, establish whether customers received correct answers, completed their intended task, or had to move to another channel.
A query may represent a resolved balance question. It may also be the first of six attempts to explain a failed payment. One customer can generate multiple queries because the bot misunderstood the request, repeated a generic response, or lost context. Another customer may leave after one exchange and call the support desk, visit a branch, message an employee, or abandon the issue.
Those outcomes look identical in a volume report: one or more queries were processed.
Language can widen this measurement gap. A customer may begin in English, switch to Twi to explain the important detail, then stop when the system no longer follows. If the chatbot records the conversation as completed because the session ended normally, the dashboard gains another interaction while the customer keeps the original problem.
This is why language fit belongs inside resolution measurement. An English-only transcript can create the wrong operational conclusion even when the system captured every message.
Define a resolved problem before counting one
Support teams need a resolution definition tied to the customer’s intended outcome. “Session completed” is too weak. “Answer delivered” is also weak unless the team can verify that the answer was relevant and sufficient.
For a simple information request, resolution might mean the customer received the requested information and did not repeat the question or contact another channel about the same issue within an agreed review window. For a transaction problem, resolution might require a confirmed status change in the underlying system. For a complaint, it may require a case reference, correct routing, and a recorded next step.
The exact definition will vary by use case, but it should answer four questions:
- What was the customer trying to accomplish?
- What evidence shows that outcome occurred?
- Did the customer repeat, escalate, or abandon the issue?
- Could the system understand the language used when the decisive detail appeared?
The last question matters in Ghana. A system that performs well on English test prompts can still fail when a customer uses Twi for names, amounts, relationships, symptoms, or the part of the story carrying the most risk. Teams should review outcomes by language and code-switch pattern, instead of averaging every conversation into one reassuring score.
Connect conversation records to operational truth
A useful support report follows the problem beyond the chat window. It links the conversation to the case, payment, appointment, delivery, complaint, or account action that the customer wanted.
That requires more than sentiment scores and session counts. Teams need outcome events from the systems where work actually happens. They also need clear failure categories: misunderstood language, incorrect answer, missing integration, customer opt-out, human escalation, repeated contact, and abandonment.
For voice support, call lifecycle records add another layer. A call marked “ended” says little about why it ended. The customer may have completed the task, opted out, lost the connection, reached voicemail, or hung up after the agent misunderstood them. The gap between call records and customer truth becomes expensive at scale.
Asenda Talk is being built around this distinction. Its current early-access platform includes native Twi speech recognition and synthesis, voice-agent configuration, consent and opt-out records, audit trails, and a telephony lifecycle webhook pipeline for call-truth tracking. Vapi orchestrates the assistant runtime. Outbound calling remains behind an operator-controlled real-money gate while the live telephony-provider decision is unresolved.
That status matters. A platform should report what happened today, including what remains blocked, instead of converting technical activity into a success claim.
Put the Monday report on trial
When the weekly headline says “100,000 queries handled,” ask the team to produce the denominator and the outcome trail. How many distinct customer problems entered the system? How many reached a verified resolution? How many moved to another channel? How many conversations contained Twi, and how did their resolution rate compare with English-only conversations?
Then inspect a sample of sessions classified as successful. Include short conversations, abrupt endings, repeated questions, code-switching, and contacts followed by a call or branch visit. These are the places where false success tends to hide.
Wells Fargo’s account totals looked productive until investigators examined what those accounts meant for customers. Support leaders face a smaller-stakes version of the same measurement error whenever interaction volume stands in for value.
Keep the big query number on the dashboard if it helps plan capacity. Place verified resolution, repeat contact, abandonment, escalation, opt-out, and language-specific performance beside it. On Monday morning, those are the numbers that tell the support manager whether customers received help.
Comments
No comments yet.