A bilingual voice agent can recognize every word and still fail the call if it loses the caller’s earlier answer after a language switch. The critical test is whether meaning survives across turns, corrections, and changes between Twi and English.
At 8:03 AM in Kumasi, Abena stood beside her kitchen table with a school bag open in front of her and a phone pressed to one ear. She was an invented composite, but the failure she illustrates is concrete. Asked for her account number, she read the digits in Twi while removing a leaking water bottle from the bag.
The agent repeated the number correctly. Then Abena switched to English: “Yes, that is the account. I’m calling about the payment from Friday.”
The agent asked for her account number again.
She repeated it, this time in English. The system captured one digit incorrectly and attached the dispute to the wrong account reference. If Abena accepted the confirmation, her payment issue could enter the wrong workflow. If she hung up, she would have to begin again later. For one beat, neither outcome was safe.
Recognition can succeed while the conversation fails
A transcription test might score the first turn as a success. The Twi digits were recognized. The spoken response sounded natural. Each isolated exchange appeared to work.
The conversation still broke.
The failure happened between turns. The agent did not preserve the account number as the same piece of information after Abena changed languages and added a new intent. It treated the English sentence like the beginning of another interaction instead of the continuation of the same one.
That distinction matters for Ghanaian support desks, payment teams, campaigns, and service lines. Callers may greet an agent in Twi, provide a name in English, explain a problem in Twi, then use an English product term because that is how it appears on a receipt. A bilingual call is rarely two tidy language blocks. It moves.
Testing vocabulary alone can therefore produce false confidence. A system may understand “account number” in both languages while failing to retain the actual number already supplied. This resembles the testing problem described in Twi and English Voice Agents: Why the First Message Must Be Tested in Context: a correct line in isolation tells you little about the exchange around it.
Test the state, not only the transcript
The useful question after each turn is simple: what does the agent believe now?
After Abena gave the digits in Twi, the structured call state should have contained one account number with an appropriate confidence level. Her switch to English should have updated the call intent to a payment dispute without clearing that number. If she corrected one digit, the audit trail should have shown the earlier value, the correction, and the final confirmed value.
A practical evaluation should include calls where:
- The caller supplies an identifier in Twi and explains the request in English.
- The caller switches languages halfway through a number or name.
- The agent repeats the value in a different language from the one used by the caller.
- The caller corrects one item without repeating the full answer.
- Background noise makes one digit uncertain, forcing a targeted confirmation.
The expected result must cover more than the transcript. Check the stored field, the call state before and after the switch, the confirmation spoken to the caller, and the downstream event produced when the call ends.
This is where call-truth tracking becomes useful. A team needs to distinguish what the caller said, what speech recognition produced, what the agent stored, what it confirmed, and what the workflow received. Collapsing those into a single “successful call” label hides the point where meaning was lost.
The routing consequences can be serious. What Happens When English-Only Automation Misroutes a Twi Payment Dispute? examines the adjacent risk: language handling can shape where a customer’s problem goes, even when the system captures plausible words.
The safe turn arrives before action
Back at the kitchen table, Abena heard the incorrect English confirmation and paused. The safer agent response would stop the workflow at that moment: preserve the number captured in Twi, identify the mismatch, and ask only for the uncertain digit.
“Please confirm the fourth digit,” is a better recovery than restarting the identification process. It lowers the burden on the caller and limits the chance of introducing a second error.
Only after Abena confirms the final value should the system attach the payment dispute to an account or trigger another action. The audit trail should record the confirmation, while consent and opt-out handling remain available throughout the call.
Asenda Talk is being built for this kind of Twi and English interaction. Its Twi speech recognition and synthesis are fine-tuned in-house, and its agent runtime supports configurable personas, first messages, voices, lifecycle webhooks, and call-truth records. The platform remains in active early access. Feature parity with established voice-agent platforms is still in progress, and outbound calling remains behind an operator-controlled real-money gate while the live telephony-provider decision is unresolved.
That status makes disciplined evaluation more important. A polished voice cannot substitute for retained context, and a correct transcript cannot prove a correct workflow.
Build the test around the handoff
Before placing a bilingual voice agent into a live process, create a small set of language-switch scenarios tied to consequential fields: account numbers, appointment choices, consent decisions, payment references, and corrections.
For each scenario, write down the state that must survive the switch. Then inspect the transcript, stored fields, spoken confirmation, lifecycle events, and final workflow result separately. A failure in any one of them should block the action.
At 8:07 AM, Abena should be able to close the school bag knowing the agent retained the number she gave once, corrected only what was uncertain, and attached her request to the intended account. That is the standard: the conversation can change languages without changing what the caller meant.
Comments
No comments yet.