What Does 100,000 Queries Prove If You Cannot Measure Completed Tasks?

A chatbot dashboard crossing 100,000 monthly queries proves that customers are typing into it. It does not prove that customers completed their tasks, received correct answers, or would have finished faster by speaking Twi.

In 1999, NASA’s Mars Climate Orbiter went silent as it approached Mars. Controllers expected contact to resume after the spacecraft passed behind the planet. It never did.

The investigation, led by Arthur Stephenson, found that one team had supplied thruster data in pound-force seconds while another part of the system expected newton seconds. The mission had accumulated plenty of precise measurements. One missing distinction made those measurements point toward the wrong outcome.

NASA documented the failure in its Mars Climate Orbiter Mishap Investigation Board Phase I Report. The lesson reaches beyond spacecraft: a number can be accurate inside its own definition and still misrepresent what matters.

A query count measures demand, not resolution

Now picture a composite support lead in Accra opening the dashboard on Monday morning. The monthly counter has passed 100,000 queries. There is a screenshot ready for the management update and a clean growth curve for the slide deck.

Then the practical questions begin.

How many customers completed the task they came to do? How many repeated the same question after receiving an incomplete answer? How many switched to an agent? How many abandoned the exchange because typing in English required more effort than explaining the issue aloud in Twi?

The dashboard cannot answer.

This does not make query volume useless. It can show adoption, demand, and changes in activity. The problem starts when activity becomes a substitute for customer outcomes. A support channel can attract more use while leaving the hardest cases unresolved.

Language adds another layer. A customer may understand English while still describing a payment dispute, delivery problem, or account concern more precisely in Twi. A text interface may record the English message that was easiest to type, rather than the issue the customer could have explained naturally.

The result is a measurement gap. The business knows that a conversation started. It lacks evidence that the customer reached the right end state.

Measure the customer’s task across every channel

The stronger metric starts with the job the customer needs to complete.

For a support desk, that could mean confirming a payment, correcting an account detail, checking an order, or recording a consent decision. Completion should have the same definition across chatbot, phone, and human-assisted support. Otherwise, comparisons reward whichever channel produces the easiest events to count.

Useful evaluation asks:

• Did the customer’s task reach a verified end state?
• How long did that take from first contact?
• Did the customer repeat information or change channels?
• Was the language they preferred available throughout the interaction?
• Did the system preserve consent, opt-out status, and an auditable record?

These questions move the focus from traffic to resolution. They also expose where language support changes the result.

A Twi voice interaction should be judged on the full exchange, not on whether the system produced recognizable Twi audio for a demonstration. Speech recognition must capture what the customer said. Synthesis must respond clearly. The calling system must preserve what happened after connection, including completion, failure, consent, and opt-out events.

This is why unsupported language handling can create misleading records, as described in how Twi becomes garbage characters in unsupported systems. A transcript can exist while failing to represent the conversation.

What Asenda Talk can evaluate today

Asenda Talk is in active early access. Users can create voice agents and configure their persona, first message, and voice. The platform includes native Twi speech recognition and synthesis fine-tuned in-house, with more African languages in progress. Vapi orchestrates the assistant runtime.

The current system also includes a telephony lifecycle webhook pipeline designed to track what happened during a call. Metered per-minute billing sits behind an operator-controlled real-money gate. Consent, opt-out, and audit records are part of the call model, and administrative secrets are write-only, masked, and environment-aware.

Those pieces provide the foundation for evaluating calls beyond a simple attempt count. They do not yet establish that outbound calling is generally live. The telephony-provider decision remains open, and outbound use stays gated until that choice is made.

That distinction matters. A responsible early-access claim is narrow: Asenda Talk can help teams build and evaluate native Twi and English voice-agent workflows with call lifecycle records. It should not claim production reach, completion gains, or feature parity with Vapi, Retell AI, Bland AI, and other established platforms until real deployments supply that evidence.

Replace the milestone with a comparison

The next useful dashboard should place query volume beside task completion, time to completion, repeat contacts, language choice, handoffs, and verified call outcomes.

Then run a contained comparison. Choose one support task with a clear end state. Measure how customers complete it through the existing text channel. Evaluate the same task through a Twi and English voice workflow, once the calling setup is approved for that use. Keep consent and opt-out records attached to every call.

The goal is not to prove that voice wins. The goal is to discover which customers finish, where each channel fails, and whether spoken Twi removes effort that the query counter never captured.

NASA’s Mars Climate Orbiter report did not blame measurement itself. It showed what happens when teams fail to agree on what a measurement means. Before celebrating the next 100,000 queries, define the unit that matters: a customer who completed the task.