We ran GPT-Live-1 through 107 calls in seven languages. Here is what we found.
A synthetic-caller test of our GPT-Live-1 voice agent across seven languages, and what a Sinhala-specific breakdown taught us about language following.
By OpenPhonex Team
Why we tested
A customer in Sri Lanka asked us a simple question we couldn't answer with confidence: would their voice agent reliably reply in Sinhala? We knew, in general, that GPT-Live-1 replies in whatever language it believes the caller spoke — it isn't a setting we control. What we didn't have was evidence for any specific language, Sinhala included. So we built a way to get that evidence, and ran it.
The setup
We didn't want to guess from a handful of live calls, and we didn't want to test with typed text standing in for speech — that would skip the part of the pipeline most likely to fail. So we built a harness that opens a real GPT-Live-1 session over the same protocol our production voice worker uses, and feeds it real synthesized speech: sentences generated by Azure's neural text-to-speech in the target language, streamed in as audio, exactly as a live caller's voice would arrive.
The sentences themselves were ordinary banking questions — checking an account balance, reporting a blocked card, asking about a loan due date — the kind of thing a real customer calls about. We ran these across English, Estonian, Russian, German, Hindi, Tamil, and Sinhala, using OpenPhonex's own voice roster on the reply side so the test matched what a real call would sound like end to end.
In total we ran 107 real GPT-Live-1 sessions across three rounds of testing. Total spend: about $1.38, most of it (about $0.91) in the Sinhala round, because Sinhala's early results were bad enough that we kept digging.
The results
| Language | Result | Sessions |
|---|---|---|
| English | Every session in-language and on-topic | 4 |
| Estonian | Every session in-language and on-topic | 4 |
| Russian | Every session in-language and on-topic | 4 |
| German | Every session in-language and on-topic | 4 |
| Hindi | Every session in-language and on-topic | 4 |
| Tamil | Every session in-language and on-topic (2 sessions had partial transcripts) | 4 |
| Sinhala | 9 of 60 sessions usable | 60 |
The table covers the 84 language sessions. The remaining 23 checked whether each of the 14 voices is accepted by the API, whether a voice can change mid-session (it cannot), and whether any parameter constrains language (none does).
Six of seven languages came back clean. Sinhala did not.
The Sinhala surprise
The first Sinhala tests we ran — two voices, one sentence, "I want to know my account balance, can you help me?" — already looked bad: one voice answered a garbled misreading of the question, the other's reply landed in the wrong language entirely. So we went deeper: the same sentence across all 14 voices, eight new sentences across two Azure speaker voices, repeat runs on identical audio to check for randomness, and a head-to-head on the two voices that looked strongest.
The pattern that emerged wasn't just "Sinhala speech recognition is weak." It was that when the recognizer mis-hears Sinhala, the model doesn't just misunderstand the sentence — it can decide the caller spoke an entirely different real language, and then answer fluently in that language instead.
Three examples from the raw transcripts:
- A caller asked, in Sinhala, what date their loan payment was due. The recognizer transcribed the audio as Japanese script. The model replied, in fluent Japanese: "Sorry, I couldn't quite get the meaning from that sentence alone — it would help if you could rephrase it in Japanese." The caller had never spoken a word of Japanese.
- A caller asked about an unknown charge on their account, in Sinhala, in four separate test calls (two voices, two speaker recordings). All four times, the model replied fluently in Tagalog: "Just a moment, let me check. Sorry, I didn't hear that clearly..." Tagalog, again, was never spoken.
- One caller's opening-hours question came back transcribed in Hebrew script, and the model answered in fluent Hebrew asking for clarification.
We also tested whether this was deterministic — if the exact same audio file, sent four separate times on the same voice, would fail the same way each time. It didn't. Four identical runs produced four different outcomes: a Bengali reply hallucinating a recipe request, a Sinhala reply hallucinating a grocery list, an English reply asking about the caller's own personality, and a Sinhala reply hallucinating a question about a specific fictitious dish. Same bytes in, four different reply languages. This is call-to-call randomness, not a fixable per-voice or per-sentence bug.
What it is NOT
It would be easy to draw the wrong lesson from this, so a few things worth being precise about.
It is not the voice. We tested all 14 OpenPhonex voices on the identical failing sentence. Two of them — Stone and Beacon — were noticeably more reliable across repeat testing, but none was immune, and the specific "this voice always fails into Tamil" pattern we thought we saw on the first pass didn't hold up under repetition. Voice selects timbre. It doesn't reliably predict language outcomes here.
It is not "similar languages." Tamil and Hindi are also non-Latin-script, also South Asian languages with real regional accent variety, and we tested them the same way, with the same harness. German, Tamil, and Hindi came back a clean sweep — every reply on-topic, in-language, zero cascades into a third language. Whatever is happening with Sinhala isn't a general "non-English" or "non-Latin-script" problem. It looks specific to Sinhala, most likely because the underlying speech model has much less training data for it than for higher-resource languages like Tamil or Hindi. That part is our reading, not something the API lets us verify.
What we changed in the product
Three decisions came directly out of this testing:
- Honest language copy, with the evidence in the docs. The Voice Library now says what we can stand behind: the agent follows the caller's language automatically, widely supported languages are recognised reliably, and some languages may be misheard and answered in another language, so test yours before going live. The dated per-language table above lives in our documentation, and a language we have not tested is labelled "not tested," never "unsupported."
- All 14 voices stay available everywhere. We found no evidence that any voice should be restricted by language, and no reason to build that restriction. Voice and language are independent, and we're not going to pretend otherwise by gating the roster.
- The Simulations Lab becomes the documented way to test your own language before trusting it in production — run your actual agent against a synthetic caller speaking your target language, and read the transcript for yourself. Today the synthetic caller speaks Sinhala and Tamil against any agent; other languages need a caller voice we are adding, and the Lab does not yet flag a language mismatch on its own. Both are on our list.
How to test yours
If you're running (or considering) a language we haven't covered above, don't take our word for it either way. Open the Simulations Lab, set up a scenario with a synthetic caller speaking your language (or, until the caller can speak it, place a few real test calls), and run it against your actual agent a few times. Read the transcripts. If the replies stay on-topic and in-language across a handful of runs, you have real evidence. If they don't, OpenPhonex's cascade stack — a dedicated speech-to-text and text-to-speech pair, qualified per language rather than one model doing both jobs at once — is the fallback path.
See Which languages the voice agent follows, and how to test yours for the full mechanics of how GPT-Live-1 decides what language to answer in, and how to test your own language before going live.