Google Researchers Detail Phoneme Fix For Hard To Pronounce Drug Names In TTS

Conversational text to speech systems built for voice agents are trained to sound natural across a whole dialogue, adapting tone and pacing to whatever came before. That same context sensitive training makes them poor at something a clinic cannot afford to get wrong: pronouncing a drug name correctly the first time it comes up, especially a newly approved one that no training run has ever seen. The paper, posted on October 16, 2026 by a team including Google affiliated researchers, addresses that gap directly rather than waiting for a bigger pretraining run to absorb the missing vocabulary.

The method does not retrain the base model. Instead it adds an ARPAbet phoneme vocabulary on top of the pretrained conversational TTS system, continues supervised fine tuning with phoneme augmented data, and applies an acoustic biasing step only at inference time, when the model is actually speaking. Because none of the three stages ever sees the actual drug names used in testing, the researchers argue the gains measure learned pronunciation skill rather than memorized answers, which is the harder and more useful claim to defend.

The headline evaluation used 7,500 clinical drug pronunciation clips, scored by Gemini acting as a strict automatic judge. Correctness rose from 54.6% without the phoneme fix to 90.7% with it, beating every other conversational and stateless baseline tested. A separate human listening study, with adult listeners judging a blinded and randomized set, found a narrower gap of 84% correct for the new method against 82% for the strongest baseline, with overlapping confidence intervals on a stratified subset, a reminder that the automatic metric and human ears do not always move together by the same margin.

The numbers

The team also checked how well Gemini’s judgments tracked actual human listeners across 250 matched clips, finding 84.8% agreement on the binary correctness label, with a kappa score of 0.57. That is a moderate level of agreement, strong enough to make Gemini a usable scoring tool for this kind of evaluation but not a replacement for the human study the researchers ran alongside it. For teams deciding whether to lean on an automatic judge when building or auditing a voice agent, that agreement figure is the number worth checking before trusting the automatic score alone.