Audible's controllable TTS attributes dialogue with 97% accuracy in English and improves character differentiation by 3.

Audible's controllable text-to-speech (TTS) system identifies which character speaks each line of a book with 97% accuracy in English and 94% in Spanish, according to a post by Arent van Korlaar on the Amazon Science blog dated October 23, 2026. Rendering distinct character voices improved dialogue differentiation by 3.7 points on a MUSHRA test, a 0 to 100 listener-rating scale.

The problem with long-form TTS

The Audible technology team says conventional TTS models do not reliably change speaking style between characters the way a human narrator would. That leaves listeners struggling to tell narration from dialogue and to identify who is speaking. The team notes that audiobook listeners engage with the same voice for 10 to 15 hours per book, so defects such as unnatural prosody or monotone delivery become increasingly grating.

General-purpose TTS models also process individual utterances with minimal context, and often voice a character in a markedly different way in chapter 1 than in chapter 10, according to the team.

Three components

Data synthesis pipeline. Controllable TTS needs annotations describing how speech sounds, and most existing training data is neutral in tone. The team expanded training sets through new recordings and human captioning of existing audio. Because existing datasets use inconsistent description formats (long, short, single tags), the team uses LLMs to rewrite prompts into different lengths, verbosity levels and structures, to improve generalization.

Human-in-the-loop collection does not scale, so the team adds synthetic data through two approaches:

• Expert-model pipeline: pitch extraction, speaking-rate extraction and emotion detection produce numerical features and classification labels, which an LLM turns into speech captions.
• Speech captioner: a model that does the inverse of controllable TTS, taking speech and outputting a description of prosodic features. Its annotations seed the initial controllable-TTS model, and both models are improved iteratively in a synthetic-data flywheel.

BookLLM. This separate component uses off-the-shelf LLMs to identify unique characters and define voice characteristics from details in the book such as age, gender, personality and role. It then determines which character speaks where and captions the dialogue with delivery descriptions that act as stage directions. The team evaluated BookLLM on 25 English and Spanish books in the mystery, sci-fi and romance genres, using ground truth from two human annotators. It reached 97% accuracy in English and 94% in Spanish at identifying which character speaks where, including separating narration from dialogue.

Full pipeline. BookLLM first reads the entire text to build a cast list, ranks characters by importance and derives a voice description for each. Each character is assigned a distinct profile from a curated voice catalogue, with contrast deliberately maximized between the most prominent characters. Minor and one-off characters fall back to the default narrator. Delivery is controlled through natural-language captions such as "whispering nervously" or "shouting in anger," and each character's voice profile is fixed once at the book level so the model renders that character consistently from beginning to end.

Results

Across tests in English and Spanish, the team reports statistically significant improvement in listening experience for eight of ten voices. Dialogue differentiation improved by 3.7 points on the MUSHRA test.

The post does not name a model, parameter count, pricing or release date, and it describes the work as the approach Audible's team is using rather than a product launch.

What this means

The work separates two problems: understanding the story (done by BookLLM with off-the-shelf LLMs) and rendering speech (done by a TTS model trained to follow natural-language captions). Fixing voice profiles once per book addresses the chapter-to-chapter inconsistency the team describes. The 97% and 94% figures measure character attribution, not audio quality, and the 3.7-point MUSHRA gain and eight-of-ten-voice result are the team's own listener-test measurements.