Audio, Voice, Accent, and Quality Troubleshooting
Comprehensive root causes, config fixes, and synthetic examples for voice gender swaps, accent/language drift, timbre mimicry, speech-rate anomalies, amplitude collapse, tonal humming, and playback jitter.
Table of Contents
- Voice Gender Swaps: Lines 20-66
- Accent Reversion, Pitch Jumps, and Code-Mixed Switching: Lines 68-117
- Output Voice Mimics User Timbre: Lines 119-141
- Multi-Turn Voice and Tone Drift: Lines 143-164
- Fast Speech Rate and TTS Markup or IPA Leaks: Lines 166-200
- Near-Zero Audio Amplitude Collapse: Lines 202-224
- Tonal Humming Buzz (150-700 Hz) and Non-Terminating Greetings: Lines 226-262
- Choppy Playback and Audio Jitter: Lines 264-278
Voice Gender Swaps
Symptom: The model speaks in the opposite gender from the configured voice on Turn 1 or switches between male and female voices mid-conversation.
Root Cause: Pairing a male prebuilt voice with a female persona in
system_instruction (name, role/title, or pronouns), or pairing a female
prebuilt voice with a male persona, creates conflicting acoustic and text
conditioning.
Prebuilt Voice Gender Reference
Match prebuilt_voice_config.voice_name to the system_instruction persona:
- Male prebuilt voices (16):
Achird,Algenib,Algieba,Alnilam,Charon,Enceladus,Fenrir,Iapetus,Orus,Puck,Rasalgethi,Sadachbia,Sadaltager,Schedar,Umbriel,Zubenelgenubi. - Female prebuilt voices (14):
Achernar,Aoede,Autonoe,Callirrhoe,Despina,Erinome,Gacrux,Kore,Laomedeia,Leda,Pulcherrima,Sulafat,Vindemiatrix,Zephyr.
Fix
Align either the configured voice or the persona name, title, and pronouns so both share the same gender.
# Bad: female preset voice paired with male persona
bad_config = {
"voice_name": "Kore", # Female
"system_instruction": "You are Arjun, a travel booking specialist.",
}
# Good: male preset voice aligned with male persona
good_config = types.LiveConnectConfig(
response_modalities=[types.Modality.AUDIO],
speech_config=types.SpeechConfig(
language_code="en-IN",
voice_config=types.VoiceConfig(
prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Puck")
),
),
system_instruction="You are Arjun, a travel booking specialist.",
)Accent Reversion, Pitch Jumps, and Code-Mixed Switching
Symptom:
- The assistant opens with an
en-USaccent or reverts toen-USafter a non-English turn or barge-in, even thoughsystem_instructionspecifies a regional accent such as Indian English (en-IN). - Pitch jumps sharply after switching languages mid-call.
- Short code-mixed utterances (for example, English mixed with Tamil or Hindi) fail to trigger a language switch.
Root Cause:
- Omitted
speech_config.language_code: Whilelanguage_codeis optional for dynamic multilingual auto-detection, omitting it defaults the initial voice reference toen-US. That acoustic reference competes with text accent rules insystem_instruction, causing pitch overcompensation and snap-back toen-USon later turns. - Single-script verbatim prompt inertia: Long
system_instructionflows written verbatim in one non-Latin script (for example, multi-paragraph Devanagari templates) bias the model against switching when a user speaks a short, fast code-mixed phrase in another regional language.
Fix
- Set
speech_config.language_codeexplicitly to a supported BCP-47 locale from the Live API language table (such as"en-IN"for Indian English; see Configure Language and Voice). - Keep
system_instructionconcise, written in neutral goal-oriented English rather than multi-paragraph single-script scripts, and add explicit code-switching triggers.
config = types.LiveConnectConfig(
response_modalities=[types.Modality.AUDIO],
speech_config=types.SpeechConfig(
language_code="en-IN", # Supported locale prevents en-US fallback
voice_config=types.VoiceConfig(
prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Aoede")
),
),
system_instruction=(
"You are Priya, a Bengaluru hotel concierge. Speak in natural Indian "
"English. If the guest switches to Hindi or Tamil, reply in that "
"language seamlessly."
),
)Output Voice Mimics User Timbre
Symptom: The model momentarily adopts the user's voice timbre and speaks in first person as if continuing the user's sentence, then recovers on the next turn.
Root Cause: In native audio-to-audio models, user input audio remains in the active context window alongside the system voice reference. When role boundaries blur (for example, after a trailing user pause or ambiguous context), decoder attention can lock onto the user's recent audio tokens instead of the system voice reference. Text prompts attempting to describe vocal timbre do not prevent this and can worsen role confusion.
Fix
- Upgrade to
gemini-3.8-liveif running an older preview model. - State assistant-vs-user role boundaries crisply in
system_instruction, and remove text instructions that attempt to dictate acoustic timbre or pitch. - Monitor
serverContent.outputTranscriptionfor first-person role confusion and open a fresh session if role leakage occurs so the polluted turn does not remain in context.
Multi-Turn Voice and Tone Drift
Symptom: Over long sessions (many turns), a prebuilt voice drifts in pitch, cadence, or perceived gender even though audio playback and client settings are unchanged.
Root Cause: Model-side discrete audio-token conditioning degrades gradually as turn history grows far past the initial voice reference tokens at the start of the session. Client-side playback speed or synthesizer settings are not the cause.
Fix
- Ensure
speech_config.language_codeis set explicitly and remove any text-based timbre/voice-acting instructions fromsystem_instruction. - Configure
contextWindowCompressionwith a sliding window (for example,trigger_tokens=24000,target_tokens=12000) to prevent unbounded history growth. - Rotate or reconnect long sessions periodically at natural conversation boundaries (with a concise text summary if continuity is required).
Fast Speech Rate and TTS Markup or IPA Leaks
Symptom:
- Spoken turns on
gemini-live-2.5-flash-native-audiorace past 240-300 WPM on long single-breath disclosures regardless of prompt instructions like "speak slowly". - On
gemini-3.8-live,output_transcriptionleaks raw IPA slashes (/ˌmɛdɪˈkeɪʃən/) or the model opens with off-script broadcast-style prose.
Root Cause:
gemini-live-2.5-flash-native-audiodoes not expose aspeaking_rateparameter; combining fast-cadence voices (such asCallirrhoe) withsetup.generationConfig.enableAffectiveDialog = trueand dense 50-word clauses accelerates speech.- Copying Gemini-TTS headers (
# Audio Profile,# Director's Notes) or raw IPA/.../pronunciation notation into a Live APIsystem_instructioncauses literal IPA leakage into transcripts and destabilizes conversational prosody.
Fix
- Upgrade to
gemini-3.8-live, which speaks naturally at a steadier conversational pace (~180-205 WPM; affective dialog is built-in by default, so omitenableAffectiveDialogongemini-3.8-live). - If remaining on
gemini-live-2.5-flash-native-audio, break long disclosures into 1-2 short sentences per turn, disablesetup.generationConfig.enableAffectiveDialog, and test steadier voices such asAoedeorPuck. - Remove
# Audio Profile/# Director's Notesheaders and replace IPA slashes with plain phonetic respellings (for example, write"ah-SET-uh-MIN-oh-fen"instead of IPA notation).
Near-Zero Audio Amplitude Collapse
Symptom: Full-length responses arrive with intact output_transcription and
normal turn duration, but the 24 kHz PCM waveform is nearly silent (~ -34 to
-44 dBFS, with peak amplitudes under 3% of normal full-scale speech).
Root Cause: Attaching massive tools declarations (>40,000 tokens or
100+ functions) or embedding pure-text rolling dialogue history blocks inside
system_instruction pushes the voice reference tokens tens of thousands of
tokens away from generation. Attention to the voice reference collapses,
producing near-zero-energy audio tokens while text generation remains intact.
Fix
- Prune
toolsinBidiGenerateContentSetupto only the active domain or workflow step (keep total tool schema size well under40,000tokens, ideally<20functions per session). - Keep
system_instructionconcise (<2,000tokens) and never inject pure-text multi-turn conversation logs insidesystem_instruction. - If multi-domain routing is needed, route in application code or expose a compact retrieval tool instead of attaching an entire catalog of schemas.
Tonal Humming Buzz (150-700 Hz) and Non-Terminating Greetings
Symptom:
- On numeric readbacks (phone numbers, verification codes) or opening
greetings, the model emits a sustained
150-700 Hzharmonic hum or endless silence without sendingturnComplete: true. - Audio output token counts spike
50x-600xrelative to text tokens. - Cancelling the stuck turn and sending a follow-up text nudge on the same WebSocket connection reproduces the hum >90% of the time.
- If the client locks the user microphone until the first bot
turnComplete, the call deadlocks permanently.
Root Cause: On gemini-live-2.5-flash-native-audio, setting low
temperature (<= 0.3) alongside
setup.generationConfig.enableAffectiveDialog = true traps the discrete audio
sampler in a low-entropy token-repetition attractor that never emits an
end-of-turn token. Because temperature <= 0.3 is near-deterministic and the
partial humming turn stays in session history, nudging on the same connection
replays the attractor.
Fix
- Keep
temperature >= 0.7(default1.0is recommended; never settemperature <= 0.3on native-audio sessions). - On
gemini-live-2.5-flash-native-audio, disablesetup.generationConfig.enableAffectiveDialog(enable_affective_dialogin the SDK) during triage and when reading structured digits/codes. (Ongemini-3.8-live, affective dialog is built-in by default — omit the flag.) - On any stalled or humming turn, close and open a new WebSocket session (or reconnect with a clean history) rather than nudging on the stalled connection.
- Never lock the user microphone indefinitely waiting for the first
turnComplete; add a short client timeout (3-5s) and avoid overly low VAD barge-in sensitivity (START_SENSITIVITY_LOW) on greeting gates.
Choppy Playback and Audio Jitter
Symptom: output_transcription is clean and complete, but spoken playback
sounds choppy with 0.1-0.5s dropouts between short bursts.
Root Cause: The server streams raw 24 kHz 16-bit PCM frames as they are
decoded, which can arrive in small or unevenly timed chunks (2-20+ bytes at
turn boundaries or under network jitter). Playing each WebSocket frame
immediately starves the client audio device buffer.
Fix
Add a ~200 ms client-side jitter buffer before starting playback for each
turn, concatenating incoming inlineData PCM chunks (audio/pcm;rate=24000)
into the audio ring buffer before draining to the speaker.