Grad-TTS developer on retainer: 300–500 ms near-silence followed by random phoneme noise burst at utterance end when input text ends with a URL or domain name — duration predictor assigns anomalously high duration to utterance-final continuant following known TLD phoneme sequence — TLD phoneme duration cap eliminates trailing silence-and-noise artifact — 3 weeks — Grad-TTS developer on monthly retainer

Grad-TTS (huawei-noah/Speech-Backbones Grad-TTS) score-based diffusion TTS • Python 3.11; PyTorch 2.1.x • voice assistant pipeline • standard text inputs synthesize at MOS 4.0 with clean audio termination • when input text ends with a URL or domain name (“visit example.com”, “go to amazon dot com”, “see github.com/readme”) Grad-TTS generates extra 300–500 ms of near-silence followed by short burst of random phoneme noise before audio terminates • artifact only when last phoneme of final token is a continuant (/m/, /n/, /l/, /r/, /v/) that happens to be the last character of a TLD or domain suffix • developer checks Grad-TTS diffusion parameters, text normalizer, and vocoder configuration for 3 weeks • root cause: Grad-TTS MAS duration predictor trained on conversational speech where utterances rarely end with TLD phoneme sequences; duration predictor assigns anomalously high duration to utterance-final /m/ when it follows /k ah/ (the “com” phoneme sequence) because a small number of training examples where speaker paused after saying “dot com” created a spurious long-duration correlation; diffusion process generates extended near-silence + noise burst for over-predicted frames • fix: add domain suffix duration cap to phoneme duration predictor post-processing; for utterances ending with known TLD phoneme sequences, clamp final phoneme duration to ≤80 ms • trailing silence: 300–500 ms → 0 ms • MOS: 4.0 → 4.0 (artifact eliminated, was 3.5 on URL-ending inputs) • 1.5h retainer engagement

Eastfield GradTTS Ltd. built a voice assistant pipeline using Grad-TTS as the TTS backend. For the vast majority of voice assistant queries, Grad-TTS synthesized high-quality speech (MOS 4.0) with clean audio termination. But a specific class of inputs revealed a consistent, reproducible artifact: whenever the voice assistant’s response text ended with a URL or domain name — a common occurrence in a web-integrated assistant (“visit example.com for more details”, “go to amazon.com to purchase”, “see github.com/readme for documentation”) — the synthesized audio exhibited a 300–500 ms near-silence period after the final spoken phoneme, followed by a brief burst of what sounded like garbled or random phonemes before the audio stream terminated. The artifact was consistently present, was proportional in length to the TLD (longer for “.org” or “.edu” than for “.io”), and was absent when the same URL appeared in the middle of an utterance rather than at the end. User feedback described it as “the voice sounds like it’s mumbling after the website address” and “there’s a weird sound at the end when it says a web address.”

The team initially checked whether the text normalizer was incorrectly expanding URLs — for instance, expanding “example.com” to include extra characters that the phoneme tokenizer was mishandling. The text normalizer was correct: it expanded “example.com” to “example dot com” and “github.com/readme” to “github dot com slash readme.” They also checked whether the Grad-TTS diffusion parameters needed adjustment: varying the number of diffusion steps (10, 20, 50, 100) had no effect on the trailing artifact. They confirmed the artifact was visible in the mel spectrogram as extra frames at the utterance end with anomalously low energy followed by a small high-frequency burst. After three weeks they engaged a Grad-TTS retainer developer who identified the root cause as a duration predictor anomaly: the MAS duration predictor was assigning 300–500 ms of duration to the final phoneme of TLD sequences, causing the diffusion model to generate mel frames for a 400 ms phoneme that should have lasted only 60–80 ms.

The Grad-TTS inference pipeline and duration predictor diagnosis at the time of the engagement:

# Eastfield GradTTS Ltd. — Grad-TTS inference pipeline with URL trailing artifact
# Grad-TTS (huawei-noah/Speech-Backbones Grad-TTS) score-based diffusion TTS
# Python 3.11, PyTorch 2.1.x

import torch
import numpy as np
import re
from gradtts import GradTTS, GradTTSConfig
from gradtts.text import phonemize, PHONEME_TO_ID

# Grad-TTS configuration:
# Architecture: score-based diffusion with monotonic alignment search (MAS)
# Duration predictor: MLP conditioned on phoneme embeddings + positional encoding
# Diffusion: 10-step inference (DDIM schedule)
# Vocoder: HiFi-GAN (22 kHz)
# Text normalizer: expands URLs, numbers, abbreviations before phonemization

model = GradTTS.from_pretrained("eastfield-tts/gradtts-en-v1")

def synthesize_buggy(text: str, n_timesteps: int = 10):
    """
    Synthesize text with Grad-TTS.
    Bug: for text ending with TLD/domain names, duration predictor assigns
    anomalously long duration to utterance-final continuant.
    """
    # Text normalization: expand URLs to spoken form
    # "visit example.com" → "visit example dot com"
    # "go to github.com/readme" → "go to github dot com slash readme"
    normalized = text_normalize(text)

    # Phonemization (espeak-ng via phonemizer):
    # "example dot com" → /ɪ ɡ z æ m p ə l d ɒ t k ɒ m/
    # Note: final /m/ is utterance-final
    phonemes = phonemize(normalized, language='en-us', backend='espeak')

    with torch.inference_mode():
        input_ids = torch.tensor([PHONEME_TO_ID[p] for p in phonemes]).unsqueeze(0)
        input_len  = torch.tensor([len(phonemes)])

        # MAS duration predictor:
        # Predicts mel frame count for each phoneme
        # Bug: for utterance-final /m/ after /k ɒ/ (the 'com' sequence),
        # duration predictor returns anomalously high frame count
        durations = model.duration_predictor(input_ids, input_len)

        # Score-based diffusion: generate mel spectrogram
        # Total mel frames = sum(durations)
        # Bug: total frames includes 300-500 ms of extra frames from TLD anomaly
        mel = model.diffusion(input_ids, durations, n_timesteps=n_timesteps)

        # HiFi-GAN vocoder: mel → waveform
        waveform = model.vocoder(mel)

    return waveform, durations

# Diagnosis: duration predictor output on URL-ending utterances
# Phoneme durations measured in mel frames (22 kHz, hop size 256 → 11.6 ms per frame)
#
# Test: "visit example dot com"
# Phoneme sequence: /v ɪ z ɪ t ɛ ɡ z æ m p ə l d ɒ t k ɒ m/
#
# Duration predictor output (mel frames, 1 frame = 11.6 ms):
#
#   Phoneme   Duration (frames)   Duration (ms)   Expected (ms)   Anomaly
#   /v/       4                   46 ms           40–60 ms        normal
#   /ɪ/       5                   58 ms           50–70 ms        normal
#   /z/       4                   46 ms           40–60 ms        normal
#   /ɪ/       4                   46 ms           40–60 ms        normal
#   /t/       4                   46 ms           40–60 ms        normal
#   /ɛ/       5                   58 ms           50–70 ms        normal
#   /ɡ/       4                   46 ms           40–60 ms        normal
#   /z/       4                   46 ms           40–60 ms        normal
#   /æ/       6                   70 ms           60–80 ms        normal
#   /m/       5                   58 ms           50–70 ms        normal
#   /p/       4                   46 ms           40–60 ms        normal
#   /ə/       4                   46 ms           40–60 ms        normal
#   /l/       5                   58 ms           50–70 ms        normal
#   /d/       4                   46 ms           40–60 ms        normal
#   /ɒ/       5                   58 ms           50–70 ms        normal
#   /t/       4                   46 ms           40–60 ms        normal
#   /k/       4                   46 ms           40–60 ms        normal
#   /ɒ/       5                   58 ms           50–70 ms        normal
#   /m/      37                  429 ms           60–80 ms        ← ANOMALY: 429 ms (5.5× expected)
#
# The utterance-final /m/ (from "com") is assigned 37 mel frames = 429 ms.
# Expected duration for /m/ in this position: 5–7 frames (58–81 ms).
# The anomalous duration causes the diffusion model to generate 37 frames of mel,
# but the score model has nothing to generate after the natural speech ends at frame 7:
# frames 8-37 are dominated by diffusion noise (low-energy spectrogram + high-freq burst).
#
# Test: "go to amazon dot com" — same anomaly on final /m/
#   Utterance-final /m/ after /k ɒ/: 34 frames = 394 ms (expected: 6 frames = 70 ms)
#
# Test: "see github dot com slash readme" — anomaly on /m/ in "com", NOT on utterance-final /iː/
#   "com" is NOT utterance-final here (followed by "slash readme")
#   Anomaly only triggers when TLD phoneme is THE LAST PHONEME in the utterance.
#
# Test: "visit example dot net" — anomaly on final /t/ from "net"
#   /t/ after /n ɛ/ (the 'net' sequence): 29 frames = 337 ms (expected: 4 frames = 46 ms)
#
# Pattern: anomaly triggers when utterance-final phoneme follows a recognized TLD phoneme sequence.
# Affected TLD phoneme sequences:
#   .com  → /k ɒ m/ — final /m/ anomalous
#   .net  → /n ɛ t/ — final /t/ anomalous
#   .org  → /ɔː r ɡ/ — final /ɡ/ anomalous (less severe: 180–240 ms)
#   .edu  → /ɛ d juː/ — final /uː/ anomalous (200–300 ms)
#   .io   → /aɪ əʊ/ — final /əʊ/ anomalous (120–180 ms)

print("Root cause: duration predictor assigns 37-frame anomaly to utterance-final /m/ after /k ɒ/")
print(".com TLD: 429 ms predicted vs 70 ms expected; .net: 337 ms predicted vs 46 ms expected")
print("Diffusion generates near-silence + noise burst for over-predicted frames")

Why Grad-TTS duration predictor assigns anomalously long durations to TLD-final phonemes

Grad-TTS uses monotonic alignment search to jointly train a duration predictor and an alignment model. The duration predictor is a small MLP that takes phoneme embeddings as input and outputs the number of mel frames to allocate for each phoneme. It is trained to minimize the mean squared error between predicted and ground-truth phoneme durations extracted by the forced alignment during training. The duration predictor’s output is conditioned on the local phoneme context, including several preceding and following phonemes. This context-conditioning is what creates the vulnerability to TLD phoneme sequences: the duration of utterance-final /m/ is predicted not just from /m/’s identity but from the phoneme context that includes /k ɒ/ (the preceding “com” phonemes) and the absence of any following phoneme (indicating utterance-final position).

The Grad-TTS training corpus consisted of conversational speech datasets (LJSpeech, LibriTTS, VCTK) where utterances are sentences and sentence fragments from books, articles, and conversations. In these corpora, utterances ending with the phoneme sequence /k ɒ m/ at utterance-final position (i.e., sentences ending with the word “come” or “com”) are rare. The small number of training examples that do match this pattern happened to include several recordings where the speaker paused after finishing the utterance but before the recording was cut: these recordings contain 300–600 ms of trailing silence within the audio segment, which the forced aligner attributes to the final phoneme /m/ as an anomalously long duration. The duration predictor, trained on these examples, learns the spurious correlation: “utterance-final /m/ following /k ɒ/ has duration ~400 ms.” In the training data for most utterance-final /m/ cases (the word “come” in “here we come” etc.), /m/ duration is 60–80 ms. But the TLD-specific pattern overrides this for the /k ɒ m/ context because the duration predictor is a gradient-descent-trained function approximator, not a rule-based system.

The fix is a post-processing duration cap applied after the duration predictor runs. For utterances whose final phoneme sequence matches a known TLD phoneme pattern, the final phoneme duration is clamped to a maximum of 80 ms (7 mel frames at 11.6 ms/frame). This cap is applied before the diffusion model runs, so the diffusion model only generates mel frames for the physically plausible phoneme duration and never reaches the noise-dominated tail frames. The fix is entirely in post-processing — no model retraining required, and no effect on non-TLD inputs.

# Fix: TLD phoneme duration cap in Grad-TTS duration predictor post-processing
# Clamps utterance-final phoneme duration to ≤80 ms when TLD phoneme sequence detected

import torch
import numpy as np
from gradtts import GradTTS
from gradtts.text import phonemize, PHONEME_TO_ID

model = GradTTS.from_pretrained("eastfield-tts/gradtts-en-v1")

# TLD phoneme sequences (IPA, as produced by espeak-ng for English-pronounced TLDs):
# Pattern: tuple of phonemes that must appear as the utterance-final sequence.
# When matched, the LAST phoneme's duration is capped to MAX_FINAL_PHONEME_FRAMES.

TLD_PHONEME_PATTERNS = {
    # .com  → "dot com"  → /d ɒ t k ɒ m/   — cap final /m/
    ".com":  ("k", "ɒ", "m"),
    # .net  → "dot net"  → /d ɒ t n ɛ t/   — cap final /t/
    ".net":  ("n", "ɛ", "t"),
    # .org  → "dot org"  → /d ɒ t ɔː r ɡ/ — cap final /ɡ/
    ".org":  ("ɔː", "r", "ɡ"),
    # .edu  → "dot edu"  → /d ɒ t ɛ d juː/ — cap final /uː/
    ".edu":  ("ɛ", "d", "juː"),
    # .io   → "dot i o"  → /d ɒ t aɪ əʊ/  — cap final /əʊ/
    ".io":   ("aɪ", "əʊ"),
    # .gov  → "dot gov"  → /d ɒ t ɡ ɒ v/  — cap final /v/
    ".gov":  ("ɡ", "ɒ", "v"),
    # .uk   → "dot u k"  → /d ɒ t juː k/  — cap final /k/
    ".uk":   ("juː", "k"),
    # .co   → "dot co"   → /d ɒ t k əʊ/  — cap final /əʊ/
    ".co":   ("k", "əʊ"),
}

# Maximum duration for TLD-final phoneme: 80 ms
# 22 kHz / hop 256 = 85.9 frames/sec → 1 frame = 11.6 ms
# 80 ms / 11.6 ms/frame = 6.9 → cap at 7 frames
MAX_FINAL_PHONEME_FRAMES = 7   # 7 × 11.6 ms = 81.2 ms

def apply_tld_duration_cap(phonemes: list, durations: torch.Tensor) -> torch.Tensor:
    """
    Apply duration cap to utterance-final phoneme when TLD phoneme sequence detected.

    phonemes:  list of IPA phoneme strings, e.g. ['v', 'ɪ', 'z', ..., 'k', 'ɒ', 'm']
    durations: tensor [1, T] of predicted mel frame counts for each phoneme
    Returns:   durations tensor with final phoneme capped if TLD pattern matched
    """
    durations_capped = durations.clone()

    for tld_name, pattern in TLD_PHONEME_PATTERNS.items():
        pattern_len = len(pattern)
        if len(phonemes) >= pattern_len:
            # Check if utterance ends with this TLD phoneme pattern
            utterance_tail = tuple(phonemes[-pattern_len:])
            if utterance_tail == pattern:
                # Cap duration of final phoneme (last element of pattern)
                final_phoneme_idx = len(phonemes) - 1
                original_duration = durations_capped[0, final_phoneme_idx].item()

                if original_duration > MAX_FINAL_PHONEME_FRAMES:
                    durations_capped[0, final_phoneme_idx] = MAX_FINAL_PHONEME_FRAMES
                    print(f"[TLD duration cap] TLD pattern '{tld_name}' matched: "
                          f"final /{phonemes[-1]}/ duration "
                          f"{original_duration} → {MAX_FINAL_PHONEME_FRAMES} frames "
                          f"({original_duration * 11.6:.0f} ms → {MAX_FINAL_PHONEME_FRAMES * 11.6:.0f} ms)")

    return durations_capped

def synthesize_fixed(text: str, n_timesteps: int = 10):
    """
    Synthesize text with Grad-TTS + TLD duration cap post-processing.
    Trailing silence + noise artifact eliminated for URL-ending utterances.
    """
    normalized = text_normalize(text)
    phonemes   = phonemize(normalized, language='en-us', backend='espeak')

    with torch.inference_mode():
        input_ids = torch.tensor([PHONEME_TO_ID[p] for p in phonemes]).unsqueeze(0)
        input_len  = torch.tensor([len(phonemes)])

        # MAS duration predictor (unchanged)
        durations_raw = model.duration_predictor(input_ids, input_len)

        # Apply TLD duration cap post-processing (fix)
        durations_fixed = apply_tld_duration_cap(phonemes, durations_raw)

        # Score-based diffusion with capped durations
        mel = model.diffusion(input_ids, durations_fixed, n_timesteps=n_timesteps)

        waveform = model.vocoder(mel)

    return waveform

# ---- Results after TLD duration cap ----
#
# Duration predictor output (mel frames) — fixed vs buggy:
#
#   Input: "visit example dot com"
#   Utterance-final /m/ after /k ɒ/:
#     Buggy:  37 frames = 429 ms   ← anomaly
#     Fixed:   7 frames =  81 ms   ← capped to MAX_FINAL_PHONEME_FRAMES
#
#   Input: "go to amazon dot com"
#   Utterance-final /m/ after /k ɒ/:
#     Buggy:  34 frames = 394 ms
#     Fixed:   7 frames =  81 ms   ← capped
#
#   Input: "see github dot net"
#   Utterance-final /t/ after /n ɛ/:
#     Buggy:  29 frames = 337 ms
#     Fixed:   7 frames =  81 ms   ← capped (note: /t/ cap; MAX same for all TLD finals)
#
# Trailing silence measurements (50 URL-ending utterances):
#   Buggy:  trailing silence 342 ms ± 78 ms (range: 280–520 ms); noise burst present
#   Fixed:  trailing silence   8 ms ±  4 ms (range: 0–18 ms); no noise burst
#
# MOS evaluation (20-listener crowdsource, 50 utterances per condition):
#   Grad-TTS on standard (non-URL) inputs:
#     Buggy:  MOS 4.0 (σ=0.2) — no artifact on standard inputs
#     Fixed:  MOS 4.0 (σ=0.2) — no regression from TLD cap (non-URL inputs unaffected)
#
#   Grad-TTS on URL-ending inputs:
#     Buggy:  MOS 3.5 (σ=0.5) — "mumbling after website address", "weird sound at end"
#     Fixed:  MOS 4.0 (σ=0.2) — clean termination; no artifact; matches standard input quality
#
# TLD coverage validation (10 TLD patterns × 5 test sentences each = 50 test cases):
#   .com   5/5 fixed   .net  5/5 fixed   .org  5/5 fixed   .edu  5/5 fixed
#   .io    5/5 fixed   .gov  5/5 fixed   .uk   5/5 fixed   .co   5/5 fixed
#
# Non-TLD regression test (100 standard sentences, no TLD pattern at end):
#   Duration cap never triggered (no false positives)
#   MOS: 4.0 on all 100 standard sentences — no regression

print("Fix: TLD phoneme duration cap ≤80 ms (7 frames at 11.6 ms/frame)")
print("Trailing silence: 342 ms → 8 ms; MOS on URL inputs: 3.5 → 4.0; no regression")

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →