FastPitch developer on retainer: hyphenated compound words produce 10–15 Hz pitch undershoot on post-hyphen word — espeak-ng assigns secondary stress to post-hyphen components in compound words causing FastPitch pitch predictor to generate lower F0 targets — hyphen-to-space pre-processing before G2P eliminates pitch undershoot — 3 weeks — FastPitch developer on monthly retainer

FastPitch (NVIDIA/FastPitch) parallel TTS with explicit pitch conditioning • Python 3.11; PyTorch 2.1.x • real-time audiobook narration service • hyphenated compound words (“state-of-the-art”, “real-time”, “end-to-end”) produce synthesized speech where the word after the hyphen has pitch 10–15 Hz lower than surrounding context — post-hyphen word sounds unexpectedly flat like it is being de-emphasized • standard words without hyphens synthesize with correct pitch • developer checks FastPitch pitch predictor, attention alignment, and text preprocessor for 3 weeks • root cause: espeak-ng assigns secondary lexical stress to post-hyphen components in compound words; FastPitch pitch predictor, trained on espeak-ng stress annotations, generates F0 targets 10–15 Hz lower for secondary-stress phonemes; fix: replace hyphens with spaces before G2P so espeak-ng assigns primary stress to each component word • post-hyphen pitch undershoot: 10–15 Hz → <1 Hz • MOS: 3.7 → 4.0 • 1.5h retainer engagement

Westfield FastPitch Ltd. built a real-time audiobook narration service using FastPitch as the TTS backbone. The service narrated technical books, instructional guides, and business publications in custom synthesized voices. For the vast majority of narrated text, FastPitch produced speech with correct pitch and natural prosody. But a consistent quality issue appeared across technical and business content: whenever the input text contained a hyphenated compound word — constructions like “state-of-the-art”, “real-time processing”, “end-to-end pipeline”, “off-the-shelf solution”, or “high-performance computing” — the word or phrase element following the hyphen was consistently synthesized 10–15 Hz lower in pitch than its surrounding context. The post-hyphen word sounded flat and de-emphasized, as if the speaker had deliberately reduced emphasis on that word mid-phrase. For audiobook narration, where natural prosody is critical to listener comprehension and engagement, this artifact was noticeable and degraded the perceived quality of technical content, which is dense with hyphenated compound terminology.

The team initially checked whether FastPitch’s attention mechanism was misaligning on hyphen characters in the input, causing duration and pitch predictions to shift for the post-hyphen phonemes. They inspected the attention heatmaps for sentences containing compound words and found no misalignment — attention was smooth across the hyphen positions. They then checked whether the pitch predictor was receiving incorrect input features for hyphenated words and whether the HiFi-GAN vocoder was introducing pitch artifacts during waveform generation. Neither showed anomalous behavior. After three weeks they engaged a FastPitch retainer developer who identified the root cause as espeak-ng’s compound word stress assignment: the hyphen was causing espeak-ng to apply secondary stress to post-hyphen components, and FastPitch’s pitch predictor had learned to generate lower F0 for secondary-stress phonemes.

The FastPitch inference pipeline and stress annotation diagnostic at the time of the engagement:

# Westfield FastPitch Ltd. — FastPitch compound-word pitch undershoot
# FastPitch (NVIDIA/FastPitch) parallel TTS with explicit pitch conditioning
# Python 3.11, PyTorch 2.1.x
# Deployment: audiobook narration service, technical documentation domain

import torch
import numpy as np
from phonemizer.backend import EspeakBackend
from nemo.collections.tts.models import FastPitchModel
from nemo.collections.tts.models import HifiGanModel

fastpitch = FastPitchModel.from_pretrained("westfield-tts/fastpitch-audiobook-v2")
hifigan   = HifiGanModel.from_pretrained("westfield-tts/hifigan-audiobook-v2")

# Inspect espeak-ng stress annotation for compound vs non-compound form
backend = EspeakBackend('en-us', preserve_punctuation=False, with_stress=True)

# Hyphenated compound: "state-of-the-art analysis"
phonemes_hyphenated = backend.phonemize(["state-of-the-art analysis"])[0]
# Output (IPA with stress markers — '1' = primary, '2' = secondary):
# "steɪt1-ʌv-ðə-ɑːt2 ænəlɪsɪs1"
#  ^^^^ primary         ^^^ secondary ← 'art' in compound gets secondary stress
#
# Non-hyphenated: "state of the art analysis"
phonemes_space = backend.phonemize(["state of the art analysis"])[0]
# Output:
# "steɪt1 ʌv ðə ɑːt1 ænəlɪsɪs1"
#  ^^^^                ^^^ primary ← 'art' as independent word gets primary stress

# FastPitch pitch predictor response to stress annotation differences
# (conceptual — FastPitch pitch predictor is a feed-forward network
#  conditioned on phoneme embeddings that include stress markers)
#
# Phoneme embedding for 'ɑː' with stress=1 (primary):   → pitch target ~185 Hz
# Phoneme embedding for 'ɑː' with stress=2 (secondary): → pitch target ~171 Hz
# Difference: 14 Hz — matches observed 10–15 Hz undershoot range
#
# Verification: extract per-word F0 from synthesized audio using CREPE pitch tracker

import crepe

def measure_word_f0(audio: np.ndarray, sr: int, word_start_s: float,
                    word_end_s: float) -> float:
    """Return mean F0 (Hz) for a word time interval in synthesized audio."""
    start_frame = int(word_start_s * sr)
    end_frame   = int(word_end_s   * sr)
    word_audio  = audio[start_frame:end_frame]
    _, frequency, confidence, _ = crepe.predict(word_audio, sr, viterbi=True)
    voiced = frequency[confidence > 0.5]
    return float(np.mean(voiced)) if len(voiced) > 0 else 0.0

# Synthesize "This delivers state-of-the-art analysis in real-time."
text_buggy = "This delivers state-of-the-art analysis in real-time."

with torch.inference_mode():
    parsed   = fastpitch.parse(text_buggy)
    spec, _  = fastpitch.generate_spectrogram(tokens=parsed)
    audio    = hifigan.convert_spectrogram_to_audio(spec=spec)

audio_np = audio.squeeze().cpu().numpy()

# Word timing from FastPitch duration predictor (approximate):
#   "This"      0.00–0.12 s
#   "delivers"  0.12–0.42 s
#   "state"     0.42–0.62 s   ← compound element 1 (primary stress)
#   "of"        0.62–0.72 s
#   "the"       0.72–0.80 s
#   "art"       0.80–1.00 s   ← compound element 4 (secondary stress from hyphen)
#   "analysis"  1.00–1.42 s
#   "in"        1.42–1.55 s
#   "real"      1.55–1.72 s   ← compound element 1 (primary stress)
#   "time"      1.72–1.95 s   ← compound element 2 (secondary stress from hyphen)

f0_state    = measure_word_f0(audio_np, 22050, 0.42, 0.62)   # primary stress
f0_art      = measure_word_f0(audio_np, 22050, 0.80, 1.00)   # secondary stress
f0_real     = measure_word_f0(audio_np, 22050, 1.55, 1.72)   # primary stress
f0_time     = measure_word_f0(audio_np, 22050, 1.72, 1.95)   # secondary stress

print(f"'state' F0 (primary stress):     {f0_state:.1f} Hz")   # ~187 Hz
print(f"'art'   F0 (secondary stress):   {f0_art:.1f} Hz")     # ~173 Hz ← 14 Hz low
print(f"'real'  F0 (primary stress):     {f0_real:.1f} Hz")    # ~185 Hz
print(f"'time'  F0 (secondary stress):   {f0_time:.1f} Hz")    # ~171 Hz ← 14 Hz low
print(f"Undershoot 'art':  {f0_state - f0_art:.1f} Hz")        # ~14 Hz
print(f"Undershoot 'time': {f0_real - f0_time:.1f} Hz")        # ~14 Hz

Why espeak-ng assigns secondary stress to post-hyphen components and why FastPitch pitch predictor follows espeak-ng stress levels

English compound word stress is governed by a consistent phonological rule: in a hyphenated compound, the first element receives primary stress and subsequent elements receive secondary (or tertiary, in longer compounds) stress. The linguistic motivation is that the hyphen signals that the elements form a single semantic unit, and English stress assignment for compound nouns and adjectives places the primary prominence on the leftmost content morpheme. The espeak-ng lexicon and stress assignment engine implements this rule: when espeak-ng encounters a hyphen-separated string, it applies compound stress rules rather than treating each element as an independent word. Words appearing after a hyphen receive a secondary stress marker (IPA stress symbol ‘ˌ’, or stress level 2 in phonemizer’s annotation scheme) rather than the primary stress marker (‘ˈ’, stress level 1) they would receive if the same word appeared independently in a phrase.

FastPitch’s pitch predictor was trained on the LJSpeech corpus using espeak-ng for G2P. The ground-truth F0 values in LJSpeech naturally reflect the speaker’s prosody, where secondary-stress syllables are spoken at measurably lower pitch than primary-stress syllables. Through training, FastPitch’s pitch predictor network has learned a strong association between the stress marker in the phoneme embedding and the predicted F0 target: primary stress (marker 1) maps to the sentence-mean F0 or above, secondary stress (marker 2) maps to 10–15 Hz below the sentence mean. This is the correct behavior for the LJSpeech domain, where compound words appear with their natural prosodic reduction. But in an audiobook narration context where technical compound words like “state-of-the-art” are intended to be spoken with full prosodic weight on each component, the espeak-ng secondary stress assignment causes FastPitch to generate inappropriately reduced F0 targets for the post-hyphen words.

# Fix: replace hyphens with spaces in compound words before FastPitch G2P
# Causes espeak-ng to assign primary stress to each component word independently

import re
import torch
from phonemizer.backend import EspeakBackend
from nemo.collections.tts.models import FastPitchModel, HifiGanModel

fastpitch = FastPitchModel.from_pretrained("westfield-tts/fastpitch-audiobook-v2")
hifigan   = HifiGanModel.from_pretrained("westfield-tts/hifigan-audiobook-v2")

# Compound word hyphen normalization
# Replace hyphens between word characters with spaces so each component
# gets independent lexical stress assignment from espeak-ng.
#
# Pattern matches hyphen between word-character sequences (letters, digits).
# Preserves hyphens used as minus signs, en-dashes, em-dashes (already handled
# by upstream Unicode normalization — not word-character boundaries).
COMPOUND_HYPHEN_RE = re.compile(r'(?<=\w)-(?=\w)')

# Allow-list: patterns where hyphen should be preserved (not expanded to space)
# Examples: "O-negative" (blood type), "T-shirt" (letter prefix), "X-ray",
#           "COVID-19" (acronym-number), "co-founder" (Latin prefix)
PRESERVE_HYPHEN_PATTERNS = [
    re.compile(r'\b[A-Z]-\w'),              # single uppercase letter prefix (T-shirt, X-ray)
    re.compile(r'\b(?:co|pre|post|re|non|anti|self|over|under)-\w', re.IGNORECASE),  # common prefixes
    re.compile(r'\b\w+-\d+\b'),             # word-number (COVID-19, Phase-3)
    re.compile(r'\b[A-Z0-9]+-[A-Z0-9]+\b'), # acronym compounds (BERT-Large, GPT-4o)
]

def should_preserve_hyphen(match: re.Match, text: str) -> bool:
    """Return True if the hyphen at match position should be preserved."""
    # Check a small window around the match for allow-list patterns
    start = max(0, match.start() - 15)
    end   = min(len(text), match.end() + 15)
    snippet = text[start:end]
    return any(pat.search(snippet) for pat in PRESERVE_HYPHEN_PATTERNS)

def normalize_compound_hyphens(text: str) -> str:
    """
    Replace compound-word hyphens with spaces so espeak-ng assigns
    primary stress to each component word.
    Preserves hyphens in allow-listed patterns (prefixes, acronyms, etc.).
    """
    result = []
    last_end = 0
    for match in COMPOUND_HYPHEN_RE.finditer(text):
        result.append(text[last_end:match.start()])
        if should_preserve_hyphen(match, text):
            result.append('-')   # preserve this hyphen
        else:
            result.append(' ')   # replace with space
        last_end = match.end()
    result.append(text[last_end:])
    return ''.join(result)

def synthesize_fixed(text: str) -> torch.Tensor:
    """Synthesize with FastPitch + compound hyphen normalization fix."""
    text_normalized = normalize_compound_hyphens(text)

    with torch.inference_mode():
        parsed  = fastpitch.parse(text_normalized)
        spec, _ = fastpitch.generate_spectrogram(tokens=parsed)
        audio   = hifigan.convert_spectrogram_to_audio(spec=spec)

    return audio

# ---- Results after compound hyphen normalization fix ----
#
# Input: "This delivers state-of-the-art analysis in real-time."
# After normalization: "This delivers state of the art analysis in real time."
#
# Per-word F0 measurement (CREPE, 100 technical sentences):
#
# Compound post-hyphen word F0:
#   Buggy:  sentence_mean - 13.4 Hz (σ=1.8 Hz) — 10–15 Hz below mean
#   Fixed:  sentence_mean -  0.6 Hz (σ=0.9 Hz) — within 1 Hz of mean
#
# MOS evaluation (20-listener crowdsource, 50 technical compound sentences):
#   Buggy:  MOS 3.7 (σ=0.5) — "some words sound flat"
#   Fixed:  MOS 4.0 (σ=0.3) — "natural, engaging narration"
#
# Standard prose (control, no compound words):
#   Before fix: MOS 4.0  →  After fix: MOS 4.0 (no regression)
#
# Allow-list validation (hyphens correctly preserved):
#   "T-shirt":    hyphen preserved ✓
#   "X-ray":      hyphen preserved ✓
#   "co-founder": hyphen preserved ✓
#   "COVID-19":   hyphen preserved ✓
#   "BERT-Large": hyphen preserved ✓
#
# Compound words correctly expanded (hyphens to spaces):
#   "state-of-the-art":       "state of the art"  ✓
#   "real-time":              "real time"          ✓
#   "end-to-end":             "end to end"         ✓
#   "off-the-shelf":          "off the shelf"      ✓
#   "high-performance":       "high performance"   ✓
#   "machine-learning-based": "machine learning based" ✓

print("Fix: compound hyphen normalization before G2P (hyphens → spaces)")
print("Post-hyphen pitch undershoot: 13.4 Hz → 0.6 Hz; MOS: 3.7 → 4.0; 1.5h")

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →