MetaVoice developer on retainer: periodic “click” artifact every 1 second in all synthesized outputs — autoregressive model learns 1-second EnCodec segment boundary pattern from training data — Hann-window crossfade at 86-token boundary eliminates click — 3 weeks — MetaVoice developer on monthly retainer

MetaVoice-1B (metavoiceio/metavoice-src) 1B parameter zero-shot TTS • Python 3.11; PyTorch 2.1.x • custom voice creation service • all synthesized outputs: periodic 2–4 ms broadband “click” transient occurring at approximately 1-second intervals (±50 ms) • click present across all reference speakers, all target texts, all generation parameters • click rate: 7 clicks per 10 seconds • developer checks MetaVoice-1B configuration, EnCodec decoder parameters, generation temperature for 3 weeks • root cause: MetaVoice-1B generates EnCodec codec tokens at 86.1 Hz; at every 86th token (approximately every 1 second) the autoregressive model encounters a positional encoding corresponding to a 1-second codec segment boundary in the training data; training audio was split at 1-second intervals for codec encoding, introducing an amplitude discontinuity at each split; autoregressive model learned this 1-second periodic boundary as a statistical pattern and reproduces it at inference • fix: apply 4 ms Hann-window crossfade at each 1-second (86-token) boundary in the generated token sequence before EnCodec decoding • click rate: 7/10s → 0.1/10s • MOS: 3.6 → 4.0 • 1.5h retainer engagement

Eastfield MetaVoice TTS Ltd. built a custom voice creation service using MetaVoice-1B for zero-shot voice cloning, allowing customers to upload a 10-second reference audio clip and immediately receive a synthesized output in their own voice. Voice similarity was strong and the product launched successfully, but users quickly reported an audio quality issue: every synthesized output had a periodic clicking sound occurring at regular approximately 1-second intervals throughout the generated audio. The click was a brief 2–4 ms broadband transient with frequency content up to 8 kHz. It was present regardless of which reference speaker was used, what text was being synthesized, or what generation temperature or sampling parameters were applied. The consistent periodicity at approximately 1-second intervals suggested a systematic structural artifact rather than random noise or a hardware issue.

The team investigated the entire pipeline: they verified the audio hardware and drivers produced no clicks with other audio sources, checked the EnCodec decoder for artifacts by passing known-clean codec token sequences through it, and varied every MetaVoice-1B generation parameter (temperature from 0.7 to 1.3, top-p from 0.8 to 1.0, number of samples). The click persisted in every configuration. They also tested whether the click was introduced by the EnCodec decoder itself — passing codec tokens derived from clean reference audio through the decoder produced no click, confirming the artifact was in the token sequences generated by the autoregressive model, not in the decoder. After three weeks, they engaged a MetaVoice retainer developer who identified the 1-second periodicity as a codec segment boundary artifact learned from the training data segmentation.

The MetaVoice-1B synthesis pipeline at the time of the engagement:

# Eastfield MetaVoice TTS Ltd. — voice creation synthesis pipeline
# MetaVoice-1B (metavoiceio/metavoice-src), Python 3.11, PyTorch 2.1.x
# Use case: zero-shot custom voice synthesis from 10-second reference audio

import torch
import numpy as np
import torchaudio
from metavoice.inference import MetaVoice

model = MetaVoice.from_pretrained("metavoiceio/metavoice-1B-v0.1")

def synthesize_buggy(text, reference_path, temperature=1.0, top_p=0.95):
    """
    Synthesize speech in the reference speaker's voice.
    Bug: periodic 2–4 ms click transient every ~1 second in output.
    """
    reference_wav, sr = torchaudio.load(reference_path)
    if sr != 24000:
        reference_wav = torchaudio.functional.resample(reference_wav, sr, 24000)

    # MetaVoice-1B generates EnCodec codec tokens autoregressively
    # at 86.1 Hz token rate. Architecture:
    # 1. Reference audio → EnCodec encoder → reference codec tokens
    # 2. Autoregressive transformer generates target codec tokens
    #    conditioned on reference tokens + target text
    # 3. EnCodec decoder → target codec tokens → synthesized waveform
    #
    # Bug: training data was segmented into 1-second chunks for codec encoding.
    # Each chunk was encoded independently by EnCodec:
    #   chunk_1 (0–1s) → EnCodec → tokens[0:86]
    #   chunk_2 (1–2s) → EnCodec → tokens[86:172]
    #   ...
    # EnCodec uses causal convolutions; boundary conditions differ between
    # independently-encoded chunks vs a single continuous encoding.
    # This creates a small amplitude discontinuity at each 1-second split:
    #   decoded(tokens[0:86]) → waveform with amplitude A at t=1.0s
    #   decoded(tokens[86:172]) → waveform starting at amplitude B ≠ A at t=1.0s
    # The discontinuity is a 2–4 ms broadband transient: a "click"
    # Autoregressive model trained on these sequences learns the 86-token
    # periodic pattern and reproduces it at inference.

    with torch.no_grad():
        generated_tokens = model.generate(
            text=text,
            reference_audio=reference_wav,
            temperature=temperature,
            top_p=top_p,
        )
        # generated_tokens contains periodic boundary artifact at positions
        # 86, 172, 258, ... (multiples of 86 tokens = 1-second intervals)

        waveform = model.decode(generated_tokens)
        # Decoded waveform has click at t = 1s, 2s, 3s, ... (every 86 tokens)

    return waveform

# Diagnosis: click timing analysis
# Measured over 50 synthesized outputs (5 reference speakers × 10 target texts)
# Detection: RMS spike detector, window 1 ms, threshold > 10× local background
#
# Click detection results:
#   Mean clicks per 10 seconds: 7.1 (σ = 0.6)
#   Mean click timing deviation from 1-second intervals: 23 ms (σ = 18 ms)
#   Click duration: 2–4 ms (consistently broadband, up to 8 kHz)
#   Click independence from reference speaker: confirmed (ρ = 0.02)
#   Click independence from temperature (0.7–1.3): confirmed (ρ = -0.04)
#
# Token position analysis:
#   Autoregressive token positions exhibiting amplitude spike: 86, 172, 258, ...
#   Pattern: exactly multiples of 86 tokens (86.1 Hz × 1s = 86.1 tokens/s)
#   Deviation from exact multiple: 0–3 tokens (explains ±50 ms timing jitter)
#
# Control: EnCodec decode of reference audio tokens (no autoregressive generation):
#   Clicks per 10 seconds: 0.0 — no clicks in reference decode
#   Confirms: artifact is in generated token sequence, not in EnCodec decoder

print("Root cause: 1-second codec segment boundary pattern in training data")
print("Autoregressive model learns 86-token periodic boundary → reproduces at inference")
print("EnCodec decoder reveals click because token amplitude discontinuity at every 86th token")

Why MetaVoice-1B’s autoregressive model learns the 1-second codec segment boundary pattern

MetaVoice-1B’s training process requires converting the training audio corpus into EnCodec codec token sequences that the autoregressive transformer can learn from. When audio is long (many seconds), it must be segmented into manageable chunks for codec encoding. The common approach — and the approach used in MetaVoice-1B’s training data preparation — is to split audio at fixed 1-second intervals and encode each chunk independently with EnCodec. This is computationally convenient and produces consistent-length token sequences for batched training.

However, EnCodec uses causal convolutional layers whose boundary conditions are sensitive to context. When a 1-second audio chunk is encoded in isolation, the convolutional encoder does not have access to the audio that immediately follows the chunk’s end. When an adjacent chunk is then encoded, the encoder also lacks context from the audio immediately before its start. This context mismatch creates a small amplitude discontinuity at the join point: the final few milliseconds of the first chunk’s decoded reconstruction and the first few milliseconds of the second chunk’s decoded reconstruction have slightly different amplitude levels, because each was reconstructed using codec tokens derived from boundary-condition-mismatched encodings. The discontinuity is typically 2–4 ms in duration and has broadband spectral content because it is an abrupt amplitude transition — a click.

The autoregressive transformer, trained on tens of thousands of token sequences derived from this segmentation scheme, encounters this acoustic pattern at every 86th token position in every training sequence. The model’s self-attention layers learn that the token distribution at positions that are multiples of 86 is statistically different from adjacent positions — specifically, that these positions have higher-amplitude broadband tokens corresponding to the click transient. At inference time, the model reproduces this learned statistical pattern: at every 86th generated token, it produces a token that decodes to an audio frame with the same click artifact, regardless of the input text or reference speaker.

The fix is a post-processing step applied to the generated token sequence after autoregressive generation but before EnCodec decoding. A 4 ms Hann-window crossfade is applied centered on each multiple-of-86 position in the token sequence. The Hann window tapers the token amplitude distribution smoothly through the boundary position, interpolating between the pre-boundary and post-boundary distributions over a 4 ms window. The resulting smoothed token sequence, when decoded by EnCodec, produces audio without the amplitude discontinuity that created the click.

# Fix: 4 ms Hann-window crossfade at each 86-token boundary before EnCodec decode
# Smooths amplitude discontinuity from learned codec segment boundary pattern

import torch
import numpy as np
import torchaudio
from metavoice.inference import MetaVoice

model = MetaVoice.from_pretrained("metavoiceio/metavoice-1B-v0.1")

ENCODEC_TOKEN_RATE_HZ = 86.1    # EnCodec codec token frame rate
TOKENS_PER_SECOND = 86          # Round number for boundary detection
CROSSFADE_DURATION_MS = 4.0     # Hann window duration at each boundary
CROSSFADE_TOKENS = 2            # Tokens on each side of boundary to crossfade
                                 # (4 ms / (1000 ms/86 tokens) ≈ 0.34 tokens → use 2 tokens each side)

def apply_boundary_crossfade(generated_tokens):
    """
    Apply 4 ms Hann-window crossfade at each 1-second (86-token) boundary.

    generated_tokens: tensor of shape [n_codebooks, n_tokens]
                      EnCodec codebook token indices

    For each codebook independently, linearly interpolate the amplitude-
    equivalent token values through each 86-token boundary using a Hann
    window to smooth the learned boundary statistical pattern.

    Note: 'token values' here means the continuous latent embeddings
    of the tokens (before quantization index lookup), not the integer indices.
    MetaVoice-1B's generate() returns continuous latents pre-quantization.
    """
    n_codebooks, n_tokens = generated_tokens.shape
    tokens_float = generated_tokens.float()

    # Find all 86-token boundary positions
    boundary_positions = list(range(TOKENS_PER_SECOND,
                                     n_tokens - CROSSFADE_TOKENS,
                                     TOKENS_PER_SECOND))

    for boundary in boundary_positions:
        # Window: 4 tokens centered on boundary (2 before, 2 after)
        # Apply Hann fade: pre-boundary side fades out, post-boundary side fades in
        hann_window = torch.hann_window(CROSSFADE_TOKENS * 2, periodic=False)
        fade_out = hann_window[:CROSSFADE_TOKENS]        # 1.0 → 0.0
        fade_in  = hann_window[CROSSFADE_TOKENS:]        # 0.0 → 1.0

        # Tokens just before boundary
        pre_start = max(0, boundary - CROSSFADE_TOKENS)
        pre_end   = boundary
        # Tokens just after boundary
        post_start = boundary
        post_end   = min(n_tokens, boundary + CROSSFADE_TOKENS)

        pre_tokens  = tokens_float[:, pre_start:pre_end]   # [n_cb, CROSSFADE_TOKENS]
        post_tokens = tokens_float[:, post_start:post_end] # [n_cb, CROSSFADE_TOKENS]

        # Crossfade: interpolate pre-boundary toward post-boundary values
        # This smooths the amplitude discontinuity over the 4-token window
        n_cross = min(pre_tokens.shape[1], post_tokens.shape[1], CROSSFADE_TOKENS)
        for i in range(n_cross):
            alpha = fade_in[i].item()  # 0 → 1 across the window
            tokens_float[:, pre_start + i] = (
                (1.0 - alpha) * pre_tokens[:, i] +
                alpha         * post_tokens[:, i]
            )

    return tokens_float

def synthesize_fixed(text, reference_path, temperature=1.0, top_p=0.95):
    """
    Synthesize speech with 86-token boundary crossfade applied before decode.
    Periodic click artifact eliminated.
    """
    reference_wav, sr = torchaudio.load(reference_path)
    if sr != 24000:
        reference_wav = torchaudio.functional.resample(reference_wav, sr, 24000)

    with torch.no_grad():
        generated_tokens = model.generate(
            text=text,
            reference_audio=reference_wav,
            temperature=temperature,
            top_p=top_p,
        )

        # Apply boundary crossfade before decoding
        # This smooths the learned periodic boundary pattern in token space
        smoothed_tokens = apply_boundary_crossfade(generated_tokens)

        waveform = model.decode(smoothed_tokens)
        # Decoded waveform: click at 1s, 2s, 3s, ... eliminated

    return waveform

# ---- Results after 86-token boundary crossfade ----
#
# Click detection (50 synthesized outputs, same test set as before):
#   Before fix: mean 7.1 clicks per 10 seconds (σ = 0.6)
#   After fix:  mean 0.1 clicks per 10 seconds (σ = 0.2) ← near-zero
#               Remaining 0.1/10s is background acoustic transients, not codec boundary
#
# MOS evaluation (20-listener crowdsource, 50 utterances per condition):
#   Before fix: 3.6 (σ=0.5) — "annoying clicks", "sounds glitchy", "periodic distortion"
#   After fix:  4.0 (σ=0.3) — "natural sounding", "no artifacts"
#
# Speaker cosine similarity:
#   Before fix: 0.912 (clicks penalize perceived speaker identity)
#   After fix:  0.914 (essentially unchanged — crossfade does not alter voice characteristics)
#
# Latency impact:
#   Crossfade post-processing: < 1 ms additional computation per utterance
#   No impact on user-facing latency
#
# Crossfade parameters validated:
#   Duration < 2 ms: clicks reduced but not eliminated (window too narrow)
#   Duration 4 ms: clicks eliminated (optimal)
#   Duration > 8 ms: MOS drops 0.1 (crossfade audible as slight smoothing artifact)
#   Hann window: best spectral properties for smooth transition
#   Linear ramp: effective but slight spectral coloring vs Hann

print("Fix: 4 ms Hann-window crossfade at 86-token boundaries before EnCodec decode")
print("Click rate: 7.1/10s → 0.1/10s; MOS: 3.6 → 4.0")
print("Speaker cosine similarity: 0.912 → 0.914 (unchanged)")

Related retainer posts

• VALL-E developer on retainer — neural codec language model TTS; 15–25 ms initial silence gap in voice-cloned outputs; reference codec tokens prepend silence onset; language model matches reference silence; fix: RMS energy threshold trim of reference before codec encoding — 3 weeks
• HierSpeech++ developer on retainer — hierarchical VAE voice conversion; pitch mean shifted 15–20 Hz higher than reference speaker; rising-intonation reference audio biases style encoder pitch representation; fix: 10+ seconds declarative reference audio — 3 weeks
• Amphion developer on retainer — VITS fine-tuned on word-list recording; 40–60 ms inter-word pause at every word boundary; word-list dataset trains VITS alignment model on silence boundaries; fix: 20 minutes continuous paragraph re-recording — 3 weeks