TRL developer on retainer: SFTTrainer dataset_text_field='messages' receives list-of-dicts — serialized as Python repr strings — model trains on [{'role': 'user', ...}] not chat template — ignores user messages at inference — 3 weeks — TRL developer on monthly retainer

The fix: apply apply_chat_template() to the dataset before SFTTrainer

Pre-format the dataset by applying apply_chat_template() to each conversation and storing the result in a new column. Then pass that column name to dataset_text_field. Alternatively, use SFTTrainer’s formatting_func parameter to apply the template at training time. Add a training sample format assertion before training starts to catch this class of error early.

# train.py — FIXED (Eastfield RLHF Ltd.)
# trl 0.8.x, Python 3.11, PyTorch 2.3.x

from trl import SFTTrainer, SFTConfig
from datasets import Dataset

# FIX Option A: Pre-format dataset with apply_chat_template before SFTTrainer
def format_conversation(example):
    """Apply chat template to messages column, storing result in 'text' column."""
    formatted = tokenizer.apply_chat_template(
        example["messages"],
        tokenize=False,
        add_generation_prompt=False,  # False for SFT — include full conversation
    )
    return {"text": formatted}

formatted_dataset = dataset.map(format_conversation, batched=False)
# formatted_dataset[0]['text']:
# "<|im_start|>user\nWhat is the refund policy?<|im_end|>\n
#  <|im_start|>assistant\nOur refund policy allows...<|im_end|>"

# Verify training format before starting
def assert_training_format(dataset, tokenizer):
    """Assert first training sample contains chat template tokens, not Python repr."""
    first_text = dataset[0]["text"]
    chat_tokens = ["<|im_start|>", "<|im_end|>", "[INST]", "<>"]
    assert any(tok in first_text for tok in chat_tokens), (
        f"Training sample missing chat template tokens. "
        f"Got: {first_text[:200]!r}. "
        "Check that apply_chat_template() was applied to the dataset before SFTTrainer."
    )
    repr_markers = ["{'role':", "\"role\":", "[{'role'"]
    if any(marker in first_text for marker in repr_markers):
        raise ValueError(
            f"Training sample contains Python repr of dict: {first_text[:100]!r}. "
            "Pass dataset_text_field to a column of formatted strings, not list-of-dicts."
        )
    print(f"[assert_training_format] OK — chat template tokens found in training sample")

assert_training_format(formatted_dataset, tokenizer)

trainer = SFTTrainer(
    model=model,
    args=SFTConfig(
        max_seq_length=2048,
        dataset_text_field="text",  # Pre-formatted text column
        per_device_train_batch_size=4,
        num_train_epochs=3,
    ),
    train_dataset=formatted_dataset,
    tokenizer=tokenizer,
)
trainer.train()

# --- FIX Option B: Use formatting_func for on-the-fly template application ---
# (alternative to pre-formatting — applies template during DataLoader iteration)

def formatting_func(examples):
    """Apply chat template to each conversation in batch."""
    return [
        tokenizer.apply_chat_template(
            conv, tokenize=False, add_generation_prompt=False
        )
        for conv in examples["messages"]
    ]

trainer_option_b = SFTTrainer(
    model=model,
    args=SFTConfig(max_seq_length=2048, per_device_train_batch_size=4, num_train_epochs=3),
    train_dataset=dataset,   # Original dataset with 'messages' column
    tokenizer=tokenizer,
    formatting_func=formatting_func,  # Applied per batch — no dataset_text_field needed
)

Validation

# Validation log — November 2026
# trl 0.8.x, transformers 4.39.x, Llama-3-8B

# ---- Before fix ----
# dataset_text_field='messages' — column contains list-of-dicts
# First training sample decoded: "[{'role': 'user', 'content': '...'}]" (Python repr)
# Chat template tokens present: False
# Training loss: 2.3 → 0.92 (model learned repr format)
# Inference: model ignores user message; same generic response to all inputs

# ---- After fix: pre-format with apply_chat_template ----
# dataset_text_field='text' — column contains apply_chat_template() output
# First training sample decoded: "<|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n..."
# Chat template tokens present: True                                             ✓
# assert_training_format() PASSED                                               ✓
# Training loss: 2.1 → 0.88 (similar convergence, but model learned chat format)

# ---- Inference quality comparison ----
# Prompt (apply_chat_template): "<|im_start|>user\nWhat is the refund policy?<|im_end|>\n<|im_start|>assistant\n"
# Before fix: [Generic irrelevant response — model ignores user content]
# After fix:  "Our refund policy allows returns within 30 days of purchase..."   ✓

# ---- Format consistency check ----
# Training format:  "<|im_start|>user\n{content}<|im_end|>\n<|im_start|>assistant\n{response}"
# Inference format: "<|im_start|>user\n{content}<|im_end|>\n<|im_start|>assistant\n"
# Formats match: ✓ — model responds correctly to the format it was trained on

# ---- Hour breakdown ----
# Triage:     0.5h — model ignores user input; training loss correct;
#             decode first training sample: Python repr format confirmed
# Root cause: 0.25h — dataset_text_field='messages' receives list-of-dicts;
#             tokenizer str() serializes to repr; chat template never applied to training data
# Fix:        0.5h — apply apply_chat_template to dataset; add assert_training_format()
# Validation: 0.25h — training sample decoded contains chat template tokens;
#             inference format matches training format; model responds correctly
# Total:      1.5h

# Retainer engagement cost: 1.5h (estimated $150–232 at mid-level rates)

Related retainer posts

• PEFT developer on retainer — LoRA adapter saved after merge_and_unload() — adapter directory contains full model weights — PeftModel.from_pretrained() loads empty adapter — inference identical to untuned base model — 3 weeks
• bitsandbytes developer on retainer — 4-bit NF4 model loaded with device_map='cpu' — silently placed on GPU — model.to('cuda') re-quantizes already-quantized weights — doubly-quantized garbage — 3 weeks
• PyTorch Lightning developer on retainer — ReduceLROnPlateau monitors 'val_loss' but developer logs 'val/loss' — scheduler never fires — learning rate never decreases — 3 weeks