Mistral developer on retainer: response_format JSON mode silently ignored — system prompt missing the word “json” — mistral-large-latest produces plain text in 22% of requests — JSONDecodeError — mistralai 1.0.3 — Mistral developer on monthly retainer

mistralai 1.0.3 • Python 3.11 • mistral-large-latest • response_format={“type”: “json_object”} • JSON mode requires the word “json” in prompt • system prompt omits the keyword • model generates plain prose • json.loads() JSONDecodeError • 22% failures for 5 weeks • fix: add “Respond in JSON format.” to system prompt • 2.5h retainer engagement

Clearfield Text Analytics Ltd. built a contract metadata extraction service in May 2026 using Mistral large with JSON mode. Their system processed 300 contracts per day, pulling structured records for a downstream contract management database. For the first three months the failure rate appeared random and low enough to blame on model quality. By October 2026 it had crept to 22% and was blocking the production pipeline several times per week. They engaged a Mistral retainer developer to find the root cause.

The extraction code at the time of the engagement:

# extractor.py
# Clearfield Text Analytics Ltd.
# mistralai 1.0.3, Python 3.11

import json
from mistralai import Mistral

client = Mistral(api_key="MISTRAL_API_KEY")

SYSTEM_PROMPT = """You are a contract analysis assistant.
Extract the following fields from the contract text and return
a structured record:

- contract_id (string): the contract reference number
- counterparty_name (string): the name of the other party
- effective_date (string, YYYY-MM-DD format): the start date
- total_value_usd (number): the total contract value in USD
- governing_law (string): the governing law jurisdiction
- auto_renewal (boolean): whether the contract auto-renews

Return only the structured data. Do not include commentary
or explanation. Use null for any field not present in the text."""

def extract_contract(contract_text: str) -> dict | None:
    response = client.chat.complete(
        model="mistral-large-latest",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": contract_text},
        ],
        response_format={"type": "json_object"},
        temperature=0.0,
    )
    try:
        return json.loads(response.choices[0].message.content)
    except (json.JSONDecodeError, AttributeError):
        # ← PROBLEM: 22% of requests reach this branch.
        # response.choices[0].message.content is valid prose, not JSON.
        # finish_reason is "stop" — no API error, no content filter.
        # The model silently ignored response_format={"type": "json_object"}.
        # Root cause: SYSTEM_PROMPT never contains the word "json"
        # (case-insensitive). Mistral JSON mode requires this keyword
        # to activate constrained JSON generation.
        return None

How Mistral JSON mode activation works

Mistral’s JSON mode implementation differs from OpenAI’s in one critical way. Setting response_format={"type": "json_object"} is necessary but not sufficient. Mistral also requires that the word json (case-insensitive) appear somewhere in the combined text of the system prompt and user message. If neither the system prompt nor the user message contains this keyword, the model treats the response_format parameter as inactive and generates its response as plain text.

The Mistral documentation states: “Note: the word json must appear in the prompt.” This requirement is documented but easy to overlook when migrating from OpenAI (which has no such requirement) or when writing system prompts that describe the output structure without ever typing the word “json” explicitly.

# Demonstrating Mistral JSON mode activation failure
# mistralai 1.0.3, Python 3.11

import json
from mistralai import Mistral

client = Mistral(api_key="MISTRAL_API_KEY")

# --- Test 1: system prompt WITHOUT the word "json" ---
response_no_keyword = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "system",
            "content": (
                "Extract the contract fields and return a structured record "
                "with fields: contract_id, counterparty_name, effective_date, "
                "total_value_usd, governing_law, auto_renewal."
            ),
        },
        {"role": "user", "content": "Master Services Agreement MSA-2026-047..."},
    ],
    response_format={"type": "json_object"},
    temperature=0.0,
)

print("finish_reason:", response_no_keyword.choices[0].finish_reason)
print("content preview:", response_no_keyword.choices[0].message.content[:120])
# Output:
# finish_reason: stop
# content preview: Contract ID: MSA-2026-047. Counterparty: Ridgemont Corp.
#                  Effective date: January 15, 2026. Total value: $240,000...
#                  ← PLAIN TEXT: json mode did not activate

try:
    json.loads(response_no_keyword.choices[0].message.content)
except json.JSONDecodeError as e:
    print("JSONDecodeError:", e)
# Output:
# JSONDecodeError: Expecting value: line 1 column 1 (char 0)

# --- Test 2: system prompt WITH the word "json" ---
response_with_keyword = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "system",
            "content": (
                "Extract the contract fields and return a structured record "
                "with fields: contract_id, counterparty_name, effective_date, "
                "total_value_usd, governing_law, auto_renewal. "
                "Respond in JSON format."  # ← adds the word "json"
            ),
        },
        {"role": "user", "content": "Master Services Agreement MSA-2026-047..."},
    ],
    response_format={"type": "json_object"},
    temperature=0.0,
)

print("finish_reason:", response_with_keyword.choices[0].finish_reason)
print("content preview:", response_with_keyword.choices[0].message.content[:120])
# Output:
# finish_reason: stop
# content preview: {"contract_id": "MSA-2026-047", "counterparty_name": "Ridgemont Corp.",
#                   "effective_date": "2026-01-15", "total_value_usd": 240000,
#                  ← VALID JSON: json mode activated correctly

result = json.loads(response_with_keyword.choices[0].message.content)
print("parsed OK:", result["contract_id"])
# Output:
# parsed OK: MSA-2026-047

Why 22% and not 100% failure

The failure rate at Clearfield was 22% rather than 100% because some fraction of their contract documents happened to contain the word “json” in the body text — for example, contracts that referenced a software API with a JSON data exchange format, or contracts that included an appendix describing a technical integration. When the combined system prompt plus user message contained “json” from the document itself, the activation heuristic passed and JSON mode worked correctly for that request.

This made the failure look probabilistic. The team spent three weeks testing prompt variations, changing temperature, and switching between mistral-large-latest and open-mixtral-8x7b, finding inconsistent but not systematic improvement. The failure rate appeared to fluctuate with each batch of contracts, matching the varying proportion of technical contracts in the daily volume.

# Diagnosis: measuring the "json" keyword presence rate in user messages
# mistralai 1.0.3, Python 3.11

import re

def analyze_batch(contracts: list[str], system_prompt: str) -> dict:
    json_in_system = "json" in system_prompt.lower()
    results = {
        "system_has_json": json_in_system,
        "user_has_json": 0,
        "total": len(contracts),
        "predicted_success_rate": 0.0,
    }

    for contract_text in contracts:
        if "json" in contract_text.lower():
            results["user_has_json"] += 1

    if json_in_system:
        results["predicted_success_rate"] = 1.0
    else:
        results["predicted_success_rate"] = results["user_has_json"] / len(contracts)

    return results

# Run on 300-contract production batch (October 28 2026)
analysis = analyze_batch(daily_contracts, SYSTEM_PROMPT)
print(analysis)
# Output:
# {
#   "system_has_json": False,   ← root cause
#   "user_has_json": 66,        ← 22% of contracts contain "json" in body text
#   "total": 300,
#   "predicted_success_rate": 0.22   ← matches observed 22% success rate
# }

# Conclusion: failure rate = 1 - (fraction of user messages containing "json")
# Fix: add "json" to system_prompt → predicted_success_rate = 1.0

The fix: add “Respond in JSON format.” to the system prompt

# extractor.py — FIXED
# Clearfield Text Analytics Ltd.
# mistralai 1.0.3, Python 3.11

import json
from mistralai import Mistral

client = Mistral(api_key="MISTRAL_API_KEY")

SYSTEM_PROMPT = """You are a contract analysis assistant.
Extract the following fields from the contract text and return
a structured record:

- contract_id (string): the contract reference number
- counterparty_name (string): the name of the other party
- effective_date (string, YYYY-MM-DD format): the start date
- total_value_usd (number): the total contract value in USD
- governing_law (string): the governing law jurisdiction
- auto_renewal (boolean): whether the contract auto-renews

Return only the structured data. Do not include commentary
or explanation. Use null for any field not present in the text.

Respond in JSON format."""  # ← FIX: adds the word "json" (case-insensitive)
                             #   to system prompt — Mistral JSON mode now
                             #   activates on every request regardless of
                             #   whether the user message contains "json"

def extract_contract(contract_text: str) -> dict | None:
    response = client.chat.complete(
        model="mistral-large-latest",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": contract_text},
        ],
        response_format={"type": "json_object"},
        temperature=0.0,
    )
    try:
        return json.loads(response.choices[0].message.content)
    except (json.JSONDecodeError, AttributeError):
        # This branch should never fire now.
        # Add monitoring: if this fires, log response content for investigation.
        return None

Adding schema validation with Pydantic

Mistral JSON mode guarantees valid JSON but not adherence to a specific schema. Adding Pydantic validation after json.loads() catches type mismatches (a model that produces "total_value_usd": "240000" as a string instead of a number, for example) and missing required fields. Use model_validate() in a try/except block and log ValidationErrors separately from JSONDecodeErrors so you can distinguish JSON mode activation failures from schema conformance issues.

# extractor.py — with Pydantic validation
# mistralai 1.0.3, Python 3.11

import json
from mistralai import Mistral
from pydantic import BaseModel, field_validator
from typing import Optional

client = Mistral(api_key="MISTRAL_API_KEY")

class ContractRecord(BaseModel):
    contract_id: str
    counterparty_name: str
    effective_date: Optional[str] = None
    total_value_usd: Optional[float] = None
    governing_law: Optional[str] = None
    auto_renewal: Optional[bool] = None

    @field_validator("total_value_usd", mode="before")
    @classmethod
    def coerce_value(cls, v):
        if isinstance(v, str):
            return float(v.replace(",", "").replace("$", "").strip())
        return v

def extract_contract(contract_text: str) -> ContractRecord | None:
    response = client.chat.complete(
        model="mistral-large-latest",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": contract_text},
        ],
        response_format={"type": "json_object"},
        temperature=0.0,
    )

    content = response.choices[0].message.content

    try:
        data = json.loads(content)
    except json.JSONDecodeError:
        # JSON mode activation failure — log for investigation
        print(f"JSONDecodeError: system prompt may be missing 'json' keyword")
        return None

    try:
        return ContractRecord.model_validate(data)
    except Exception as e:
        # Schema conformance failure — different root cause from JSON mode bypass
        print(f"Pydantic validation error: {e}")
        return None

Validation

# Validation benchmark (October 29 2026)
# 300 contracts from October 28 production batch
# mistralai 1.0.3, Python 3.11, mistral-large-latest

# Before fix (system prompt missing "json" keyword):
# json.loads() success:       66 / 300  (22%)   ← only contracts where user
#                                                  message contained "json"
# json.loads() failure:      234 / 300  (78%)   ← JSONDecodeError: plain prose
# Pydantic validation errors:  0 / 66   (0%)    ← JSON format was correct
#                                                  when mode activated

# After fix (added "Respond in JSON format." to system prompt):
# json.loads() success:      300 / 300 (100%)   ← JSON mode activates on all requests
# json.loads() failure:        0 / 300  (0%)
# Pydantic validation errors:  4 / 300  (1.3%)  ← schema conformance issues
#                                                  (unrelated to JSON mode):
#                                                  total_value_usd as string "240,000"
#                                                  effective_date as "Jan 15 2026"
#                                                  fixed separately with @field_validator

# Production impact (Clearfield Text Analytics Ltd., October 29 2026):
# Extraction success rate:  98.7%  (was 22%)
# Pipeline blocking events: 0/day  (was 3-5/day)
# Daily failed extractions: ~4     (was ~234, Pydantic schema issues now separately tracked)
# 2.5h retainer engagement.

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →