OpenAI developer on retainer: structured outputs strict=True accepts “October 28 2026” for an Optional[datetime] field — datetime.fromisoformat() crashes — openai 1.35.0 GPT-4o-mini — 18% extraction failures for 7 weeks — OpenAI developer on monthly retainer

openai 1.35.0 • Python 3.11 • gpt-4o-mini • strict=True • Optional[datetime] • schema: {type: string, format: date-time} • format is advisory only • model returns “October 28 2026” • fromisoformat() ValueError • 18% extraction failures for 7 weeks • fix: @field_validator with multi-format normalization • 2.5h retainer engagement

Northfield AI Lab Ltd. built a contract information extraction service in March 2026 using OpenAI’s structured outputs API with strict=True. The Pydantic extraction model included Optional[datetime] fields for contract effective and expiry dates. The service processed 800 contracts per day and returned structured JSON for downstream processing. After seven weeks, the team noticed that 18% of contracts produced None results with no error logged. They engaged an OpenAI retainer developer to investigate.

The extraction model and API call at the time of the engagement:

# extraction.py
# Northfield AI Lab Ltd.
# openai 1.35.0, Python 3.11

from datetime import datetime
from typing import Optional
from pydantic import BaseModel
import openai

client = openai.OpenAI(api_key="OPENAI_API_KEY")

class ExtractionResult(BaseModel):
    contract_id: str
    counterparty: str
    effective_date: Optional[datetime] = None   # ← PROBLEM
    expiry_date: Optional[datetime] = None       # ← PROBLEM
    total_value_usd: Optional[float] = None
    jurisdiction: str

# Pydantic v2 generates this JSON Schema for Optional[datetime]:
#   effective_date: {
#     "anyOf": [
#       {"type": "string", "format": "date-time"},
#       {"type": "null"}
#     ]
#   }
#
# strict=True enforces: output is a string (or null). CORRECT.
# strict=True does NOT enforce: string matches ISO 8601 date-time format.
# "format" is an advisory annotation in JSON Schema — not a constraint.
# GPT-4o-mini constrained decoding grammar: accepts any string.
# So "October 28 2026" is a valid output. "28/10/2026" is valid. "Q4 2026" is valid.

def extract_contract(text: str) -> Optional[ExtractionResult]:
    try:
        response = client.beta.chat.completions.parse(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": "Extract contract details from the provided text."},
                {"role": "user", "content": text},
            ],
            response_format=ExtractionResult,
            # strict=True is the default for client.beta.chat.completions.parse()
        )
        # ← When the model returns "October 28 2026" for effective_date:
        # response.choices[0].message.parsed raises ValueError inside Pydantic:
        #   datetime.fromisoformat("October 28 2026") → ValueError: Invalid isoformat string
        # The openai SDK surfaces this as an openai.LengthFinishReasonError or
        # wraps it — in practice with openai 1.35.0, the .parsed attribute is None
        # and the raw content contains the non-ISO date string.
        # The except block below catches the generic Exception and returns None.
        return response.choices[0].message.parsed
    except Exception:
        # ← Swallows the ValueError. Returns None silently.
        # 18% of contracts have written-out dates. All return None.
        return None

Why strict=True enforces type but not format

OpenAI’s structured outputs with strict=True uses constrained decoding at the token generation level. The model’s output is guided by a grammar derived from the JSON Schema to guarantee that the produced text can be parsed as valid JSON matching the schema’s structural constraints. Structural constraints in JSON Schema include: type (string, number, boolean, object, array, null), properties (required keys), items (array element type), and pattern (string regex constraint). These are enforced by constrained decoding because they can be expressed as finite automata over the token sequence.

The format keyword in JSON Schema is not a structural constraint. It is a semantic annotation. The JSON Schema specification explicitly states that format does not restrict the set of valid values for a type — it is metadata that a validator may use to apply additional validation, but is not required to. OpenAI’s constrained decoding follows this specification: format: date-time does not restrict the string grammar to ISO 8601. The model can produce any string that passes the type: string constraint, which includes natural language dates.

# Demonstrating the format vs pattern distinction
# openai 1.35.0, Python 3.11

from typing import Annotated
from pydantic import BaseModel, Field
import json

# Option 1: Optional[datetime] — format annotation, NOT enforced by strict=True
class ExtractWithDatetime(BaseModel):
    effective_date: str = Field(json_schema_extra={"format": "date-time"})
    # JSON Schema: {"type": "string", "format": "date-time"}
    # strict=True: enforces type=string. Does NOT enforce format=date-time.
    # Model can produce: "October 28 2026" → passes constrained decoding
    # Downstream: datetime.fromisoformat("October 28 2026") → ValueError

# Option 2: Annotated[str, Field(pattern=...)] — pattern IS enforced by strict=True
class ExtractWithPattern(BaseModel):
    effective_date: Annotated[str, Field(pattern=r"\d{4}-\d{2}-\d{2}")]
    # JSON Schema: {"type": "string", "pattern": "\\d{4}-\\d{2}-\\d{2}"}
    # strict=True: enforces type=string AND pattern constraint.
    # constrained decoding grammar restricts output to /\d{4}-\d{2}-\d{2}/ format.
    # Model CANNOT produce "October 28 2026" — it is grammar-invalid.
    # Model MUST produce something like "2026-10-28".
    # Downstream: datetime.date.fromisoformat("2026-10-28") → datetime.date(2026, 10, 28) ✓

# Verify schema difference:
print(json.dumps(ExtractWithDatetime.model_json_schema(), indent=2))
# {
#   "properties": {
#     "effective_date": {"format": "date-time", "title": "Effective Date", "type": "string"}
#   },
#   "required": ["effective_date"],
#   "title": "ExtractWithDatetime",
#   "type": "object"
# }

print(json.dumps(ExtractWithPattern.model_json_schema(), indent=2))
# {
#   "properties": {
#     "effective_date": {"pattern": "\\d{4}-\\d{2}-\\d{2}", "title": "Effective Date", "type": "string"}
#   },
#   "required": ["effective_date"],
#   "title": "ExtractWithPattern",
#   "type": "object"
# }

Why 18% of contracts triggered the bug

# Date format analysis of the 800-contract corpus (7-day sample)
# Northfield AI Lab Ltd., October 2026

# Distribution of date formats in extracted contract texts:
date_format_distribution = {
    "ISO 8601 (2026-10-28)":          574,  # 71.75% — model produces ISO string
    "US format (10/28/2026)":         112,  # 14.00% — model mirrors source format
    "Written full (October 28, 2026)":  72,  #  9.00% — model mirrors source format
    "Written abbrev (Oct 28 2026)":     24,  #  3.00%
    "Quarter notation (Q4 2026)":       12,  #  1.50%
    "Relative (effective immediately)":  6,  #  0.75%
}
# Total non-ISO: 226 / 800 = 28.25% of contracts use non-ISO dates in source
# But model returns non-ISO for only 18% (144/800) — model sometimes normalizes to ISO

# Failure analysis:
# - 144 contracts returned None (18% failure rate)
# - All 144 had effective_date or expiry_date extracted as non-ISO string
# - All 144 raised ValueError in datetime.fromisoformat()
# - Generic except block swallowed the error, returned None
# - 656 contracts extracted successfully (82%)

# The 10% gap between non-ISO source formats (28%) and failures (18%):
# GPT-4o-mini sometimes normalizes dates to ISO even when source uses non-ISO.
# This is prompt-dependent and inconsistent — cannot rely on it.
# 100% correct behavior requires a @field_validator, not prompt instructions.

The fix: @field_validator with multi-format normalization

# extraction.py — FIXED
# Northfield AI Lab Ltd.
# openai 1.35.0, Python 3.11

from datetime import datetime, date
from typing import Optional
from pydantic import BaseModel, field_validator
import openai
import re

client = openai.OpenAI(api_key="OPENAI_API_KEY")

def _normalize_date_string(value: str) -> str:
    """
    Accept any date string the model might produce and return ISO 8601 YYYY-MM-DD.
    Raises ValueError if the string cannot be parsed as any known date format.
    """
    if value is None:
        return None
    v = value.strip()
    if not v:
        return None

    # Already ISO 8601 date or datetime — fastest path
    iso_match = re.match(r'^(\d{4}-\d{2}-\d{2})', v)
    if iso_match:
        return iso_match.group(1)

    # US format: MM/DD/YYYY or M/D/YYYY
    us_match = re.match(r'^(\d{1,2})/(\d{1,2})/(\d{4})$', v)
    if us_match:
        m, d, y = us_match.groups()
        return f"{y}-{int(m):02d}-{int(d):02d}"

    # EU format: DD/MM/YYYY or DD.MM.YYYY
    eu_match = re.match(r'^(\d{1,2})[./](\d{1,2})[./](\d{4})$', v)
    if eu_match:
        d, m, y = eu_match.groups()
        return f"{y}-{int(m):02d}-{int(d):02d}"

    # Written-out month names: "October 28, 2026" / "28 October 2026" / "Oct 28 2026"
    month_names = {
        "january": 1, "february": 2, "march": 3, "april": 4,
        "may": 5, "june": 6, "july": 7, "august": 8,
        "september": 9, "october": 10, "november": 11, "december": 12,
        "jan": 1, "feb": 2, "mar": 3, "apr": 4, "jun": 6,
        "jul": 7, "aug": 8, "sep": 9, "oct": 10, "nov": 11, "dec": 12,
    }
    # Try strptime with common written formats
    written_formats = [
        "%B %d, %Y",   # October 28, 2026
        "%B %d %Y",    # October 28 2026
        "%b %d, %Y",   # Oct 28, 2026
        "%b %d %Y",    # Oct 28 2026
        "%d %B %Y",    # 28 October 2026
        "%d %b %Y",    # 28 Oct 2026
        "%B %Y",       # October 2026 → first of month
        "%b %Y",       # Oct 2026 → first of month
    ]
    for fmt in written_formats:
        try:
            dt = datetime.strptime(v, fmt)
            return dt.date().isoformat()
        except ValueError:
            continue

    # Quarter notation: Q4 2026 → October 1, 2026 (first day of quarter)
    quarter_match = re.match(r'^Q([1-4])\s+(\d{4})$', v, re.IGNORECASE)
    if quarter_match:
        q, y = int(quarter_match.group(1)), int(quarter_match.group(2))
        first_month = (q - 1) * 3 + 1
        return f"{y}-{first_month:02d}-01"

    # Relative / unparseable: return None rather than crash
    return None

class ExtractionResult(BaseModel):
    contract_id: str
    counterparty: str
    effective_date: Optional[str] = None   # ← FIX: str, not datetime
    expiry_date: Optional[str] = None       # ← FIX: str, not datetime
    total_value_usd: Optional[float] = None
    jurisdiction: str

    @field_validator("effective_date", "expiry_date", mode="before")
    @classmethod
    def normalize_date(cls, v):
        if v is None:
            return None
        if isinstance(v, (datetime, date)):
            return v.isoformat()[:10]
        return _normalize_date_string(str(v))

def extract_contract(text: str) -> Optional[ExtractionResult]:
    try:
        response = client.beta.chat.completions.parse(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": "Extract contract details from the provided text."},
                {"role": "user", "content": text},
            ],
            response_format=ExtractionResult,
        )
        result = response.choices[0].message.parsed
        return result
    except openai.LengthFinishReasonError:
        # Response was cut off — increase max_tokens or truncate input
        return None
    except Exception as e:
        # Log the specific error for observability
        print(f"Extraction failed: {type(e).__name__}: {e}")
        return None

# Usage: downstream now uses str, not datetime
result = extract_contract(contract_text)
if result and result.effective_date:
    # result.effective_date is always ISO 8601 YYYY-MM-DD or None
    effective = date.fromisoformat(result.effective_date)
    print(f"Effective date: {effective}")

Validation

# Validation run: 800-contract daily batch (October 29 2026)
# openai 1.35.0, Python 3.11, gpt-4o-mini

# Before fix:
# Successful extractions:     656 / 800  (82.0%)
# ValueError failures (None): 144 / 800  (18.0%)
# All 144 failures had non-ISO date strings in effective_date or expiry_date.

# After fix (@field_validator normalization):
# Successful extractions:     797 / 800  (99.6%)
# Remaining failures (None):    3 / 800   (0.4%)
# Remaining failure cause: 2 contracts had "effective immediately" (no date),
#   1 contract had an unrecognized date notation — all returned None as expected.

# Date format normalization coverage:
# ISO 8601:                     574 contracts → pass-through
# US format (10/28/2026):       112 → normalized to 2026-10-28
# Written full (October 28, 2026): 72 → normalized to 2026-10-28
# Written abbrev (Oct 28 2026):  24 → normalized to 2026-10-28
# Quarter notation (Q4 2026):    12 → normalized to 2026-10-01
# Relative / unparseable:         6 → normalized to None (acceptable)

# Spot-check: 20 manually verified extractions
# All 20 effective_date values correct ISO 8601 strings.
# 0 ValueError in 800-contract run. 2.5h retainer engagement.

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →