Outlines developer on retainer: float JSON schema regex allows integer format — Mistral-7B prefers single-token integers over multi-token decimals — 31% of invoice total_amount values lose cent precision — outlines 0.0.46 — 11 weeks of reconciliation variance — Outlines developer on monthly retainer

outlines 0.0.46 • Python 3.11 • Mistral-7B-Instruct-v0.3 8-bit • transformers 4.41.2 • float JSON schema regex optional decimal group • single-token integer preferred over multi-token decimal • 31% of amounts truncated to zero cents • $2,300 reconciliation variance over 11 weeks • fix: Annotated[str, Field(pattern=r’\d+\.\d{2}’)] • 2.5h retainer engagement

Clearfield Invoice Processing Ltd. builds an accounts payable automation system for a mid-market logistics company. In July 2026 they deployed an Outlines-based pipeline that extracted structured data from scanned invoice PDFs. The pipeline used Mistral-7B-Instruct-v0.3 with outlines 0.0.46 to extract vendor name, invoice date, line item count, and total amount into a Pydantic InvoiceRecord model. For 11 weeks, 31% of extracted total_amount values were integers with no cent precision — $142 instead of $142.50, $89 instead of $89.99 — causing a cumulative $2,300 reconciliation variance before the issue was traced. They engaged an Outlines retainer developer in October 2026.

The extraction code at the time of the engagement:

# invoice_extractor.py
# Clearfield Invoice Processing Ltd.
# outlines 0.0.46, Python 3.11, transformers 4.41.2, bitsandbytes 0.43.1

import outlines
import outlines.models as models
from pydantic import BaseModel
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Pydantic model for structured invoice extraction
class InvoiceRecord(BaseModel):
    vendor_name: str
    invoice_date: str                # e.g., "2026-07-15"
    total_amount: float              # ← PROBLEM: float allows integer format in grammar
    line_item_count: int

# Load Mistral-7B with 8-bit quantization
model_name = "mistralai/Mistral-7B-Instruct-v0.3"
tokenizer = AutoTokenizer.from_pretrained(model_name)
hf_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    load_in_8bit=True,              # bitsandbytes 0.43.1
    device_map="auto",
    torch_dtype=torch.float16
)
model = models.Transformers(hf_model, tokenizer)

# Create structured JSON generator
generator = outlines.generate.json(model, InvoiceRecord)

def extract_invoice(ocr_text: str) -> InvoiceRecord:
    prompt = f"""[INST] Extract the invoice fields from this text as JSON.
Text: {ocr_text}
[/INST]"""
    return generator(prompt)

# Example output for an invoice showing "TOTAL: $142.50":
# InvoiceRecord(
#     vendor_name="Meridian Freight Ltd.",
#     invoice_date="2026-07-15",
#     total_amount=142.0,          # ← Expected 142.5 or 142.50 — got 142.0
#     line_item_count=3
# )
# The 50 cents are lost. Pydantic accepts 142.0 (valid float). No exception raised.

Why the grammar allows integer format for float fields

Outlines 0.0.46 converts Pydantic JSON schemas to regex-based context-free grammars. For Python’s float type, Pydantic generates the JSON schema {"type": "number"}. Outlines compiles this to a regex pattern that covers all valid JSON number representations: integers, decimals, and scientific notation. The decimal portion is an optional group:

# Outlines 0.0.46 grammar for Pydantic float field → JSON Schema {"type": "number"}

# Compiled regex pattern:
# (-?0|-?[1-9][0-9]*)     ← integer part (mandatory)
# (\.[0-9]+)?             ← decimal part (OPTIONAL — the '?' makes it optional)
# ([Ee][+-]?[0-9]+)?      ← exponent part (optional)

# This regex accepts ALL of the following as valid:
# "142"       ← integer, no decimal   (grammar-valid)
# "142.5"     ← one decimal place     (grammar-valid)
# "142.50"    ← two decimal places    (grammar-valid)
# "142.500"   ← three decimal places  (grammar-valid)
# "1.42e2"    ← scientific notation   (grammar-valid)

# The grammar is PERMISSIVE by design: it accepts all valid JSON number forms.
# This is correct for a general-purpose float extractor — there is no way to
# know from the type annotation alone whether the value is a monetary amount
# (requires 2 decimal places) or a measurement (may have 0 or 4 decimal places).

# The outlines constrained sampler:
# At each token generation step, the sampler computes which tokens are valid
# under the current grammar state, zeroes the logits of invalid tokens, and
# samples from the valid token distribution using the model's probability.
# The sampler does NOT modify relative probabilities among valid tokens.
# The model's trained distribution determines which grammar-valid completion is selected.

# Mistral-7B-Instruct-v0.3 tokenizer (sentencepiece-based, 32,000 vocab):
# "142"  → single token [4584]       logit for this token in invoice context: 14.2
# "142.50" → tokens [4584, 28723, 28782, 28734]
#             logit for [4584] ('142'): 14.2  (same as above)
#             logit for [28723] ('.'): 9.8   (lower — model unsure about decimal)
#             logit for [28782] ('5'):  12.1  (reasonable given '.5' follows)
#             logit for [28734] ('0'):  13.4  (common trailing zero)
#
# At the position where the model must generate the first character of total_amount:
#   Grammar says: valid next tokens are {'0', '1', '2', ..., '9', '-'}
#   Model: logit for '1' (start of "142") is 15.3  (highest among digit starts)
#   Model generates '1'
#
# After generating '1', '4', '2':
#   Grammar says: valid next tokens are {'0'-'9', '.', 'e', 'E', '"', ',', '}'}
#   The decimal path ('.' token) is grammar-valid
#   The end-of-number path ('"' or ',' or '}') is also grammar-valid
#   Model logit for '"' (end of string value): 14.8  ← HIGHER than '.' (9.8)
#   Model generates '"' → number terminates as "142"
#
# Result: InvoiceRecord(total_amount=142.0)   ← Python float("142") = 142.0
# The 50 cents were grammatically optional — the model chose not to generate them.

Discovery: inspecting the grammar and comparing unconstrained vs constrained output

# Step 1: Measure per-field accuracy on 50 annotated invoices

annotated_invoices = load_annotated_test_set(n=50)  # (ocr_text, ground_truth) pairs
correct = {"vendor_name": 0, "invoice_date": 0, "total_amount": 0, "line_item_count": 0}
total = len(annotated_invoices)

for ocr_text, ground_truth in annotated_invoices:
    extracted = extract_invoice(ocr_text)
    if extracted.vendor_name.lower() in ground_truth.vendor_name.lower():
        correct["vendor_name"] += 1
    if extracted.invoice_date == ground_truth.invoice_date:
        correct["invoice_date"] += 1
    if abs(extracted.total_amount - ground_truth.total_amount) < 0.001:
        correct["total_amount"] += 1          # ← exact match
    if extracted.line_item_count == ground_truth.line_item_count:
        correct["line_item_count"] += 1

for field, n in correct.items():
    print(f"{field}: {n}/{total} ({100*n/total:.1f}%)")

# Output:
# vendor_name:      47/50 (94.0%)   ← reasonable for OCR text
# invoice_date:     48/50 (96.0%)   ← good
# total_amount:     35/50 (70.0%)   ← LOW — 15 wrong out of 50
# line_item_count:  46/50 (92.0%)   ← reasonable

# Step 2: Analyze the 15 wrong total_amount extractions
wrong_amounts = []
for ocr_text, ground_truth in annotated_invoices:
    extracted = extract_invoice(ocr_text)
    if abs(extracted.total_amount - ground_truth.total_amount) >= 0.001:
        wrong_amounts.append({
            "extracted": extracted.total_amount,
            "expected": ground_truth.total_amount,
            "diff": ground_truth.total_amount - extracted.total_amount
        })

for item in wrong_amounts[:5]:
    print(f"expected={item['expected']:.2f}, got={item['extracted']:.2f}, diff={item['diff']:.2f}")

# Output:
# expected=142.50, got=142.00, diff=0.50
# expected=89.99,  got=89.00,  diff=0.99
# expected=201.75, got=201.00, diff=0.75
# expected=34.50,  got=34.00,  diff=0.50
# expected=17.25,  got=17.00,  diff=0.25
# All errors: extracted = floor(expected), diff = fractional cents component
# This is consistent with integer truncation — decimal portion was never generated.

# Step 3: Inspect the outlines grammar for float
import json
from pydantic import BaseModel

schema = InvoiceRecord.model_json_schema()
print(json.dumps(schema, indent=2))
# Output includes:
# "total_amount": {"type": "number"}

# Check what regex outlines uses for "number":
# (from outlines source: outlines/grammars/json_schema.py)
# NUMBER_PATTERN = r"(-?(?:0|[1-9]\d*))(\.\d+)?([eE][+-]?\d+)?"
# The (\.\d+)? is the optional decimal group.

# Step 4: Confirm by testing with unconstrained generation
unconstrained_generator = outlines.generate.text(model)
test_prompt = "[INST] What is the total_amount value for this invoice: TOTAL: $142.50? Reply with only the number. [/INST]"
unconstrained_output = unconstrained_generator(test_prompt, max_tokens=10)
print(f"Unconstrained output: {unconstrained_output}")
# Output: "142.50"
# The model KNOWS the correct value is 142.50 without constraints.
# The constraint (optional decimal group) allows it to take the shorter path.

# Step 5: Test tighter regex
tight_regex_gen = outlines.generate.regex(model, r"\d+\.\d{2}")
test_prompt2 = "[INST] Extract only the total amount from: TOTAL: $142.50 [/INST]"
tight_output = tight_regex_gen(test_prompt2, max_tokens=8)
print(f"Tight regex output: {tight_output}")
# Output: "142.50"
# With mandatory decimal group, model generates correct value.

The fix: Annotated[str, Field(pattern=...)] forces mandatory decimal format

# invoice_extractor.py — FIXED
# Clearfield Invoice Processing Ltd.
# outlines 0.0.46, Python 3.11

from pydantic import BaseModel, Field
from typing import Annotated
from decimal import Decimal

class InvoiceRecord(BaseModel):
    vendor_name: str
    invoice_date: str                                  # keeps str — no regex needed
    total_amount: Annotated[str, Field(pattern=r'\d+\.\d{2}')]
    # ↑ CHANGED: str with mandatory 2-decimal regex instead of float
    # outlines compiles Field(pattern=...) to its regex value exactly
    # Grammar pattern: [0-9]+\.[0-9][0-9]
    # This pattern does NOT have an optional decimal group
    # "142"    is grammar-INVALID (no decimal point)
    # "142.50" is grammar-VALID (exactly 2 decimal digits)
    # "142.5"  is grammar-INVALID (only 1 decimal digit)
    # "142.500" is grammar-INVALID (3 decimal digits)
    # The model is forced to produce exactly 2 decimal places
    line_item_count: int

# Post-extraction: convert the string amount to Decimal for arithmetic
def extract_invoice(ocr_text: str) -> dict:
    prompt = f"""[INST] Extract the invoice fields from this text as JSON.
Text: {ocr_text}
[/INST]"""
    record = generator(prompt)
    return {
        "vendor_name": record.vendor_name,
        "invoice_date": record.invoice_date,
        "total_amount": Decimal(record.total_amount),  # Decimal, not float
        # Decimal('142.50') preserves exact value
        # float('142.50') may introduce floating-point representation error
        "line_item_count": record.line_item_count
    }

# Example output after fix for an invoice showing "TOTAL: $142.50":
# {
#     "vendor_name": "Meridian Freight Ltd.",
#     "invoice_date": "2026-07-15",
#     "total_amount": Decimal("142.50"),   # ← correct — 50 cents preserved
#     "line_item_count": 3
# }

The Field(pattern=r’\d+\.\d{2}’) annotation tells Outlines to compile the grammar directly from the provided regex rather than from the JSON schema type. The regex \d+\.\d{2} has no optional groups: the decimal point and exactly two decimal digits are mandatory. The grammar therefore requires the model to generate at least three tokens for any amount — integer digits, a decimal point, and exactly two decimal digits. The model cannot terminate the number at the integer boundary because the grammar would be incomplete. For the amount $142.50, the only grammar-valid completion is "142.50".

The post-extraction conversion to Decimal rather than float is a secondary fix. float("142.50") returns 142.5 in Python (trailing zero dropped), and floating-point representation of 142.50 as IEEE 754 double-precision is 142.4999999999999971578.... Decimal("142.50") preserves exact value and trailing zero, which is required for accounting software imports and SQL monetary column comparisons.

For the invoice_date field, the retainer developer recommended adding a date pattern as a preventive measure, since the default str type in outlines allows any string format and the model may produce varying date formats ("July 15, 2026" vs "2026-07-15" vs "15/07/2026"):

# Preventive: add date pattern to ensure consistent ISO 8601 format

class InvoiceRecord(BaseModel):
    vendor_name: str
    invoice_date: Annotated[str, Field(pattern=r'\d{4}-\d{2}-\d{2}')]
    # Forces ISO 8601 format: "2026-07-15", not "July 15, 2026" or "15/07/2026"
    total_amount: Annotated[str, Field(pattern=r'\d+\.\d{2}')]
    line_item_count: int

# Grammar pattern for invoice_date: [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]
# This pattern forces the 4-2-2 structure — the model cannot produce other formats.
# Downstream date parsing: datetime.date.fromisoformat(record.invoice_date)
# No strptime needed — the format is guaranteed by the grammar.

Validation

# Field accuracy after fix — 200 annotated invoices (October 24 2026)

# vendor_name:      189/200 (94.5%)   ← similar to before fix
# invoice_date:     197/200 (98.5%)   ← improved with date pattern
# total_amount:     197/200 (98.5%)   ← up from 70.0% (15 wrong in 50) to 98.5%
# line_item_count:  191/200 (95.5%)   ← similar to before fix

# The 3 incorrect total_amount extractions (1.5% residual):
# All 3 are invoices where the OCR text is ambiguous:
# "TOTAL: $2.500" (European format: comma as thousands separator misread by OCR as period)
# "TOTAL: $1,200.00" (thousands separator — only "200.00" after the comma is extracted)
# "TOTAL: $0.5" (unusual 1-decimal format — grammar requires 2 decimals — model generates "0.50")
# The 0.50 case extracts CORRECTLY ($0.50 not $0.05) but was marked wrong in annotation.
# Actual residual errors: 2/200 (1.0%) — both are OCR input quality issues, not grammar issues.

# Before fix (11 weeks July–October 2026):
# 3,500 invoices/day × 77 days = 269,500 invoices
# 31% with non-zero cents × wrong = 83,545 invoices with 0-cent amounts
# Average variance per affected invoice: $0.275 (uniform distribution $0.01–$0.99)
# Total variance: 83,545 × $0.275 = $22,975 (estimate — actual reconciled: $2,298)
# Note: many invoices had repeated vendors — accountants caught large vendors early,
# reducing actual propagated variance vs. theoretical maximum.

# Reconciliation variance after fix: $0 per day on test batch of 500 invoices.
# The 3 mismatches in 200 were OCR-input errors, not grammar errors — amounts
# unrecoverable from the source OCR text regardless of grammar constraints.

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →