Ollama developer on retainer: requests.post() to Ollama REST API raises JSONDecodeError — stream=true default returns NDJSON, not a single JSON document — response.json() fails on line 2 — 100% of production calls fail for 3 weeks — Ollama developer on monthly retainer

requests 2.31.0 • Python 3.11 • Ollama 0.3.9 • llama3.1:8b • POST /api/generate • stream=true default • NDJSON response • response.json() → JSONDecodeError: Extra data line 2 • 100% failure • 3,200 daily jobs fail 3 weeks • fix: "stream": False in payload • 2.0h retainer engagement

Northfield Content Processing Ltd. built an automated content generation pipeline in September 2026 using Ollama’s REST API to run llama3.1:8b locally on an RTX 4090 24GB VRAM. The self-hosted approach eliminated cloud API costs and kept data on-premises. The pipeline performed flawlessly in a curl-based proof of concept. As soon as the developer replaced the curl test with a Python requests integration, every single one of the 3,200 daily jobs failed with the same exception: json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 87). They engaged an Ollama retainer developer to investigate.

The content generator at the time of the engagement:

# content_generator.py
# Northfield Content Processing Ltd.
# requests 2.31.0, Python 3.11, Ollama 0.3.9

import requests

OLLAMA_BASE_URL = "http://localhost:11434"

def generate_content(prompt: str) -> str:
    """Generate product description using local Ollama model."""
    response = requests.post(
        f"{OLLAMA_BASE_URL}/api/generate",
        json={
            "model": "llama3.1:8b",
            "prompt": prompt,
            # ← BUG: no "stream" key in payload.
            # Ollama's default is stream=true.
            # The response body will be NDJSON (newline-delimited JSON):
            # multiple JSON objects, one per generated token, each on its own line.
            # This is NOT a single JSON document.
        },
        timeout=120,
    )
    response.raise_for_status()
    data = response.json()
    # ↑ json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 87)
    # response.json() calls json.loads(response.text) internally.
    # json.loads() requires a single complete JSON document.
    # The response body contains dozens to hundreds of JSON objects, one per line.
    # json.loads() parses the first object successfully, then encounters the second
    # object on line 2 and raises JSONDecodeError: Extra data.
    return data["response"]

# All 3,200 daily calls raise JSONDecodeError.
# The Ollama server logs show successful generation — it is working correctly.
# The failure is entirely on the client side.

# Example of what Ollama actually sends in the HTTP response body (stream=true):
# {"model":"llama3.1:8b","created_at":"2026-10-30T09:14:22Z","response":"The","done":false}
# {"model":"llama3.1:8b","created_at":"2026-10-30T09:14:22Z","response":" product","done":false}
# {"model":"llama3.1:8b","created_at":"2026-10-30T09:14:22Z","response":" features","done":false}
# ... (one JSON object per generated token; typically 50-500 lines per request)
# {"model":"llama3.1:8b","created_at":"2026-10-30T09:14:25Z","response":"","done":true,
#  "context":[128006,882,...],"total_duration":3421810583,"load_duration":5034542,
#  "prompt_eval_count":18,"prompt_eval_duration":85000000,"eval_count":47,"eval_duration":3200000000}
# ← json.loads() parses the first line ({...,"response":"The","done":false}) correctly.
# Then it sees "{\n" at line 2 and raises JSONDecodeError: Extra data.

Why Ollama’s streaming default surprises Python developers

Ollama’s REST API was designed for interactive use: a terminal or chat UI that wants to display tokens as they arrive. For that use case, streaming is the right default — the user sees the first word of the response within milliseconds instead of waiting for the full generation to complete. The stream parameter defaults to true in both /api/generate and /api/chat.

NDJSON (newline-delimited JSON) is a valid and common format for streaming data. Each line is an independent JSON object. But it is not a single JSON document. The Python json.loads() function — which requests.Response.json() calls internally — requires a single JSON document as input. It does not know about NDJSON. When it encounters a second JSON object starting on line 2, it raises JSONDecodeError: Extra data.

# Diagnosing the NDJSON streaming issue
# requests 2.31.0, Python 3.11

import requests
import json

# Step 1: print the raw response body before calling .json()
# This reveals what Ollama is actually sending.
response = requests.post(
    "http://localhost:11434/api/generate",
    json={"model": "llama3.1:8b", "prompt": "Say hello in one word."},
    timeout=30,
)
response.raise_for_status()

print("Status code:", response.status_code)   # 200 ← server is healthy
print("Content-Type:", response.headers.get("Content-Type"))
# application/x-ndjson ← the Content-Type reveals NDJSON format

print("\nRaw response body:")
print(response.text[:600])
# {"model":"llama3.1:8b","created_at":"2026-10-30T09:15:01Z","response":"Hello","done":false}
# {"model":"llama3.1:8b","created_at":"2026-10-30T09:15:01Z","response":"!","done":false}
# {"model":"llama3.1:8b","created_at":"2026-10-30T09:15:02Z","response":"","done":true,
#  "total_duration":421810583,"load_duration":12034542,"prompt_eval_count":9,
#  "prompt_eval_duration":80000000,"eval_count":2,"eval_duration":85000000}

# Step 2: count the JSON objects to confirm NDJSON
lines = [l for l in response.text.strip().split("\n") if l]
print(f"\nNumber of JSON objects: {len(lines)}")
# Number of JSON objects: 3
# ← For "Hello!": 2 token objects + 1 done=true final object.
# For a 200-token response: 200+ JSON objects in the body.

# Step 3: confirm that json.loads() raises on multi-object response
try:
    json.loads(response.text)
except json.JSONDecodeError as e:
    print(f"\njson.loads() error: {e}")
    # json.loads() error: Extra data: line 2 column 1 (char 87)
    # ← Confirmed: the standard JSON parser sees the first object
    #   as a complete document and then raises on the second object.

# Step 4: parse correctly using iter_lines()
full_text = []
for line in response.text.strip().split("\n"):
    if line:
        chunk = json.loads(line)  # parse each line individually
        if chunk.get("response"):
            full_text.append(chunk["response"])
        if chunk.get("done"):
            final = chunk  # store the final metadata chunk

print(f"\nParsed response: {''.join(full_text)!r}")
print(f"Eval count: {final.get('eval_count')} tokens")
print(f"Total duration: {final.get('total_duration', 0)/1e9:.2f}s")
# Parsed response: 'Hello!'
# Eval count: 2 tokens
# Total duration: 0.42s

The fix: add "stream": False to the payload

# content_generator.py — FIXED
# Northfield Content Processing Ltd.
# requests 2.31.0, Python 3.11, Ollama 0.3.9

import requests
import json

OLLAMA_BASE_URL = "http://localhost:11434"

def generate_content(prompt: str) -> str:
    """Generate content via Ollama REST API with streaming disabled."""
    response = requests.post(
        f"{OLLAMA_BASE_URL}/api/generate",
        json={
            "model": "llama3.1:8b",
            "prompt": prompt,
            "stream": False,
            # ← FIX: explicit stream=False.
            # Ollama waits for the entire generation to complete,
            # then returns a single JSON object with the full response text.
            # response.json() can now parse it correctly.
            # Trade-off: client waits 2-8 seconds for full generation
            # before receiving any data. Acceptable for batch pipelines.
            # For UIs that need real-time token display, use the
            # streaming approach below instead.
            "options": {
                "temperature": 0.7,
                "num_predict": 512,   # max tokens to generate
                "num_ctx": 4096,      # context window size
            }
        },
        timeout=120,
    )
    response.raise_for_status()
    data = response.json()  # ← now returns single JSON object
    assert data["done"] is True, f"Unexpected: done={data.get('done')}"
    return data["response"]


def generate_content_streaming(prompt: str) -> str:
    """Alternative: use streaming mode and collect tokens with iter_lines().

    Use this when the caller needs real-time token display (e.g., a web UI
    that proxies the Ollama stream to a browser via SSE).
    """
    response = requests.post(
        f"{OLLAMA_BASE_URL}/api/generate",
        json={"model": "llama3.1:8b", "prompt": prompt, "options": {"num_predict": 512}},
        stream=True,   # ← requests stream=True: don't buffer response body
        timeout=120,
    )
    response.raise_for_status()

    tokens = []
    for line in response.iter_lines():
        if not line:
            continue
        chunk = json.loads(line)   # ← parse each NDJSON line individually
        token = chunk.get("response", "")
        if token:
            tokens.append(token)
        if chunk.get("done"):
            break   # final chunk received; stop iteration

    return "".join(tokens)


# After fix (stream=False):
# 0 JSONDecodeError. 3,200 daily content jobs complete successfully.
# Average generation time: 3.4 seconds per 200-token response.
# Throughput: 3,200 jobs / 8 hours = 6.7 jobs/minute → well within RTX 4090 capacity.

# Validation test:
def validate_ollama_integration():
    known_prompt = "Say exactly: OK"
    result = generate_content(known_prompt)
    assert "OK" in result, f"Unexpected output: {result!r}"
    assert len(result) > 0
    print(f"Validation passed: {result!r}")
    return True

validate_ollama_integration()
# Validation passed: 'OK'

Using the Ollama Python library instead of raw HTTP

The ollama Python package (install with pip install ollama) wraps the REST API and handles the streaming/non-streaming distinction for you. It is the recommended approach for new Python integrations because it abstracts away the NDJSON parsing issue entirely.

# Using the official ollama Python library
# ollama 0.3.9, Python 3.11
# pip install ollama

import ollama

def generate_with_sdk(prompt: str) -> str:
    """Generate content via official ollama Python SDK."""
    response = ollama.generate(
        model="llama3.1:8b",
        prompt=prompt,
        # stream defaults to False in the ollama Python library
        # when called as a direct function (not as a context manager)
        options={"temperature": 0.7, "num_predict": 512}
    )
    # response is an ollama.GenerateResponse object (a Pydantic model)
    # response.response contains the generated text as a string
    return response.response

def chat_with_sdk(messages: list[dict]) -> str:
    """Multi-turn chat via ollama Python SDK."""
    response = ollama.chat(
        model="llama3.1:8b",
        messages=messages,
        # Corresponds to POST /api/chat endpoint
        # messages format: [{"role": "user", "content": "..."}, ...]
    )
    # response is an ollama.ChatResponse object
    # response.message.content contains the assistant reply
    return response.message.content

# After switching to the SDK:
# No JSON parsing code needed. No stream=true/false distinction.
# The SDK handles both correctly.
# response.response (generate) and response.message.content (chat) are plain Python strings.

# For streaming with the SDK (real-time token display):
def stream_with_sdk(prompt: str):
    """Stream tokens as they are generated."""
    stream = ollama.generate(
        model="llama3.1:8b",
        prompt=prompt,
        stream=True,   # ← SDK returns a generator of GenerateResponse objects
    )
    for chunk in stream:
        if chunk.response:
            yield chunk.response
        if chunk.done:
            break

Validation

# Pipeline validation (October 29 2026)
# 200 test jobs with known prompts and expected outputs

# Before fix:
# Success rate: 0%
# Error: json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 87)
# All 3,200 daily jobs failing. Pipeline dead.

# After fix (stream=False in payload):
# Success rate: 100% (200/200 test jobs)
# Average latency: 3,412ms per job (model warm in VRAM after first request)
# First-request cold-start: 18.4s (llama3.1:8b loading into 24GB VRAM)
# Subsequent requests: 2.8-4.1s depending on output length
# JSONDecodeError count: 0

# Production monitoring (October 30 2026, first 4 hours post-fix):
# Total jobs: 1,067
# Success: 1,067 (100%)
# Errors: 0
# Avg latency: 3.2s
# P95 latency: 5.8s (longer output jobs)
# GPU VRAM used: 8.4GB / 24GB (llama3.1:8b Q4_K_M)
# Headroom: 15.6GB available (enough for a second llama3.1:8b or one llama3.1:70b)

# Note on cold-start handling added to the fixed implementation:
# First request after Ollama server (re)start loads model into VRAM.
# Solution: startup health check that fires a warm-up request on application boot.
import requests
def warmup_ollama():
    """Load model into VRAM before serving production traffic."""
    requests.post(
        "http://localhost:11434/api/generate",
        json={"model": "llama3.1:8b", "prompt": "ok", "stream": False, "options": {"num_predict": 1}},
        timeout=120,   # first load may take 10-60s
    )

# Call warmup_ollama() during application startup, before the request handler starts.

Related retainer posts

• OpenRouter developer on retainer — HTTP 200 with empty choices array when HTTP-Referer header is missing for Anthropic models — openai 1.40.0 — IndexError on response.choices[0] — 100% of Claude calls fail 4 weeks
• vLLM developer on retainer — continuous batching causes p99 latency spike when request queue depth exceeds prefill memory budget — vllm 0.4.2
• Modal developer on retainer — lazy CLIP model initialization races Modal container pre-heating probe — 8% of product embeddings are zero vectors — modal 0.63.47