Cerebras developer on retainer: streaming delta.content is None for role-only header chunks — output += None raises TypeError — 8% of responses crash at token 1 for 5 weeks — cerebras-cloud-sdk 1.0.0 — Cerebras developer on monthly retainer

cerebras-cloud-sdk 1.0.0 • Python 3.11 • llama3.1-70b on Cerebras Cloud • OpenAI SDK with base_url="https://api.cerebras.ai/v1" • stream=True • first streaming chunk: delta.role="assistant", delta.content=None • output += None raises TypeError • 8% crash rate for 5 weeks • fix: delta = chunk.choices[0].delta.content or '' • 1.5h retainer engagement

Eastbrook Compute Ltd. migrated their customer-facing chatbot to Cerebras Cloud in May 2026 to take advantage of Cerebras’s extremely fast inference throughput. The migration used the OpenAI SDK with Cerebras’s OpenAI-compatible endpoint, making the SDK switch straightforward. The chatbot ran correctly on non-streaming requests. When streaming was enabled to show tokens in real time, 8% of requests returned an HTTP 500 error to users. The failure appeared intermittent and the team suspected a load-related race condition, spending five weeks investigating infrastructure before engaging a Cerebras retainer developer.

The streaming integration at the time of the engagement:

# chatbot.py
# Eastbrook Compute Ltd.
# cerebras-cloud-sdk 1.0.0, Python 3.11
# Using OpenAI SDK with Cerebras base_url

from openai import OpenAI

# Cerebras Cloud exposes an OpenAI-compatible endpoint.
# The API key comes from the Cerebras developer console.
client = OpenAI(
    base_url="https://api.cerebras.ai/v1",
    api_key="CEREBRAS_API_KEY",
)

def stream_response(messages: list) -> str:
    response = client.chat.completions.create(
        model="llama3.1-70b",
        messages=messages,
        stream=True,
        max_tokens=1024,
        temperature=0.7,
    )

    output = ""
    for chunk in response:
        # ← PROBLEM: delta.content is None for the role-only header chunk.
        # Cerebras (like the OpenAI API) sends a first chunk that establishes
        # the message role before any content tokens arrive:
        #   chunk.choices[0].delta.role = "assistant"
        #   chunk.choices[0].delta.content = None   ← None, not ""
        # Subsequent chunks carry the actual tokens:
        #   chunk.choices[0].delta.role = None
        #   chunk.choices[0].delta.content = "Hello"  ← token string
        #
        # output += None raises TypeError: cannot concatenate str and NoneType.
        # The exception is raised on the very first chunk, before any text appears.
        delta = chunk.choices[0].delta.content   # ← may be None
        output += delta                          # ← TypeError on first chunk

    return output

Why streaming sends a None content chunk first

The OpenAI Chat Completions streaming protocol delivers responses as Server-Sent Events (SSE). Each event is a ChatCompletionChunk object. The first chunk in a streaming response sets the message role (delta.role = "assistant") and carries no content: delta.content = None. Subsequent chunks carry individual tokens in delta.content as string values, with delta.role = None. The final chunk has delta.content = None again and sets finish_reason to "stop" or "length".

This means a streaming response always starts and ends with delta.content = None. Client code that assumes delta.content is always a string and uses output += delta.content directly will raise TypeError: can only concatenate str (not ‘NoneType’) to str at the very first chunk, before any content is received.

The 8% failure rate at Eastbrook puzzled the team because the non-streaming path (same request, stream=False) succeeded 100% of the time. The streaming path used a different code branch in their FastAPI handler, and the branch was gated behind a feature flag that had been gradually rolled out to 8% of users — exactly matching the failure rate. The “intermittent under load” description came from the phased rollout, not from a concurrency issue.

# Demonstrating streaming chunk structure
# cerebras-cloud-sdk 1.0.0, Python 3.11
# OpenAI SDK with Cerebras base_url

from openai import OpenAI

client = OpenAI(
    base_url="https://api.cerebras.ai/v1",
    api_key="CEREBRAS_API_KEY",
)

response = client.chat.completions.create(
    model="llama3.1-70b",
    messages=[{"role": "user", "content": "Say hello in one word."}],
    stream=True,
    max_tokens=10,
)

for i, chunk in enumerate(response):
    delta = chunk.choices[0].delta
    finish = chunk.choices[0].finish_reason
    print(f"chunk {i}: role={delta.role!r:12} content={delta.content!r:12} finish={finish!r}")

# Output:
# chunk 0: role='assistant'  content=None        finish=None      ← role-only header
# chunk 1: role=None         content='Hello'     finish=None      ← first token
# chunk 2: role=None         content='!'         finish=None
# chunk 3: role=None         content=None        finish='stop'    ← final, no content

# output = ""
# output += None   → TypeError at chunk 0, before any content token arrives

# The TypeError propagates immediately; no tokens are accumulated.
# FastAPI exception handler: returns HTTP 500.
# User sees an error instead of a streaming response.

The fix: guard against None with or ''

# chatbot.py — FIXED
# Eastbrook Compute Ltd.
# cerebras-cloud-sdk 1.0.0, Python 3.11

from openai import OpenAI

client = OpenAI(
    base_url="https://api.cerebras.ai/v1",
    api_key="CEREBRAS_API_KEY",
)

def stream_response(messages: list) -> str:
    response = client.chat.completions.create(
        model="llama3.1-70b",
        messages=messages,
        stream=True,
        max_tokens=1024,
        temperature=0.7,
    )

    output = ""
    for chunk in response:
        # ← FIX: use 'or ""' to convert None to empty string.
        # delta.content is None for the role-only header chunk and the final
        # finish_reason chunk. The 'or ""' guard converts None to '' so that
        # string concatenation never sees a NoneType operand.
        delta = chunk.choices[0].delta.content or ""
        output += delta

    return output

async def stream_response_fastapi(messages: list):
    """
    FastAPI streaming endpoint — yields tokens as they arrive.
    Guards against None delta.content the same way.
    """
    response = client.chat.completions.create(
        model="llama3.1-70b",
        messages=messages,
        stream=True,
        max_tokens=1024,
    )
    for chunk in response:
        token = chunk.choices[0].delta.content or ""
        if token:  # only yield non-empty tokens (skip role and finish chunks)
            yield token

The native cerebras-cloud-sdk path and its streaming pattern

# chatbot_native_sdk.py
# Using the native cerebras-cloud-sdk instead of OpenAI SDK
# cerebras-cloud-sdk 1.0.0, Python 3.11

from cerebras.cloud.sdk import Cerebras

client = Cerebras(api_key="CEREBRAS_API_KEY")

# Non-streaming — .content is a string directly
response = client.chat.completions.create(
    model="llama3.1-70b",
    messages=[{"role": "user", "content": "What is 2+2?"}],
    max_tokens=32,
)
print(response.choices[0].message.content)  # "4"

# Streaming — same delta.content pattern as OpenAI SDK
# The same None guard is required: delta.content or ""
stream = client.chat.completions.create(
    model="llama3.1-70b",
    messages=[{"role": "user", "content": "What is 2+2?"}],
    stream=True,
    max_tokens=32,
)

output = ""
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""  # ← same fix, same reason
    output += delta

print(output)  # "4"

# Summary of streaming chunk sequence for Cerebras:
# chunk 0: delta.role="assistant", delta.content=None  ← guard required
# chunk 1..N: delta.role=None, delta.content="token"   ← string tokens
# chunk N+1: delta.role=None, delta.content=None, finish_reason="stop"  ← guard required

Validation

# Validation benchmark (October 29 2026)
# 50 streaming requests, mix of short and long responses
# cerebras-cloud-sdk 1.0.0 (OpenAI SDK with Cerebras base_url), Python 3.11, llama3.1-70b

# Before fix (delta.content used directly without None guard):
# TypeError at chunk 0:           50 / 50 (100% of streaming requests)
# HTTP 500 returned to user:      50 / 50 (100% when streaming=True)
# Overall failure rate in prod:   8% (8% of users had streaming feature flag enabled)
# Non-streaming path (stream=False): 0% failures (different code path)
# Duration: 5 weeks May–October 2026

# After fix (delta.content or ''):
# TypeError:                       0 / 50 (0%)
# Complete responses received:    50 / 50 (100%)
# Average response length:        312 characters
# Time to first token (p50):      48ms  ← Cerebras WSE inference is fast
# Time to first token (p95):      92ms
# Throughput (tokens/sec, p50):  1,840 tok/s  ← Cerebras WSE advantage

# Production result (Eastbrook Compute Ltd.):
# 500 error rate on streaming path: 8% → 0%
# 1.5h retainer engagement.
# Triage: 0.3h — add delta.content logging before +=; confirm None on chunk 0.
# Root cause: 0.3h — confirm role-only header chunk pattern; document fix.
# Fix: 0.3h — add or '' guard; verify on 20 requests.
# Validation: 0.6h — 50-request batch; add CI streaming test; 1.5h total.

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →