xAI developer on retainer: grok-beta has live web search but grok-1 does not — 40% of news summaries return outdated results — model name inconsistency across codebase — 4 weeks — xAI Grok developer on monthly retainer

openai 1.40.0 (OpenAI-compatible) • Python 3.11 • xAI Grok API (api.x.ai) • grok-beta has live web search • grok-1 uses static training data only • 40% of calls use grok-1 from copy-pasted code • outdated results framed as current • 4 weeks • fix: standardize to MODEL = 'grok-beta' constant • 1.5h retainer engagement

Eastfield Real-Time AI Ltd. built a news summarization service in September 2026 using the xAI Grok API via the OpenAI-compatible Python SDK. Grok's live web search capability was the primary reason for choosing xAI over other providers — summaries needed to reference current events, not a static training corpus. The service worked correctly in initial testing. After four weeks in production, the team noticed that approximately 40% of summaries referenced events from several months ago as if they were recent news. Grok's live search appeared to be unreliable. The team spent four weeks adjusting prompt engineering, inspecting rate limit behaviour, and reviewing xAI documentation on search configuration. They engaged an xAI retainer developer.

The summarization client code at the time of the engagement:

# summarizer.py  (initial module — written from scratch)
# Eastfield Real-Time AI Ltd.
# openai 1.40.0, Python 3.11

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["XAI_API_KEY"],
    base_url="https://api.x.ai/v1",
)

# Model string set correctly in the first module
def summarize_topic(topic: str) -> str:
    response = client.chat.completions.create(
        model="grok-beta",    # ← live web search enabled
        messages=[
            {"role": "system", "content": "You are a news summarizer. Summarize the latest developments on the given topic."},
            {"role": "user", "content": f"Summarize recent news about: {topic}"},
        ],
        max_tokens=512,
    )
    return response.choices[0].message.content
# batch_processor.py  (added 2 weeks later — copied from a GitHub tutorial)
# The tutorial used "grok-1" — an early model name from xAI's initial release

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["XAI_API_KEY"],
    base_url="https://api.x.ai/v1",
)

def process_batch(topics: list[str]) -> list[str]:
    results = []
    for topic in topics:
        response = client.chat.completions.create(
            model="grok-1",   # ← BUG: grok-1 is a legacy alias
                              # ← grok-1 does NOT have live web search
                              # ← grok-1 uses static training data only
                              # ← xAI API accepts grok-1 without error
                              # ← response is well-formed — no indication of stale data
                              # ← response.model == "grok-1" (logged but not checked)
            messages=[
                {"role": "system", "content": "You are a news summarizer."},
                {"role": "user", "content": f"Summarize recent news about: {topic}"},
            ],
            max_tokens=512,
        )
        results.append(response.choices[0].message.content)
    return results

# daily_digest.py  (added 3 weeks later — copied from internal wiki)
# The wiki snippet also used "grok-1"

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["XAI_API_KEY"],
    base_url="https://api.x.ai/v1",
)

def generate_digest(categories: list[str]) -> dict[str, str]:
    digest = {}
    for category in categories:
        response = client.chat.completions.create(
            model="grok-1",   # ← same bug — copied from the wiki
            messages=[
                {"role": "system", "content": "Summarize today's key developments."},
                {"role": "user", "content": category},
            ],
            max_tokens=256,
        )
        digest[category] = response.choices[0].message.content
    return digest

# Traffic split at the time of the retainer engagement:
# summarize_topic() (grok-beta, live search): 60% of summarization calls
# process_batch() (grok-1, static training data): 25% of calls
# generate_digest() (grok-1, static training data): 15% of calls
# Total calls with grok-1 (stale results): ~40%
#
# Observable symptom: 40% of summaries reference events from months ago
# No errors logged. No exceptions raised. All responses are valid JSON.
# grok-1 returns coherent, well-written summaries — they are just not current.

How grok-beta and grok-1 differ

The xAI Grok API offers multiple model identifiers that differ significantly in capability. grok-beta (and its successor grok-2) includes live web search access: when a query is submitted, Grok automatically retrieves relevant recent content from the web and incorporates it into the response. This is the capability that distinguishes Grok from static LLMs for time-sensitive applications. The legacy identifier grok-1, introduced during xAI's early API access period and preserved for backwards compatibility, is a previous generation model that uses only static training data. It has no live web search capability.

Both model names are accepted by the xAI API without error. Both return well-formed chat completion responses indistinguishable in structure from the OpenAI SDK's perspective. The response object does not include a field indicating whether live search was used. The only differences are in the response content — and for many queries (especially those where the correct answer has not changed recently), the difference is not visible. The inconsistency only surfaced at scale, when approximately 40% of summaries were clearly outdated.

# Diagnosing the model name inconsistency
# Step 1: grep all model= strings in the codebase

# $ grep -rn 'model=' src/ --include="*.py" | grep -E '"grok|'"'"'grok'
# src/summarizer.py:14:        model="grok-beta",
# src/batch_processor.py:18:  model="grok-1",
# src/batch_processor.py:52:  model="grok-1",
# src/daily_digest.py:15:     model="grok-1",
# src/utils/test_client.py:8: model="grok-beta",

# Result: 3 call sites use "grok-1", 2 use "grok-beta"
# Traffic analysis: grok-1 calls = 40% of production volume

# Step 2: validate that grok-1 returns stale data
from openai import OpenAI
import os, datetime

client = OpenAI(api_key=os.environ["XAI_API_KEY"], base_url="https://api.x.ai/v1")

def test_freshness(model_name: str) -> str:
    response = client.chat.completions.create(
        model=model_name,
        messages=[{"role": "user", "content": "What is today's date? What happened in the news yesterday?"}],
        max_tokens=200,
    )
    return response.choices[0].message.content

grok_beta_response = test_freshness("grok-beta")
grok_1_response    = test_freshness("grok-1")

print("grok-beta:", grok_beta_response[:200])
# Output: "Today is October 31, 2026. Yesterday's news included..."
# ↑ References current events — live search active

print("grok-1:", grok_1_response[:200])
# Output: "I don't have access to today's specific date or real-time news..."
# ↑ Falls back to static training data — confirms no live search

current_year = datetime.datetime.now().year
assert str(current_year) in grok_beta_response, "grok-beta should know the current year"
assert str(current_year) not in grok_1_response, "grok-1 does not have live data"
# Both assertions pass — confirms the capability gap between the two model names

The fix: standardize to a MODEL constant and add startup validation

# xai_client.py — shared client module (FIXED)
# openai 1.40.0, Python 3.11

from openai import OpenAI
import datetime
import os

# Single source of truth for the model name
# Changing this constant propagates to all call sites
XAI_MODEL = "grok-beta"   # ← grok-beta: live web search enabled
                           # ← do NOT use "grok-1" (no live search)
                           # ← for higher capacity: "grok-2" or "grok-2-mini"

client = OpenAI(
    api_key=os.environ["XAI_API_KEY"],
    base_url="https://api.x.ai/v1",
)

def assert_live_search_available() -> None:
    """Startup assertion: verify the configured model has live web search."""
    current_year = str(datetime.datetime.now().year)
    response = client.chat.completions.create(
        model=XAI_MODEL,
        messages=[{"role": "user", "content": "What year is it right now?"}],
        max_tokens=50,
    )
    content = response.choices[0].message.content
    if current_year not in content:
        raise RuntimeError(
            f"xAI model '{XAI_MODEL}' does not appear to have live web search. "
            f"Response did not contain the current year ({current_year}): {content!r}. "
            f"Check that XAI_MODEL is set to a live-search-enabled model (grok-beta or grok-2)."
        )

# Call at application startup — catches model name issues before any real traffic
assert_live_search_available()
# summarizer.py — FIXED (imports MODEL constant from shared client module)
from xai_client import client, XAI_MODEL

def summarize_topic(topic: str) -> str:
    response = client.chat.completions.create(
        model=XAI_MODEL,    # ← constant, not a string literal
        messages=[
            {"role": "system", "content": "You are a news summarizer. Summarize the latest developments on the given topic."},
            {"role": "user", "content": f"Summarize recent news about: {topic}"},
        ],
        max_tokens=512,
    )
    return response.choices[0].message.content

# batch_processor.py — FIXED
from xai_client import client, XAI_MODEL

def process_batch(topics: list[str]) -> list[str]:
    results = []
    for topic in topics:
        response = client.chat.completions.create(
            model=XAI_MODEL,   # ← was: "grok-1" — fixed by importing constant
            messages=[
                {"role": "system", "content": "You are a news summarizer."},
                {"role": "user", "content": f"Summarize recent news about: {topic}"},
            ],
            max_tokens=512,
        )
        results.append(response.choices[0].message.content)
    return results

# daily_digest.py — FIXED
from xai_client import client, XAI_MODEL

def generate_digest(categories: list[str]) -> dict[str, str]:
    digest = {}
    for category in categories:
        response = client.chat.completions.create(
            model=XAI_MODEL,   # ← was: "grok-1" — fixed by importing constant
            messages=[
                {"role": "system", "content": "Summarize today's key developments."},
                {"role": "user", "content": category},
            ],
            max_tokens=256,
        )
        digest[category] = response.choices[0].message.content
    return digest

# CI check (add to pre-commit hook or CI pipeline):
# grep -rn 'model="grok' src/ --include="*.py"
# Should produce zero output — all model strings replaced by XAI_MODEL constant
# If any literal "grok-*" string appears, the build fails

Why model name drift is difficult to detect

The xAI API's backwards-compatibility guarantee means that grok-1 continues to work indefinitely — requests succeed, responses are well-formed, and there is no deprecation warning at call time. The capability difference between grok-1 (static training data) and grok-beta (live web search) is invisible in the response structure: both return a choices[0].message.content string. The content itself is the only differentiator, and for many topics the content is plausible regardless of model — Grok's training data is comprehensive enough that a summary without live search reads as credible on topics the model has seen before.

The Eastfield team's debugging strategy was focused on the live search feature itself — they assumed grok-beta was sometimes failing to search the web. The actual cause was that grok-beta was not being called at all for 40% of requests. Log analysis after the fix revealed that the batch and digest modules had been running on grok-1 since their deployment, with response.model == "grok-1" logged in every response. The model field in the response object was being logged but not monitored — there were no alerts or dashboards comparing expected versus actual model names across call sites.

Validation

# Validation log — October 31 2026

# ---- Codebase grep after fix ----
# $ grep -rn 'model="grok' src/ --include="*.py"
# (no output)
# All literal "grok-*" strings replaced with XAI_MODEL constant
# CI check passes

# ---- Startup assertion ----
# assert_live_search_available() — PASSED
# XAI_MODEL = "grok-beta"
# Test query response: "Today is October 31, 2026..."
# Current year (2026) found in response — live search confirmed

# ---- Freshness test (50 time-sensitive queries) ----
# Before fix (40% grok-1, 60% grok-beta):
#   Stale summary rate (events >30 days old referenced as recent): 40%
#
# After fix (100% grok-beta):
#   Stale summary rate: 2%   (residual: ~2% of topics have genuinely slow news cycles)

# ---- Log monitoring ----
# response.model field monitored post-fix:
# 100% of calls log model = "grok-beta"
# 0% log model = "grok-1"
# Alert configured: if response.model != XAI_MODEL, page on-call

# Retainer engagement cost: 1.5h (estimated $112-172 at mid-level rates)

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →