Portkey developer on retainer: fallback from GPT-3.5-turbo to Claude Haiku passes OpenAI-only name field in user messages — Anthropic rejects with HTTP 400 — both gateway targets fail — 23% of production API calls returning 500 for 6 weeks — Portkey developer on monthly retainer

portkey-ai 1.7.3 • Python 3.11 • FastAPI 0.111.0 • OpenAI 1.35.0 • Anthropic 0.28.0 • gpt-3.5-turbo primary • claude-haiku-20240307 fallback on 429 • name field in user messages (OpenAI-only) • Anthropic HTTP 400 invalid_request_error • fallback exhausted • 23% of peak-hour chat completions failing • 6 weeks August–September 2026 • fix: remove name field from user messages • 2.0h retainer engagement

Birchstone Media Lab runs a conversational AI customer support assistant built on FastAPI and powered by GPT-3.5-turbo through the Portkey AI gateway. In July 2026 they added a fallback rule to their Portkey configuration: if the primary target (GPT-3.5-turbo on Birchstone’s Tier 1 rate-limited API key) returned HTTP 429, Portkey should automatically retry the request against Claude Haiku as a cost-effective fallback. The configuration looked correct, the Portkey dashboard confirmed the fallback was active, and a quick integration test — manually sending a test message and forcing a 429 from the primary — returned a response from Claude Haiku. The team deployed the configuration and considered the cost-optimization initiative complete. Six weeks later, after a customer success manager noticed that users were consistently seeing “service unavailable” errors during the afternoon peak traffic window, a senior engineer reviewed the Portkey request logs in detail and discovered that 100% of the fallback attempts to Claude Haiku were failing with HTTP 400. The root cause was a single field in the message payload that the integration test had not used.

Birchstone’s chat_service.py built message arrays for each conversation turn. The utility function included the name field on user messages to track which user was speaking in each conversation turn — a pattern documented in the OpenAI API reference:

# chat_service.py
# Birchstone Media Lab
# portkey-ai 1.7.3, Python 3.11, FastAPI 0.111.0

from portkey_ai import Portkey
import os

portkey_client = Portkey(
    api_key=os.environ["PORTKEY_API_KEY"],
    config={
        "strategy": {"mode": "fallback"},
        "targets": [
            {
                "virtual_key": "openai-tier1",
                "model": "gpt-3.5-turbo",
                "on_status_codes": [429],
            },
            {
                "virtual_key": "anthropic-haiku",
                "model": "claude-haiku-20240307",
            },
        ],
    },
)


def build_messages(conversation_history: list, user_id: str, new_message: str) -> list:
    """Build message array for Portkey/OpenAI from conversation history."""
    messages = [
        {
            "role": "system",
            "content": "You are a helpful customer support assistant for Birchstone Media Lab."
        }
    ]

    for turn in conversation_history:
        # Add name field to track user identity in multi-turn conversations
        # BUG: name is an OpenAI-specific extension — Anthropic rejects it with 400
        messages.append({
            "role": "user",
            "content": turn["user"],
            "name": turn["user_id"],   # e.g. "usr_a3b4c5d6"
        })
        messages.append({
            "role": "assistant",
            "content": turn["assistant"],
        })

    messages.append({
        "role": "user",
        "content": new_message,
        "name": user_id,   # BUG: same issue on the current turn's message
    })

    return messages


async def get_chat_response(user_id: str, conversation_history: list, message: str) -> str:
    """Call LLM through Portkey gateway with fallback."""
    messages = build_messages(conversation_history, user_id, message)

    response = portkey_client.chat.completions.create(
        messages=messages,
        max_tokens=512,
        temperature=0.7,
    )
    return response.choices[0].message.content

The name field on user messages is documented in the OpenAI chat completions API and accepted without issue by GPT-3.5-turbo. OpenAI describes it as “an optional name for the participant. Provides the model information to differentiate between participants of the same role.” Birchstone’s integration test sent a single message with no conversation history and a fixed user_id, then manually simulated a 429 to trigger the fallback. The integration test message had the name field. However, the test response came back successfully from Claude Haiku because — in hindsight — the test had been run against a different name value format. In production, user_id was a UUID prefixed with usr_: usr_a3b4c5d6. In the integration test, it had been a plain name string: test_user. Anthropic’s API rejected both formats identically with HTTP 400, but the integration test’s Claude Haiku call had actually gone to OpenAI’s API due to a misconfigured virtual key in the test environment that pointed at the wrong backend. The integration test had validated nothing about Claude Haiku’s behavior.

In production, the failure chain was:

1. Peak traffic window: ~75 requests per minute against a GPT-3.5-turbo Tier 1 key with a 60 RPM limit.
2. ~23% of requests (approximately 15 per minute) receive HTTP 429 from OpenAI.
3. Portkey correctly detects 429 and triggers fallback to Claude Haiku.
4. Portkey forwards the original request payload — including messages with name fields — to the Anthropic API.
5. Anthropic returns HTTP 400: {"type": "error", "error": {"type": "invalid_request_error", "message": "Extra inputs are not permitted"}}.
6. Portkey receives the 400 response. HTTP 400 is not in Portkey’s default fallback trigger list (which includes 429, 500, 502, 503, 504), so Portkey does not attempt a third target.
7. Portkey returns HTTP 500 to Birchstone’s FastAPI application.
8. FastAPI’s exception handler catches the error and returns {"error": "service_unavailable"} to the front end.
9. The front end displays a generic “service unavailable” toast notification.

The application logs showed 500 from Portkey gateway entries but no information about why. Portkey’s logs — which Birchstone had not configured to forward to their central logging system — showed the full detail: every 500 response corresponded to a fallback attempt to Claude Haiku that was rejected with 400. But nobody was reading the Portkey logs. The team spent six weeks assuming the errors were transient rate limiting events or intermittent Anthropic service degradation.

The retainer developer’s first action was to query the Portkey logs dashboard with a filter for status=500, grouped by fallback target:

# Portkey log dashboard query results (6-week period Aug 12–Sep 23, 2026):
#
# Status   | Target             | Count  | Fallback 400 rate
# ---------|--------------------|--------|------------------
# 200 OK   | gpt-3.5-turbo      | 47,821 | N/A
# 200 OK   | claude-haiku-...   | 0      | N/A
# 500      | claude-haiku-...   | 14,362 | 100.0%
#
# All 14,362 fallback attempts to Claude Haiku returned HTTP 400.
# Zero successful Claude Haiku responses in 6 weeks.
# Root cause: Portkey is routing to Claude Haiku correctly,
# but every request is rejected by Anthropic.

# Next: inspect actual 400 error body from Anthropic
# From Portkey log detail for request_id=pk_req_7a3b4c5d:
# {
#   "type": "error",
#   "error": {
#     "type": "invalid_request_error",
#     "message": "Extra inputs are not permitted",
#     "param": "messages.0.name"   ← the specific field Anthropic rejects
#   }
# }

The Portkey log detail for any individual 500 response showed the Anthropic error body: "param": "messages.0.name". The first user message in the conversation history had a name field. Anthropic’s Messages API does not accept name on any message type. The fix was a one-line change in build_messages():

# Fixed version — remove name field from all user messages
def build_messages(conversation_history: list, user_id: str, new_message: str) -> list:
    """Build message array compatible with both OpenAI and Anthropic via Portkey."""
    messages = [
        {
            "role": "system",
            "content": "You are a helpful customer support assistant for Birchstone Media Lab."
        }
    ]

    for turn in conversation_history:
        # Remove name field — Anthropic rejects it with 400 invalid_request_error
        # user_id tracking moved to a separate metadata field in the application layer
        messages.append({
            "role": "user",
            "content": turn["user"],
        })
        messages.append({
            "role": "assistant",
            "content": turn["assistant"],
        })

    messages.append({
        "role": "user",
        "content": new_message,
        # name field removed
    })

    return messages

The user_id tracking that the name field had been providing was moved to a metadata field stored alongside the conversation record in Birchstone’s database — the application already had the user_id in the session context and did not need it embedded in the message array. The fix was deployed in the same PR.

After deploying the fix, the first afternoon peak window produced zero 500 errors from the Portkey gateway. All 23% of requests that had previously been failing now succeeded via the Claude Haiku fallback. The Portkey logs showed successful Claude Haiku responses for the first time since the fallback configuration was deployed in July. The team had been effectively paying for Claude Haiku capacity in their Portkey plan for six weeks without receiving a single successful response.

The retainer developer also identified a secondary issue: the Portkey integration test environment had a misconfigured anthropic-haiku virtual key that pointed to the OpenAI API instead of the Anthropic API. This was the reason the integration test had appeared to pass: it had sent a test message to OpenAI (which accepts name) while believing it was testing Claude Haiku. The virtual key configuration in the test environment was corrected, and the integration test was updated to assert the model field in the response header matches the expected Anthropic model identifier. A second test was added that explicitly sends a message with a name field to the Anthropic virtual key and asserts it returns HTTP 400 — a guardrail test that would catch any future reintroduction of the name field in a code change.

The postmortem defined two new operational requirements. First, Portkey gateway logs must be forwarded to Birchstone’s central Datadog instance via Portkey’s log export webhook, with an alert configured on fallback 400 error rate > 5% over a 15-minute window. Second, any new Portkey routing configuration that adds a non-OpenAI provider to any target slot (primary or fallback) must be validated by running the application’s existing message construction utilities directly against that provider’s API using the provider’s native SDK, before deploying the Portkey configuration. The validation must include a conversation with at least five turns of history to exercise all code paths in build_messages().

Total retainer engagement: 2.0 hours (0.5h initial triage: query Portkey logs dashboard for 6-week window; filter by status=500; group by fallback target; confirm 100% failure rate on Claude Haiku fallback; inspect individual request log detail; identify "param": "messages.0.name" in Anthropic error body; 0.5h root cause: review build_messages() source; identify name field in all user message dicts; verify Anthropic Messages API documentation does not accept name on user messages; confirm OpenAI accepts name, creating silent per-provider schema incompatibility; check Portkey integration test virtual key configuration and identify misconfigured test-environment anthropic-haiku key pointing to OpenAI backend; 0.5h fix: remove name field from user message dicts in build_messages(); update integration test to assert correct model identifier in response; add anti-regression test that sends name field to Anthropic virtual key and asserts 400; correct test-environment virtual key configuration; 0.5h validation: deploy fix; monitor Portkey logs for one afternoon peak window; confirm zero fallback 400 errors; confirm Claude Haiku responses appearing in logs; confirm application 500 error rate drops to 0%).

Related retainer posts

• PromptLayer developer on retainer — mutable default argument accumulates 31 tags, 256-char limit silently drops env:production tag
• Helicone developer on retainer — per-user UUID in cache key, 2.1% cache hit rate instead of expected 60%
• LangSmith developer on retainer — evaluate() dataset split mode causes 40% of test cases to be skipped silently