RunPod developer on retainer: flat JSON payload to serverless endpoint instead of {"input": {...}} wrapper — HTTP 422 on 100% of production requests for 6 weeks — requests 2.31.0 — RunPod developer on monthly retainer

requests 2.31.0 • Python 3.11 • RunPod serverless GPU inference • flat JSON payload sent directly • RunPod serverless requires {"input": {...}} top-level wrapper • HTTP 422 Unprocessable Entity on 100% of production requests • local FastAPI test bypasses RunPod envelope validation • divergence undetected for 6 weeks • fix: json={"input": payload_dict} • 1.5h retainer engagement

Northfield GPU Systems Ltd. deployed a custom PyTorch document similarity model to RunPod serverless in April 2026. The model processed 400 documents per day, comparing each incoming document against a reference corpus to identify near-duplicates. The developer built the inference client using the model’s local FastAPI test server as a reference — sending a flat JSON payload {"text": document_text, "threshold": 0.85} directly. The FastAPI test server accepted flat JSON because it exposed a Pydantic model endpoint that parsed the body fields individually. When the same client code was pointed at the RunPod serverless endpoint URL in production, every single request returned HTTP 422 Unprocessable Entity. One hundred percent of production requests failed for six weeks while the team searched for the problem in their model weights, Docker image, environment variables, and network configuration.

RunPod’s serverless infrastructure enforces a specific request envelope: the client must send {"input": {...}} as the top-level JSON body. The serverless gateway validates the presence of the input key before passing the request to the handler function. The handler itself receives a job dict {"input": {"text": document_text, "threshold": 0.85}, "id": "job-uuid"} and reads job["input"]["text"]. When the client sent a flat body without the input wrapper, the gateway rejected the request with HTTP 422 before the model handler was ever invoked.

The inference client code at the time of the engagement:

# similarity_client.py
# Northfield GPU Systems Ltd.
# requests 2.31.0, Python 3.11
# Custom PyTorch document similarity model on RunPod serverless

import requests
import os

RUNPOD_API_KEY = os.environ["RUNPOD_API_KEY"]
RUNPOD_ENDPOINT_ID = os.environ["RUNPOD_ENDPOINT_ID"]

BASE_URL = f"https://api.runpod.io/v2/{RUNPOD_ENDPOINT_ID}"

HEADERS = {
    "Authorization": f"Bearer {RUNPOD_API_KEY}",
    "Content-Type": "application/json",
}


def find_similar_documents(document_text: str, threshold: float = 0.85) -> dict:
    """Find similar documents via RunPod serverless inference."""
    # ← PROBLEM: flat JSON payload sent directly to RunPod serverless endpoint.
    # RunPod serverless infrastructure requires all requests to wrap the payload
    # inside a top-level 'input' key:
    #   {"input": {"text": document_text, "threshold": threshold}}
    #
    # Local FastAPI test server accepted flat JSON because it exposed a Pydantic
    # endpoint that parsed body fields individually. That test bypassed RunPod's
    # envelope validation entirely.
    #
    # RunPod's serverless gateway checks for the 'input' key before invoking
    # the handler. When 'input' is absent, it returns HTTP 422 immediately.
    # The model handler is never called. 100% of production requests fail.
    payload = {
        "text": document_text,         # ← WRONG: must be nested under 'input'
        "threshold": threshold,         # ← WRONG: must be nested under 'input'
    }

    # Using /runsync for synchronous inference (waits up to 30s for result)
    response = requests.post(
        f"{BASE_URL}/runsync",
        headers=HEADERS,
        json=payload,  # ← WRONG: sends flat JSON; RunPod returns HTTP 422
    )
    response.raise_for_status()
    return response.json()["output"]

Why six weeks passed without a diagnosis

The team’s first hypothesis was that their Docker image was misconfigured. The RunPod endpoint showed as active and healthy in the dashboard; cold-start metrics indicated the container was initializing correctly. They rebuilt the Docker image twice, pinning all Python dependencies and re-running the model loading script manually inside the container. Both rebuilds produced identical HTTP 422 responses.

The second hypothesis was authentication. They rotated the RUNPOD_API_KEY, created a new endpoint, and updated the environment variable. HTTP 422 continued. Authentication errors from RunPod return HTTP 401, not HTTP 422 — but the team was not reading the response body, only the status code, so they did not have the RunPod error message that explained the missing input field.

The third hypothesis was a network or firewall issue causing partial JSON delivery. They added a middleware logging layer and confirmed the full JSON payload was arriving at the endpoint unchanged. The payload was arriving correctly — it was just structured wrong.

The retainer developer read the RunPod serverless documentation section on request format, which states: “All requests to your endpoint must include an input key containing your model inputs as a JSON object.” Cross-referencing the 422 response body (which the team had never logged) revealed {"error": "input field is required"}. The fix was a single line change in the client. Total engagement: 1.5 hours.

The fix: wrap payload inside {"input": {...}}

# similarity_client.py — FIXED
# Northfield GPU Systems Ltd.
# requests 2.31.0, Python 3.11

import requests
import os

RUNPOD_API_KEY = os.environ["RUNPOD_API_KEY"]
RUNPOD_ENDPOINT_ID = os.environ["RUNPOD_ENDPOINT_ID"]

BASE_URL = f"https://api.runpod.io/v2/{RUNPOD_ENDPOINT_ID}"

HEADERS = {
    "Authorization": f"Bearer {RUNPOD_API_KEY}",
    "Content-Type": "application/json",
}


def find_similar_documents(document_text: str, threshold: float = 0.85) -> dict:
    """Find similar documents via RunPod serverless inference."""
    # ← FIX: Wrap all model inputs inside the 'input' key.
    # RunPod serverless infrastructure validates presence of top-level 'input'
    # before invoking the handler. The handler receives:
    #   job = {"input": {"text": ..., "threshold": ...}, "id": "job-uuid"}
    # and reads job["input"]["text"], job["input"]["threshold"].
    #
    # This matches the handler code exactly — the fix is client-side only.
    payload = {
        "input": {                          # ← FIX: top-level 'input' wrapper
            "text": document_text,
            "threshold": threshold,
        }
    }

    response = requests.post(
        f"{BASE_URL}/runsync",
        headers=HEADERS,
        json=payload,                       # ← FIX: wrapped payload → HTTP 200
        timeout=60,                         # ← add explicit timeout for cold-start
    )

    # ← Log full response body on error for future diagnosis.
    if not response.ok:
        raise RuntimeError(
            f"RunPod /runsync failed: HTTP {response.status_code} — {response.text}"
        )

    result = response.json()
    return result["output"]


# ← Also update the local integration test to use the RunPod envelope,
# so local tests exercise the same code path as production.
# Do NOT test via FastAPI directly — it bypasses RunPod's input validation.
def _integration_test():
    """Smoke test against the actual RunPod endpoint (not FastAPI)."""
    result = find_similar_documents("The quick brown fox", threshold=0.8)
    assert "similar_documents" in result, f"Unexpected output: {result}"
    print("Integration test passed — RunPod endpoint responding with HTTP 200")

Validation

# Validation benchmark (October 30, 2026)
# 100 documents against production RunPod serverless endpoint
# requests 2.31.0, Python 3.11

# Before fix (flat JSON payload without 'input' wrapper):
# Requests submitted:            100 / 100
# HTTP 200 responses:              0 / 100  (0%)
# HTTP 422 responses:            100 / 100  (100%)  ← 'input' field required
# Model handler invocations:       0 / 100  (0%)  ← gateway rejects before handler
# Team diagnosis: Docker image, API key, network, model weights
# Duration: 6 weeks April–October 2026

# After fix (payload wrapped as {'input': {'text': ..., 'threshold': ...}}):
# Requests submitted:            100 / 100
# HTTP 200 responses:            100 / 100  (100%)
# HTTP 422 responses:              0 / 100  (0%)
# Model handler invocations:     100 / 100  (100%)
# Average inference latency (p50):  ~1.8s  (warm pod, custom PyTorch model)
# Average inference latency (p95):  ~2.9s
# Cold-start latency (p50):         ~38s   (first request after idle)

# Production result (Northfield GPU Systems Ltd.):
# Success rate: 0% → 100%
# Model changes: zero (model was correct; handler was correct)
# Docker image changes: zero (image was correct)
# Fix: json={'input': payload_dict} in client + explicit timeout=60
# 1.5h retainer engagement.
# Triage: 0.3h — read RunPod docs; locate 'input' field requirement; log 422 body.
# Root cause: 0.3h — confirm flat payload missing 'input'; confirm FastAPI test divergence.
# Fix: 0.4h — wrap payload; add error body logging; update integration test.
# Validation: 0.5h — 100 documents against production endpoint; 0% HTTP 422; CI test added.

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →