Banana.dev developer on retainer: hardcoded callID='req-001' reused across concurrent requests — async result polling returns wrong results — 30–40% result swaps under load for 7 weeks — requests 2.31.0 — Banana.dev developer on monthly retainer

requests 2.31.0 • Python 3.11 • Banana.dev serverless GPU inference • hardcoded callID='req-001' in start request • concurrent requests overwrite each other’s callID registration • result polling returns mismatched output • 30–40% result swaps under 8-worker concurrent load • both results valid JSON — swaps undetected for 7 weeks • fix: callID=str(uuid.uuid4()) per request • 1.5h retainer engagement

Eastfield Serverless Ltd. deployed a custom document classification model on Banana.dev’s serverless GPU platform in March 2026. The pipeline classified 500 documents per day using 8 concurrent worker threads, each sending requests to the Banana.dev async inference API. A developer built the client using a Banana.dev quickstart example that included a hardcoded callID='req-001' in the request payload. The quickstart example was designed for sequential single-request usage; the callID was never intended to be static in production. Under concurrent load, 30–40% of documents received the wrong classification — they got the result from a different document’s inference job. Both results were structurally valid JSON (same model, same output schema), so the type validation layer passed. The misclassifications propagated silently through the downstream database for seven weeks before a data quality audit surfaced the problem.

Banana.dev’s async inference protocol uses a two-step pattern: (1) POST to start inference, providing a callID that registers the job; (2) GET to poll for the result, supplying the same callID to retrieve the completed output. The callID is a client-supplied correlation key. When all 8 concurrent workers used the same callID='req-001', each new start request overwrote the previous registration. When workers polled for their result, they received whatever job was last registered under 'req-001' — frequently someone else’s job.

The classification pipeline code at the time of the engagement:

# classifier.py
# Eastfield Serverless Ltd.
# requests 2.31.0, Python 3.11
# Custom Truss document classification model on Banana.dev

import requests
import time
import os

BANANA_API_KEY = os.environ["BANANA_API_KEY"]
BANANA_MODEL_KEY = os.environ["BANANA_MODEL_KEY"]

START_URL = "https://api.banana.dev/start/v4/"
CHECK_URL = "https://api.banana.dev/check/v4/"

def classify_document(document_text: str) -> dict:
    """Classify document via Banana.dev async inference."""
    # ← PROBLEM: hardcoded callID reused for every request.
    # The callID is a client-supplied correlation key in Banana.dev's protocol:
    # - POST to START registers the inference job under callID
    # - GET to CHECK retrieves the result for that callID
    #
    # When 8 concurrent workers all use callID='req-001':
    # Worker A starts → registers 'req-001' → job_A
    # Worker B starts → overwrites 'req-001' → job_B
    # Worker A polls 'req-001' → gets job_B's result  ← SWAP
    # Worker B polls 'req-001' → gets job_B's result  ← correct for B, wrong for A
    #
    # Under 8 concurrent workers with ~2s inference time:
    # → 30-40% of polls return the wrong result
    # → Both results are valid JSON → no ValidationError
    # → Misclassifications propagate silently
    call_id = "req-001"  # ← WRONG: must be unique per request

    # Step 1: Start inference
    start_payload = {
        "id": BANANA_API_KEY,
        "created": int(time.time()),
        "apiKey": BANANA_API_KEY,
        "modelKey": BANANA_MODEL_KEY,
        "callID": call_id,  # ← registers the job under this callID
        "startOnly": False,
        "modelInputs": {"text": document_text},
    }
    start_resp = requests.post(START_URL, json=start_payload)
    start_resp.raise_for_status()

    # Step 2: Poll for result
    for attempt in range(30):
        time.sleep(2)
        check_payload = {
            "id": BANANA_API_KEY,
            "created": int(time.time()),
            "longPoll": False,
            "callID": call_id,  # ← retrieves the job registered under callID
        }
        check_resp = requests.post(CHECK_URL, json=check_payload)
        check_resp.raise_for_status()
        result = check_resp.json()
        if result.get("finished"):
            # ← returns whatever job is currently registered under 'req-001'
            # Under concurrent load, frequently the wrong job
            return result["modelOutputs"][0]

    raise TimeoutError(f"Inference did not complete after 30 polls")

How the callID correlation causes result swaps under concurrent load

The swap rate depends on the inference latency relative to the request arrival rate. For Eastfield Serverless Ltd., their classification model took approximately 1.8–2.2 seconds per document. With 8 concurrent workers firing continuously, a new start request arrived roughly every 250ms. Because inference took 1.8–2.2 seconds, at any given polling moment there were 7–8 active concurrent jobs all registered under the same 'req-001' callID. When worker A polled, the most recently registered job under 'req-001' was statistically likely to be a different worker’s job.

The reason the swap went undetected for seven weeks was the output schema. The model returned {"category": "string", "confidence": float, "subcategory": "string"} for any document. All categories were valid strings. All confidence values were valid floats between 0 and 1. A document classified as “Contract” when it should have been “Invoice” passed JSON schema validation because both “Contract” and “Invoice” are valid category strings. The downstream database simply stored whichever classification was returned. No exception was raised, no error was logged, no alert was triggered.

The fix: unique callID per request and input hash validation

# classifier.py — FIXED
# Eastfield Serverless Ltd.
# requests 2.31.0, Python 3.11

import requests
import hashlib
import time
import uuid
import os

BANANA_API_KEY = os.environ["BANANA_API_KEY"]
BANANA_MODEL_KEY = os.environ["BANANA_MODEL_KEY"]

START_URL = "https://api.banana.dev/start/v4/"
CHECK_URL = "https://api.banana.dev/check/v4/"


def classify_document(document_text: str) -> dict:
    """Classify document via Banana.dev async inference."""
    # ← FIX 1: Generate unique callID per request.
    # uuid.uuid4() generates a cryptographically random UUID.
    # Each request gets its own registration key → no overwrites under concurrency.
    call_id = str(uuid.uuid4())

    # ← FIX 2: Compute input hash to verify the result is for THIS document.
    # Embed the hash in modelInputs so the model receives and echoes it back.
    input_hash = hashlib.sha256(document_text.encode()).hexdigest()[:16]

    # Step 1: Start inference
    start_payload = {
        "id": BANANA_API_KEY,
        "created": int(time.time()),
        "apiKey": BANANA_API_KEY,
        "modelKey": BANANA_MODEL_KEY,
        "callID": call_id,        # ← FIX 1: unique UUID per request
        "startOnly": False,
        "modelInputs": {
            "text": document_text,
            "request_hash": input_hash,  # ← FIX 2: echo for verification
        },
    }
    start_resp = requests.post(START_URL, json=start_payload)
    start_resp.raise_for_status()

    # Step 2: Poll for result with timeout
    for attempt in range(30):
        time.sleep(2)
        check_payload = {
            "id": BANANA_API_KEY,
            "created": int(time.time()),
            "longPoll": False,
            "callID": call_id,  # ← FIX 1: same unique callID for this request only
        }
        check_resp = requests.post(CHECK_URL, json=check_payload)
        check_resp.raise_for_status()
        result = check_resp.json()

        if result.get("finished"):
            output = result["modelOutputs"][0]

            # ← FIX 2: Verify the result is for THIS document via input hash.
            # If the model echoes back request_hash, compare it.
            returned_hash = output.get("request_hash")
            if returned_hash and returned_hash != input_hash:
                raise ValueError(
                    f"Result swap detected: expected hash {input_hash!r}, "
                    f"got {returned_hash!r} for callID {call_id!r}"
                )

            return output

    raise TimeoutError(
        f"Inference did not complete after 30 polls (callID={call_id!r})"
    )


# ← FIX 3: For concurrent workloads, use ThreadPoolExecutor with per-thread
# UUID callIDs. Each thread generates its own callID independently.
from concurrent.futures import ThreadPoolExecutor, as_completed

def classify_batch(documents: list[str], workers: int = 8) -> list[dict]:
    """Classify a batch of documents with concurrent workers."""
    results = [None] * len(documents)
    with ThreadPoolExecutor(max_workers=workers) as executor:
        future_to_index = {
            executor.submit(classify_document, doc): i
            for i, doc in enumerate(documents)
        }
        for future in as_completed(future_to_index):
            i = future_to_index[future]
            results[i] = future.result()  # ← each future has its own callID
    return results

Why seven weeks of data quality investigation found nothing in the infrastructure

The Eastfield team’s first hypothesis after discovering the misclassifications in the data quality audit was model drift: the classification model had been retrained since deployment and was performing worse than at launch. They filed a support request with Banana.dev asking about model versioning and retraining. Banana.dev confirmed that the deployed model had not changed. The team spent two weeks analyzing whether the documents themselves had changed in character — whether new document types had entered the pipeline that the model was not trained on.

The second hypothesis was race conditions in the database write layer. The team suspected that concurrent threads were writing to the same database row, with the last write overwriting earlier correct classifications. They added row-level locking and per-document transaction isolation. The misclassification rate did not change because the corruption happened before the database write, in the inference client’s result retrieval step.

The retainer developer read the Banana.dev API documentation section on the callID field, which states: “callID: A unique identifier for this specific inference call. Used to retrieve the result via the check endpoint. Must be unique per call.” The word “unique” in the documentation was the diagnosis. Adding call_id = str(uuid.uuid4()) took thirty seconds to type. The input hash verification took another fifteen minutes to implement. The concurrent batch test with 50 documents and 8 workers confirmed zero swaps. Total engagement: 1.5 hours.

Validation

# Validation benchmark (October 30, 2026)
# 50-document concurrent batch (8 workers), Banana.dev serverless GPU
# requests 2.31.0, Python 3.11

# Before fix (callID='req-001' hardcoded across all workers):
# Documents submitted:               50 / 50
# Documents with correct result:     31 / 50  (62%)
# Documents with swapped result:     19 / 50  (38%)  ← wrong document's output
# JSON schema validation passed:     50 / 50 (100%)  ← both results are valid JSON
# ValueError raised:                  0        (swap undetected by type checking)
# Team diagnosis: model drift, database race condition
# Duration: 7 weeks March–October 2026

# After fix (callID=str(uuid.uuid4()) per request + input hash verification):
# Documents submitted:               50 / 50
# Documents with correct result:     50 / 50 (100%)
# Documents with swapped result:      0 / 50   (0%)
# Input hash verification passed:    50 / 50 (100%)
# ValueError for swap detection:      0 / 50   (0 swaps triggered ValueError)
# Average inference latency (p50):  ~2.0s  (custom Truss classification model)
# Average inference latency (p95):  ~3.1s

# Production result (Eastfield Serverless Ltd.):
# Misclassification rate (from swaps):  30-40% → 0%
# Model changes:                         zero (model was correct all along)
# Database changes:                      zero (locking was not the issue)
# Infrastructure changes:                zero (Banana.dev platform was correct)
# Fix: callID=str(uuid.uuid4()) per request + input hash verification
# 1.5h retainer engagement.
# Triage: 0.3h — read Banana.dev callID docs; identify 'unique per call' requirement;
#   add callID logging; reproduce swap with 10 concurrent requests.
# Root cause: 0.3h — confirm concurrent overwrite of callID='req-001'; confirm swap rate.
# Fix: 0.4h — uuid.uuid4() per request; input hash field; hash verification assertion.
# Validation: 0.5h — 50-document concurrent batch (8 workers); 0% swaps; CI test added.

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →