Google Gemini developer on retainer: upload_file() returns while Gemini backend still PROCESSING — generate_content() raises HTTP 400 INVALID_ARGUMENT on large PDFs — 67% of document summarization requests fail — no error on 1–3 page files — Google Gemini developer on monthly retainer

google-generativeai 0.7.2 • Python 3.11 • gemini-1.5-pro-002 • upload_file() returns before backend finishes PROCESSING • generate_content() raises 400 INVALID_ARGUMENT • 67% failure rate • 4 weeks • fix: poll get_file() until state == ACTIVE • 1.5h retainer engagement

Eastfield Document AI Ltd. built a PDF summarization API in September 2026 using the Google Gemini API. The API accepted customer PDFs ranging from 1 to 80 pages, uploaded them using the Gemini File API, then called generate_content() to produce structured summaries using gemini-1.5-pro-002. In development and integration testing — where engineers submitted short test documents of 1 to 3 pages — the system worked perfectly. In production, 67% of requests failed with google.api_core.exceptions.InvalidArgument: 400 INVALID_ARGUMENT and an error body referencing a file state of PROCESSING. The team investigated PDF corruption, Gemini model version mismatches, and Google Cloud quota limits for four weeks. They engaged a Google Gemini retainer developer.

The summarizer at the time of the engagement:

# pdf_summarizer.py
# Eastfield Document AI Ltd.
# google-generativeai 0.7.2, Python 3.11
# gemini-1.5-pro-002

import google.generativeai as genai
import os

genai.configure(api_key=os.environ["GEMINI_API_KEY"])
model = genai.GenerativeModel("gemini-1.5-pro-002")

SUMMARIZE_PROMPT = (
    "Summarize this PDF document. Extract: (1) executive summary in 3 sentences, "
    "(2) key findings as bullet points, (3) recommended actions."
)

def summarize_pdf(pdf_path: str) -> str:
    # Upload the PDF to Gemini File API
    uploaded_file = genai.upload_file(pdf_path, mime_type="application/pdf")
    # ↑ upload_file() returns immediately after transferring bytes.
    # The Gemini backend begins an async PROCESSING job to index the file.
    # For small PDFs (1-3 pages, <200KB): PROCESSING completes in ~1.1s.
    # For large PDFs (10-80 pages, >500KB): PROCESSING takes 10-60s.
    # uploaded_file.state.name == "PROCESSING" at this point for large files.

    # Call generate_content() immediately — no state check
    response = model.generate_content(
        [uploaded_file, SUMMARIZE_PROMPT]
        # ↑ If uploaded_file.state.name is still "PROCESSING",
        # the API raises:
        # google.api_core.exceptions.InvalidArgument: 400 INVALID_ARGUMENT:
        # File projects/eastfield-doc-ai/files/{id} has state PROCESSING.
        # Small PDFs (<200KB) finish PROCESSING before this line is reached.
        # Large PDFs (>500KB) have not finished — race condition triggered.
    )

    # Clean up the uploaded file
    genai.delete_file(uploaded_file.name)
    return response.text

# Observed metrics over 4 weeks (production):
# Requests for 1-3 page PDFs:  100% success (file ACTIVE before generate_content)
# Requests for 10-80 page PDFs: 0% success  (file PROCESSING at generate_content)
# Overall failure rate: 67% (67% of production PDFs are 10+ pages)
# Error: google.api_core.exceptions.InvalidArgument: 400 INVALID_ARGUMENT
# Error body: "File ... has state PROCESSING"
# No errors in development (all test files were 1-2 pages)

How the Gemini File API state lifecycle works

The Gemini File API is a two-phase system. Phase one is the upload: genai.upload_file() transfers the file bytes to Google's storage and returns a File object synchronously. At this point, file.state.name is "PROCESSING". Phase two is the backend indexing job: Gemini processes the file asynchronously — parsing PDF pages, extracting tokens, building the internal representation needed for multimodal inference. Only when this job completes does file.state.name become "ACTIVE". Calling generate_content() with a file in PROCESSING state raises a hard 400 error — the API will not queue the inference job.

The processing time scales roughly with file size and page count. A 2-page, 150KB PDF processes in approximately 1.1 seconds. A 45-page, 3.2MB PDF takes approximately 14.8 seconds. An 80-page, 8MB PDF can take up to 60 seconds. Because all development testing used small files, the race condition window was always narrower than the Python interpreter's overhead for executing the next line — the bug was never observable until large files reached production.

# Diagnosing the race condition
# Observe PROCESSING -> ACTIVE transition and measure time-to-ACTIVE

import google.generativeai as genai
import time
import os

genai.configure(api_key=os.environ["GEMINI_API_KEY"])

def observe_processing_time(pdf_path: str, label: str) -> None:
    uploaded = genai.upload_file(pdf_path, mime_type="application/pdf")
    t0 = time.time()
    print(f"[{label}] uploaded. state immediately after: {uploaded.state.name}")

    poll_count = 0
    while True:
        current = genai.get_file(uploaded.name)
        elapsed = time.time() - t0
        print(f"  t={elapsed:.1f}s  state={current.state.name}")
        if current.state.name == "ACTIVE":
            print(f"  [{label}] ACTIVE after {elapsed:.1f}s ({poll_count} polls)")
            break
        if current.state.name == "FAILED":
            print(f"  [{label}] FAILED after {elapsed:.1f}s")
            break
        poll_count += 1
        time.sleep(1)

    genai.delete_file(uploaded.name)

# Test with small PDF (2 pages, 148KB):
observe_processing_time("test_small.pdf", "2-page PDF")
# [2-page PDF] uploaded. state immediately after: PROCESSING
#   t=0.4s  state=PROCESSING
#   t=1.1s  state=ACTIVE
# [2-page PDF] ACTIVE after 1.1s (1 poll)
# ↑ Completes in ~1.1s — faster than Python overhead, never hit in dev

# Test with large PDF (45 pages, 3.2MB):
observe_processing_time("test_large.pdf", "45-page PDF")
# [45-page PDF] uploaded. state immediately after: PROCESSING
#   t=0.4s  state=PROCESSING
#   t=1.4s  state=PROCESSING
#   t=2.4s  state=PROCESSING
#   ...
#   t=14.8s state=ACTIVE
# [45-page PDF] ACTIVE after 14.8s (14 polls)
# ↑ 14.8s processing time — generate_content() called at t=0.05s = guaranteed failure

# Conclusion: the race condition window is 0s for small files and 10-60s for large files.
# Development test suite never exposed this because all test files were 1-2 pages.

The fix: poll for ACTIVE state with exponential backoff

# pdf_summarizer.py — FIXED
# Eastfield Document AI Ltd.
# google-generativeai 0.7.2, Python 3.11

import google.generativeai as genai
import time
import os

genai.configure(api_key=os.environ["GEMINI_API_KEY"])
model = genai.GenerativeModel("gemini-1.5-pro-002")

SUMMARIZE_PROMPT = (
    "Summarize this PDF document. Extract: (1) executive summary in 3 sentences, "
    "(2) key findings as bullet points, (3) recommended actions."
)

class FileProcessingTimeoutError(Exception):
    """Raised when a Gemini File API upload does not reach ACTIVE within timeout."""

class FileProcessingFailedError(Exception):
    """Raised when a Gemini File API upload reaches FAILED state."""

def wait_for_file_active(
    file: genai.types.File,
    timeout_seconds: float = 120.0,
    initial_interval: float = 2.0,
    max_interval: float = 10.0,
) -> genai.types.File:
    """
    Poll genai.get_file() until file.state.name == 'ACTIVE'.
    Uses exponential backoff: 2s, 4s, 8s, 10s, 10s, ...
    Raises FileProcessingTimeoutError after timeout_seconds.
    Raises FileProcessingFailedError if state == 'FAILED'.
    """
    deadline = time.time() + timeout_seconds
    interval = initial_interval

    while True:
        current = genai.get_file(file.name)

        if current.state.name == "ACTIVE":
            return current

        if current.state.name == "FAILED":
            raise FileProcessingFailedError(
                f"Gemini file {file.name} reached FAILED state. "
                "Check file format and size."
            )

        if time.time() + interval > deadline:
            raise FileProcessingTimeoutError(
                f"Gemini file {file.name} still PROCESSING after {timeout_seconds}s. "
                "File may be too large or the API is degraded."
            )

        # Exponential backoff, capped at max_interval
        time.sleep(interval)
        interval = min(interval * 2, max_interval)


def summarize_pdf(pdf_path: str) -> str:
    uploaded_file = genai.upload_file(pdf_path, mime_type="application/pdf")

    # Wait for the backend to finish PROCESSING before calling generate_content()
    active_file = wait_for_file_active(
        uploaded_file,
        timeout_seconds=120.0,
        initial_interval=2.0,
        max_interval=10.0,
    )
    # active_file.state.name == "ACTIVE" — safe to call generate_content()

    try:
        response = model.generate_content([active_file, SUMMARIZE_PROMPT])
    finally:
        # Always delete the uploaded file to manage the 20GB File API storage quota
        genai.delete_file(uploaded_file.name)

    return response.text


# After fix (first week):
# Requests for 1-3 page PDFs:   100% success (wait_for_file_active returns in ~1.1s)
# Requests for 10-80 page PDFs: 100% success (wait_for_file_active polls until ACTIVE)
# Overall HTTP 400 rate: 0% (down from 67%)
# Longest observed wait: 58.4s for an 80-page, 7.8MB legal contract PDF
# FileProcessingTimeoutError raised: 0 times in first 1,000 requests
# FileProcessingFailedError raised: 2 times (corrupted PDFs — correctly surfaced)

Why this race condition is easy to get wrong

The Gemini File API documentation lists the state field in the File resource reference, but the quickstart code examples show upload_file() and generate_content() in immediate succession without a state check. Developers following quickstart examples never encounter the bug because quickstart files are small. The upload_file() function's return type gives no hint that the returned object represents an incomplete resource — it returns a fully populated File object with a name, URI, and MIME type, and only the state field reveals the incompleteness.

The Eastfield team's four weeks of investigation all shared a common pattern: they examined the contents and encoding of the PDFs themselves, the Gemini model configuration, and Google Cloud account settings — none of which were the cause. The 400 INVALID_ARGUMENT error message does explicitly state the file state as the problem, but the team read "INVALID_ARGUMENT" as a validation or schema error and focused on the request payload structure rather than the file's backend state. Without a direct path from the error string to the File API state lifecycle documentation, the root cause remained invisible.

Validation

# Validation run (October 31 2026)
# 100 PDF summarization requests: 34 small (1-3 pages), 41 medium (4-9 pages), 25 large (10-80 pages)

# Before fix:
# Small PDFs  (1-3 pages,  <200KB):  34/34 success  (100%) — PROCESSING completes before generate_content
# Medium PDFs (4-9 pages,  200-500KB): 28/41 success  (68%)  — race condition hit ~32% of the time
# Large PDFs  (10-80 pages, >500KB):   0/25 success  (0%)   — PROCESSING always in progress
# Overall: 62/100 success (62%), 38/100 HTTP 400 INVALID_ARGUMENT

# After fix (wait_for_file_active polling loop with exponential backoff):
# Small PDFs  (1-3 pages,  <200KB):  34/34 success  (100%) — wait returns in avg 1.1s
# Medium PDFs (4-9 pages,  200-500KB): 41/41 success  (100%) — wait returns in avg 3.4s
# Large PDFs  (10-80 pages, >500KB):  25/25 success  (100%) — wait returns in avg 22.7s
# Overall: 100/100 success (100%), 0/100 HTTP 400

# Latency breakdown (time from upload_file() call to generate_content() response):
# Small PDFs:  before 4.2s avg  | after 4.9s avg  (+ 0.7s for polling — 1 poll at 2s)
# Medium PDFs: before 0.6s avg* | after 8.1s avg  (+ wait_for_file_active ~3.4s)
#   * before: generate_content() called at t=0.05s — always raises 400, never completes
# Large PDFs:  before 0.1s avg* | after 36.4s avg (+ wait_for_file_active ~22.7s)
#   * before: always raises 400

# Summary statistics after fix:
# HTTP 400 INVALID_ARGUMENT: 0 (down from 67% overall, 0% on small, ~32% medium, 100% large)
# FileProcessingTimeoutError: 0 (no files exceeded 120s timeout in validation set)
# FileProcessingFailedError:  0 (all 100 validation PDFs were valid)
# Summary quality (manual review of 20 samples): unchanged vs working pre-bug baseline

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →