Pinecone developer on retainer: metadata type mismatch silently returns empty results — priority stored as integer in new batch, queried as string — 20% of support tickets missing from RAG retrieval — LLM answers “I don’t know” — 5 weeks — Pinecone developer on monthly retainer

pinecone-client 3.2.2 • Python 3.11 • OpenAI text-embedding-3-small • {"priority": 0} (int) in new batch vs {"priority": "high"} (str) in old batch • $eq filter type-strict • 10,000 tickets invisible • 5 weeks • fix: normalize metadata types before upsert • 2.0h retainer engagement

Westfield Knowledge Retrieval Ltd. built a RAG-powered support assistant in August 2026 using Pinecone serverless and OpenAI text-embedding-3-small. The system ingested 50,000 support ticket chunks from two source systems: 40,000 tickets from a legacy CRM and 10,000 from a newly migrated system. Each ticket carried a priority metadata field used to pre-filter retrieval results. For five weeks, queries about recent high-priority incidents — which were exclusively in the new-system batch — returned no relevant context, and the LLM consistently answered "I don't have enough information about recent incidents." All 50,000 upserts returned HTTP 200. Cosine similarity scores on unfiltered queries looked reasonable. The team investigated embedding model quality, chunk size, similarity thresholds, and Pinecone index configuration. They engaged a Pinecone retainer developer.

The ingestion code at the time of the engagement:

# ingest_tickets.py
# Westfield Knowledge Retrieval Ltd.
# pinecone-client 3.2.2, Python 3.11
# Embeddings: OpenAI text-embedding-3-small (1536 dims)

from pinecone import Pinecone
from openai import OpenAI
import os

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
index = pc.Index("support-tickets")
oai = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

def embed(text: str) -> list[float]:
    resp = oai.embeddings.create(model="text-embedding-3-small", input=text)
    return resp.data[0].embedding

# ---- BATCH 1: Legacy CRM tickets (40,000 chunks) ----
# Legacy system stored priority as a string label
for ticket in legacy_crm_tickets:
    vec = embed(ticket["body"])
    index.upsert(vectors=[{
        "id": f"legacy-{ticket['id']}",
        "values": vec,
        "metadata": {
            "source": "legacy_crm",
            "priority": ticket["priority"],
            # ticket["priority"] is a str: "high", "medium", or "low"
            # Pinecone stores: {"priority": "high"}
        }
    }])
# All 40,000 upserts: HTTP 200

# ---- BATCH 2: New-system tickets (10,000 chunks) ----
# New system stored priority as an integer code: 0=high, 1=medium, 2=low
for ticket in new_system_tickets:
    vec = embed(ticket["body"])
    index.upsert(vectors=[{
        "id": f"newsys-{ticket['id']}",
        "values": vec,
        "metadata": {
            "source": "new_system",
            "priority": ticket["priority_code"],
            # ticket["priority_code"] is an int: 0, 1, or 2
            # Pinecone stores: {"priority": 0}
            # ↑ Different Python type than Batch 1 — no error raised
        }
    }])
# All 10,000 upserts: HTTP 200

# Query at runtime (rag_retriever.py):
def retrieve(query: str, priority_level: str = "high") -> list[dict]:
    emb = embed(query)
    results = index.query(
        vector=emb,
        top_k=10,
        filter={"priority": {"$eq": priority_level}},
        # ↑ priority_level is always a str: "high", "medium", or "low"
        # Pinecone $eq is type-strict:
        #   {"priority": {"$eq": "high"}} matches vectors where priority == "high" (str)
        #   It does NOT match vectors where priority == 0 (int)
        # Result: 40,000 legacy vectors are searchable; 10,000 new-system vectors are invisible
        include_metadata=True,
    )
    return results["matches"]

# Observed symptoms over 5 weeks:
# Queries about incidents from legacy system: correct retrieval
# Queries about incidents from new system (recent): 0 matches returned
# LLM response: "I don't have enough information about recent incidents"
# describe_index_stats(): 50,000 vectors total — looks correct
# No HTTP errors on upsert or query
# No Pinecone warnings about metadata type inconsistency

How Pinecone metadata filtering works

Pinecone stores metadata exactly as provided at upsert time — it does not enforce a schema, does not coerce types, and does not warn when the same field name is stored with different Python types across different vectors. The $eq filter operator performs strict type-aware comparison: {"priority": {"$eq": "high"}} matches only vectors where the priority metadata value is the string "high". It does not match vectors where priority is the integer 0, even if 0 semantically means "high" in the source system's data model.

This silent behaviour means that a RAG system built on top of two heterogeneous data sources can have a subset of its vector population become permanently invisible to all filtered queries — with no errors, no reduced similarity scores, and no observable signal beyond the LLM receiving less context than expected. The describe_index_stats() method reports the correct total vector count (50,000), giving no indication that 10,000 of those vectors are unreachable under the query patterns in use.

# Diagnosing the metadata type mismatch
# Use index.fetch() to inspect metadata types on sample vectors from each batch

from pinecone import Pinecone
import os

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
index = pc.Index("support-tickets")

# Step 1: fetch a sample from Batch 1 (legacy CRM)
legacy_sample = index.fetch(ids=["legacy-1001", "legacy-1002", "legacy-1003"])
for vid, vec in legacy_sample["vectors"].items():
    priority_val = vec["metadata"]["priority"]
    print(f"{vid}: priority = {priority_val!r} (type: {type(priority_val).__name__})")
# legacy-1001: priority = 'high'   (type: str)
# legacy-1002: priority = 'medium' (type: str)
# legacy-1003: priority = 'high'   (type: str)

# Step 2: fetch a sample from Batch 2 (new system)
newsys_sample = index.fetch(ids=["newsys-0001", "newsys-0002", "newsys-0003"])
for vid, vec in newsys_sample["vectors"].items():
    priority_val = vec["metadata"]["priority"]
    print(f"{vid}: priority = {priority_val!r} (type: {type(priority_val).__name__})")
# newsys-0001: priority = 0 (type: int)
# newsys-0002: priority = 1 (type: int)
# newsys-0003: priority = 0 (type: int)
# ↑ Type mismatch confirmed: str in Batch 1, int in Batch 2

# Step 3: confirm $eq type-strictness with explicit tests
emb = embed("high priority incident")  # any query embedding

# Query with string filter (matches Batch 1 only):
res_str = index.query(vector=emb, top_k=5, filter={"priority": {"$eq": "high"}})
print(f"String filter matches: {len(res_str['matches'])}")
# String filter matches: 5  (all from legacy batch — correct legacy results)

# Query with integer filter (matches Batch 2 only):
res_int = index.query(vector=emb, top_k=5, filter={"priority": {"$eq": 0}})
print(f"Integer filter matches: {len(res_int['matches'])}")
# Integer filter matches: 5  (all from new-system batch — the 'invisible' tickets)

# Unfiltered query for comparison:
res_none = index.query(vector=emb, top_k=10)
print(f"Unfiltered matches: {len(res_none['matches'])}")
# Unfiltered matches: 10  (mixes both batches — confirms new-system vectors exist)

# Conclusion: new-system vectors are present and semantically relevant
# but unreachable when the $eq filter uses string "high"

The fix: normalize metadata types before upsert

# ingest_tickets.py — FIXED
# Westfield Knowledge Retrieval Ltd.
# pinecone-client 3.2.2, Python 3.11

from pinecone import Pinecone
from openai import OpenAI
import os

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
index = pc.Index("support-tickets")
oai = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

# Metadata schema contract — all filterable fields and their required Python types
METADATA_SCHEMA = {
    "source": str,
    "priority": str,   # must be one of: "high", "medium", "low"
}

# New-system integer priority codes to canonical string labels
PRIORITY_INT_TO_STR = {0: "high", 1: "medium", 2: "low"}

def normalize_metadata(raw_metadata: dict, source_system: str) -> dict:
    """Normalize metadata to match the METADATA_SCHEMA type contract."""
    metadata = dict(raw_metadata)
    metadata["source"] = source_system

    # Normalize priority: convert int codes to canonical string labels
    priority = metadata.get("priority")
    if isinstance(priority, int):
        if priority not in PRIORITY_INT_TO_STR:
            raise ValueError(f"Unknown integer priority code: {priority}")
        metadata["priority"] = PRIORITY_INT_TO_STR[priority]
    elif isinstance(priority, str):
        if priority not in ("high", "medium", "low"):
            raise ValueError(f"Unknown string priority value: {priority!r}")
        # already correct type and value — no change
    else:
        raise ValueError(f"priority must be int or str, got {type(priority).__name__}")

    return metadata

def validate_metadata(metadata: dict) -> None:
    """Assert all fields match the METADATA_SCHEMA type contract before upsert."""
    for field, expected_type in METADATA_SCHEMA.items():
        value = metadata.get(field)
        if value is None:
            raise ValueError(f"Required metadata field missing: {field!r}")
        if not isinstance(value, expected_type):
            raise TypeError(
                f"Metadata field {field!r}: expected {expected_type.__name__}, "
                f"got {type(value).__name__} ({value!r})"
            )

def embed(text: str) -> list[float]:
    resp = oai.embeddings.create(model="text-embedding-3-small", input=text)
    return resp.data[0].embedding

# ---- BATCH 1: Legacy CRM tickets (40,000 chunks) ----
for ticket in legacy_crm_tickets:
    raw_meta = {"priority": ticket["priority"]}
    metadata = normalize_metadata(raw_meta, source_system="legacy_crm")
    validate_metadata(metadata)  # assert type contract before upsert
    vec = embed(ticket["body"])
    index.upsert(vectors=[{"id": f"legacy-{ticket['id']}", "values": vec, "metadata": metadata}])

# ---- BATCH 2: New-system tickets — RE-INGEST with corrected metadata ----
for ticket in new_system_tickets:
    raw_meta = {"priority": ticket["priority_code"]}
    # normalize_metadata converts int 0 -> str "high", 1 -> "medium", 2 -> "low"
    metadata = normalize_metadata(raw_meta, source_system="new_system")
    validate_metadata(metadata)  # assert type contract before upsert
    vec = embed(ticket["body"])
    index.upsert(vectors=[{"id": f"newsys-{ticket['id']}", "values": vec, "metadata": metadata}])
    # ↑ Re-upsert overwrites the existing vector's metadata in-place.
    # No re-embedding required if ticket body has not changed.

# After fix:
# Both batches: metadata["priority"] is str "high"/"medium"/"low"
# {"priority": {"$eq": "high"}} now matches all 40,000 + 10,000 high-priority vectors
# 10,000 new-system tickets are now visible to all priority-filtered queries

Why metadata type mismatches are easy to get wrong

Pinecone's permissive metadata storage is a deliberate design choice — it allows flexible schema evolution without requiring a migration for every new field. But this permissiveness removes the last guardrail against type inconsistencies that would be caught immediately by a typed database schema. The upsert() method returns HTTP 200 for every vector regardless of metadata type, which confirms the data reached Pinecone but says nothing about whether it is queryable under the application's filter patterns.

The Westfield team's five weeks of investigation focused on the retrieval side — embedding quality, chunk size, similarity thresholds, index configuration — because the symptoms (LLM answering "I don't know") are indistinguishable from poor retrieval quality versus silently excluded vectors. The team never inspected the ingestion side for type differences because both ingestion jobs ran without errors and describe_index_stats() reported the expected total vector count. The missing step was auditing metadata field types using index.fetch() on representative samples from each ingestion batch and comparing them side by side.

Validation

# Retrieval validation (October 31 2026)
# Labeled evaluation set: 200 queries with known relevant ticket IDs
# Metric: recall@10 (fraction of relevant tickets appearing in top-10 results)

# Before fix (10,000 new-system tickets have priority=int, queried as priority=str):
# Recall@10 overall:              78.0%
# Recall@10 for legacy tickets:   97.5%  (all visible under string filter)
# Recall@10 for new-sys tickets:   0.0%  (all invisible under string filter)
# Queries returning 0 results:    19.5%  (queries where all relevant tickets are new-sys)

# After fix (all 50,000 tickets have priority=str "high"/"medium"/"low"):
# Recall@10 overall:              97.0%  (+19.0pp)
# Recall@10 for legacy tickets:   97.5%  (unchanged)
# Recall@10 for new-sys tickets:  96.0%  (new-system tickets now fully retrievable)
# Queries returning 0 results:     0.0%  (down from 19.5%)

# Metadata type audit after re-ingestion:
# Fetched 100 random vectors: 50 from legacy batch, 50 from new-system batch
# All 100: priority field type == str
# All 100: priority value in {"high", "medium", "low"}
# validate_metadata() pass rate: 100% (0 TypeErrors, 0 ValueErrors)

# Pre-upsert validator integration test:
# Passed {"priority": 0} to validate_metadata():
#   TypeError: Metadata field 'priority': expected str, got int (0) — correctly raised
# Passed {"priority": "high"} to validate_metadata():
#   No exception — correctly passes

# LLM answer quality (manual review of 20 recent-incident queries):
# Before fix: 18/20 answered "I don't have enough information"
# After fix:   1/20 answered "I don't have enough information"
#              (1 remaining case: ticket was below cosine threshold — unrelated to metadata fix)

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →