MongoDB developer on retainer: Atlas Vector Search returns 0 results — index on field “embedding” but documents use field “embeddings” — silent field name mismatch — 3 weeks — MongoDB Atlas Vector Search developer on monthly retainer

MongoDB Atlas Vector Search • pymongo 4.6.x • Python 3.11 • M20 cluster • 75,000 documents • vector dimensions: 1536 • similarity: cosine • index definition: path: “embedding” (singular) • documents inserted with key “embeddings” (plural) • Atlas indexes field silently — finds 0 matching fields • index builds successfully with 0 indexed vectors • $vectorSearch returns 0 results • no error raised • 3 weeks • fix: delete index, recreate with path: “embeddings” • 1.5h retainer engagement

Westfield Document Store Ltd. built a semantic document retrieval system in September 2026 using MongoDB Atlas Vector Search. The ingestion pipeline inserted 75,000 documents with OpenAI embeddings. The $vectorSearch aggregation pipeline returned an empty array for every query. The developer verified the numCandidates parameter, the cosine similarity metric, the vector dimensions, and the aggregation pipeline syntax. Every query returned zero results. They engaged a MongoDB retainer developer.

The Atlas Vector Search index definition and the document insertion code at the time of the engagement:

# Atlas Vector Search index definition (created via Atlas UI → Search Indexes → Create Index)
# JSON configuration submitted to Atlas:
{
    "fields": [
        {
            "type": "vector",
            "path": "embedding",          # ← singular: "embedding"
            "numDimensions": 1536,
            "similarity": "cosine"
        }
    ]
}
# Atlas confirms: "Index created successfully. Status: BUILDING → READY"
# The index is ready — it has simply indexed 0 documents because no document
# has a field named "embedding" (singular). Atlas does not raise an error for this.

# vector_store.py
# Westfield Document Store Ltd.
# pymongo 4.6.x, Python 3.11, MongoDB Atlas Vector Search

from pymongo import MongoClient

ATLAS_URI = "mongodb+srv://user:pass@cluster0.mongodb.net/?retryWrites=true&w=majority"
DB_NAME = "westfield"
COLLECTION_NAME = "documents"

def get_collection():
    client = MongoClient(ATLAS_URI)
    return client[DB_NAME][COLLECTION_NAME]

def insert_documents(documents: list[dict]) -> int:
    collection = get_collection()
    records = [
        {
            "title": doc["title"],
            "content": doc["content"],
            "embeddings": doc["embedding"],   # ← plural: "embeddings" — BUG
            # Atlas Vector Search index was created on field "embedding" (singular)
            # Documents are inserted with field "embeddings" (plural)
            # Atlas silently ignores documents that don't have the indexed field
            # The index builds to READY status with 0 indexed vectors
        }
        for doc in documents
    ]
    result = collection.insert_many(records)
    return len(result.inserted_ids)   # returns 75,000 — documents were inserted
    # But no documents were indexed by Atlas Vector Search

def search_documents(query_embedding: list[float], top_k: int = 10) -> list[dict]:
    collection = get_collection()
    pipeline = [
        {
            "$vectorSearch": {
                "index": "vector_index",
                "path": "embedding",         # ← singular: matches index definition
                "queryVector": query_embedding,
                "numCandidates": 150,
                "limit": top_k,
            }
        },
        {
            "$project": {
                "title": 1,
                "content": 1,
                "score": {"$meta": "vectorSearchScore"},
            }
        }
    ]
    return list(collection.aggregate(pipeline))
    # Returns []   ← 0 results: index has 0 indexed vectors because
    # documents use "embeddings" (plural) but index scans "embedding" (singular)

Why Atlas Vector Search silently returns 0 results for field mismatches

MongoDB Atlas Vector Search builds an HNSW (Hierarchical Navigable Small World) graph index over the vectors stored in a specific document field. The index definition specifies the path — the dot-notation field name in each document to index. When Atlas builds the index, it scans every document in the collection and reads the value at the specified path. If a document does not have a field at that path, Atlas skips it silently — no error, no warning in the logs, no indication in the Atlas UI that documents were skipped.

The Atlas UI reports the index as READY even when 0 documents were successfully indexed. The $vectorSearch aggregation stage treats a query against a 0-document index as a valid operation that matches nothing — it returns an empty array with no error. The developer has no signal from the pipeline that something is wrong; the query is syntactically correct and MongoDB processes it without complaint.

This bug is especially hard to diagnose because the singular/plural distinction is visually subtle. In the Atlas UI, the index definition appears on one screen and the document insertion code is on another. The listSearchIndexes aggregation stage, which returns the full index configuration, is the fastest way to verify the field path — but most developers do not know it exists.

# Diagnosing the field name mismatch
# pymongo 4.6.x, Python 3.11

from pymongo import MongoClient

client = MongoClient(ATLAS_URI)
collection = client["westfield"]["documents"]

# Step 1: Retrieve the current Atlas Vector Search index configuration
indexes = list(collection.aggregate([{"$listSearchIndexes": {}}]))
for idx in indexes:
    print(f"Index name: {idx['name']}")
    print(f"Status: {idx['status']}")
    print(f"Queryable: {idx['queryable']}")
    # The 'latestDefinition' field shows the exact field path being indexed
    for field in idx.get("latestDefinition", {}).get("fields", []):
        print(f"Indexed field path: {field.get('path')}")
# Output:
# Index name: vector_index
# Status: READY
# Queryable: True
# Indexed field path: embedding     ← singular — the index definition

# Step 2: Check what field name a sample document actually uses
sample_doc = collection.find_one({})
print(f"Document keys: {list(sample_doc.keys())}")
# Output: Document keys: ['_id', 'title', 'content', 'embeddings']
#                                                              ^^^ plural — mismatch

# The mismatch is now visible:
# Index scans: "embedding" (singular)
# Documents have: "embeddings" (plural)
# Atlas silently skips all 75,000 documents during index build
# $vectorSearch returns 0 results: the HNSW graph has no nodes

# Step 3: Verify that the index has 0 indexed documents
# Atlas does not expose a direct numIndexedDocs counter in listSearchIndexes,
# but you can verify by checking the numCandidates output in explain():
explain_result = collection.command(
    "aggregate",
    "documents",
    pipeline=[
        {
            "$vectorSearch": {
                "index": "vector_index",
                "path": "embedding",
                "queryVector": [0.0] * 1536,
                "numCandidates": 150,
                "limit": 10,
            }
        }
    ],
    explain=True,
)
# The winningPlan will show the vectorSearch stage used the index,
# but the nReturned will be 0 with no filter applied — confirming 0 indexed vectors

The fix: delete the index and recreate it with the correct field path

# Option A: Fix the index definition to match the documents (recommended)
# Delete the existing index in the Atlas UI:
#   Atlas UI → Database Deployments → Cluster → Search → vector_index → Delete

# Recreate the index with the correct field path "embeddings" (plural):
# Atlas UI → Search → Create Search Index → JSON Editor:
{
    "fields": [
        {
            "type": "vector",
            "path": "embeddings",       # ← plural: matches document key name
            "numDimensions": 1536,
            "similarity": "cosine"
        }
    ]
}

# Wait for status READY (~5-15 minutes for 75,000 documents)
# Then update the $vectorSearch pipeline to use "embeddings" in the path field:

def search_documents_fixed(query_embedding: list[float], top_k: int = 10) -> list[dict]:
    collection = get_collection()
    pipeline = [
        {
            "$vectorSearch": {
                "index": "vector_index",
                "path": "embeddings",    # ← updated to match documents and index
                "queryVector": query_embedding,
                "numCandidates": max(10 * top_k, 100),   # best practice: >= 10x limit
                "limit": top_k,
            }
        },
        {
            "$project": {
                "title": 1,
                "content": 1,
                "score": {"$meta": "vectorSearchScore"},
            }
        }
    ]
    return list(collection.aggregate(pipeline))

# Option B: Rename the field in all existing documents to match the index
# Use update_many() with $rename — no need to recreate the index

collection.update_many(
    {"embeddings": {"$exists": True}},    # find docs with the "embeddings" field
    {"$rename": {"embeddings": "embedding"}}   # rename to "embedding" (singular)
)
# After this runs, all documents have "embedding" (singular)
# The existing index on "embedding" (singular) now indexes all 75,000 documents
# Wait for the index to finish indexing the updated documents

# Poll until the index has indexed all documents:
import time

def wait_for_index_ready(collection, index_name: str, expected_docs: int, timeout: int = 900):
    start = time.time()
    while time.time() - start < timeout:
        indexes = list(collection.aggregate([{"$listSearchIndexes": {"name": index_name}}]))
        if indexes and indexes[0].get("queryable"):
            # Index is queryable — verify a sample query returns results
            sample = list(collection.find_one({}))
            if sample:
                results = list(collection.aggregate([
                    {
                        "$vectorSearch": {
                            "index": index_name,
                            "path": "embedding",
                            "queryVector": collection.find_one()["embedding"],
                            "numCandidates": 10,
                            "limit": 1,
                        }
                    }
                ]))
                if results:
                    print(f"[OK] Index {index_name} is queryable and returning results")
                    return
        time.sleep(15)
    raise TimeoutError(f"Index {index_name} did not become queryable within {timeout}s")

# Field name assertion — add to the ingestion pipeline startup check
# Prevents the field name mismatch from reaching production

def assert_index_field_matches_documents(
    collection,
    index_name: str,
    document_field_name: str,
) -> None:
    """
    Verifies that the Atlas Vector Search index field path matches the field name
    used in document insertions. Raises ValueError if there is a mismatch.
    """
    indexes = list(collection.aggregate([{"$listSearchIndexes": {"name": index_name}}]))
    if not indexes:
        raise ValueError(f"Atlas Vector Search index '{index_name}' not found. Create it first.")

    index_def = indexes[0]
    if index_def.get("status") not in ("READY", "STALE"):
        raise ValueError(
            f"Atlas Vector Search index '{index_name}' is not ready. "
            f"Status: {index_def.get('status')}. Wait for READY before ingesting."
        )

    indexed_paths = [
        field.get("path")
        for field in index_def.get("latestDefinition", {}).get("fields", [])
        if field.get("type") == "vector"
    ]

    if document_field_name not in indexed_paths:
        raise ValueError(
            f"Field name mismatch: documents use field '{document_field_name}' "
            f"but index '{index_name}' indexes field(s): {indexed_paths}. "
            f"Either rename the field in the index definition or rename it in document insertions."
        )

    print(f"[OK] Index '{index_name}' field path '{document_field_name}' matches document field.")

# Call at application startup before any ingestion
assert_index_field_matches_documents(collection, "vector_index", "embeddings")

Validation

# Validation log — October 31 2026
# pymongo 4.6.x, Python 3.11, MongoDB Atlas Vector Search M20 cluster

# ---- Before fix ----
# listSearchIndexes: index field path = "embedding" (singular)
# Sample document keys: ['_id', 'title', 'content', 'embeddings']   ← plural
# Index status: READY (but 0 vectors indexed — all documents skipped silently)
# $vectorSearch with numCandidates=150, limit=10: returned []   ← 0 results
# Developer checked numCandidates, similarity metric, dimensions — all correct
# Root cause: index path "embedding" ≠ document field "embeddings"

# ---- After fix (Option A: recreate index with path="embeddings") ----
# assert_index_field_matches_documents(collection, "vector_index", "embeddings"):
#   [OK] Index 'vector_index' field path 'embeddings' matches document field.
# Index rebuild: 75,000 documents indexed in 8 minutes
# $vectorSearch with path="embeddings", numCandidates=150, limit=10:
#   returned 10 results with vectorSearchScore > 0.85   ← correct results
# Top result: document most semantically similar to query vector   ← verified
# Additional queries with varied vectors: all returned relevant results

# ---- Hour breakdown ----
# Triage: 0.5h — run listSearchIndexes; check field path; compare to document keys
# Root cause: 0.25h — confirm 0 indexed documents via explain(); verify singular/plural mismatch
# Fix: 0.5h — delete index; recreate with path="embeddings"; update $vectorSearch path; add assertion
# Validation: 0.25h — index READY; $vectorSearch returns 10 results; smoke test passes
# Total: 1.5h

# Retainer engagement cost: 1.5h (estimated $105-165 at mid-level rates)

Related retainer posts

• Supabase developer on retainer — RLS enabled on documents table with no policy — every INSERT silently ignored — every SELECT returns 0 rows — fix: CREATE POLICY
• Milvus developer on retainer — collection.search() raises MilvusException: collection not loaded after create_index() — collection.load() missing
• Redis vector search developer on retainer — HNSW index on HASH type but vectors inserted as JSON — index silently ignores JSON fields — 0 results — fix: re-ingest via hset()