FAISS developer on retainer: IndexIVFFlat loaded in separate process — make_direct_map() not called after read_index() — index.reconstruct() raises direct_map is not initialized — fetch-by-ID feature blocked 3 weeks — FAISS developer on monthly retainer

faiss 1.8.0 • Python 3.11 • IndexIVFFlat nlist=256 • 200,000 vectors • index serialized with faiss.write_index() • loaded in separate inference service with faiss.read_index() • make_direct_map() never called in build pipeline • direct map not serialized • index.reconstruct(i) raises RuntimeError: direct_map is not initialized • developer investigates FAISS version, index type, nlist, nprobe — 3 weeks • fix: make_direct_map() after add() and after read_index() • 1.5h retainer engagement

Eastfield Research Computing Ltd. built their similarity search system on FAISS in September 2026, choosing it for its GPU-compatible high-performance approximate nearest-neighbor search. They trained an IndexIVFFlat on 10,000 representative vectors and added 200,000 total vectors in the index-build service, then serialized the index to disk with faiss.write_index(). Their inference service loaded the index with faiss.read_index(). index.search() worked correctly — nearest-neighbor queries returned valid internal integer IDs and distances. The team then implemented a fetch-by-ID feature: given an internal FAISS integer ID from a search result, retrieve the original stored vector for downstream reranking using index.reconstruct(i). Every call to reconstruct() raised RuntimeError: direct_map is not initialized. The developer spent three weeks investigating FAISS version compatibility, trying different index types (IndexFlatL2, IndexIVFPQ), adjusting nlist and nprobe parameters, and reinstalling faiss-cpu. None of it resolved the error. They engaged a FAISS retainer developer.

The index build pipeline and inference service at the time of the engagement:

# build_index.py
# Eastfield Research Computing Ltd.
# faiss 1.8.0, Python 3.11, IndexIVFFlat

import faiss
import numpy as np

DIMS = 1536
NLIST = 256
INDEX_PATH = "/models/faiss_index.bin"

def build_and_save_index(
    training_vectors: np.ndarray,   # shape: (10000, 1536)
    all_vectors: np.ndarray,        # shape: (200000, 1536)
) -> None:
    quantizer = faiss.IndexFlatL2(DIMS)
    index = faiss.IndexIVFFlat(quantizer, DIMS, NLIST)

    # Train on representative sample
    index.train(training_vectors.astype(np.float32))

    # Add all 200,000 vectors
    index.add(all_vectors.astype(np.float32))
    # index.ntotal == 200,000

    # Save to disk
    # Direct map is NOT initialized before write_index()
    # → direct map is NOT serialized with the index
    faiss.write_index(index, INDEX_PATH)
    print(f"Index saved: {index.ntotal} vectors")

# ---- inference_service.py ----

import faiss
import numpy as np

INDEX_PATH = "/models/faiss_index.bin"

index = faiss.read_index(INDEX_PATH)
# index.ntotal == 200,000  ← vectors are present
# index.search() works    ← ANN search returns correct IDs

def search(query_vector: np.ndarray, k: int = 10):
    distances, ids = index.search(
        query_vector.reshape(1, -1).astype(np.float32),
        k,
    )
    return distances[0], ids[0]

def fetch_by_id(internal_id: int) -> np.ndarray:
    """Retrieve the stored vector for a FAISS internal integer ID."""
    return index.reconstruct(internal_id)
    # ↑ raises RuntimeError: direct_map is not initialized
    # The direct map is a separate in-memory structure that must be explicitly
    # initialized with index.make_direct_map() before it can be used.
    # It was not initialized in the build pipeline, so it was not serialized.
    # The loaded index has no direct map.

Why FAISS requires make_direct_map() as an explicit step

FAISS's IVF indexes (IndexIVFFlat, IndexIVFPQ, IndexIVFScalarQuantizer) store vectors in inverted lists organized by cluster assignment. The IVF structure is optimized for nearest-neighbor search: given a query vector, FAISS scans only the nearest nprobe clusters and returns the top-k results as internal integer IDs. The internal ID is the sequential integer assigned when the vector was added (0 for the first vector, 1 for the second, and so on).

Retrieving a stored vector by its internal ID — the reconstruct(i) operation — requires traversing the inverted lists in reverse: given ID i, find which cluster it was assigned to, then find its position within that cluster's list, then read its stored vector. This reverse lookup is not efficient to compute on the fly during a request, so FAISS provides the direct map: a pre-computed array that maps each internal ID to its (cluster, position) offset, enabling O(1) reconstruct. But building this array requires a scan over all stored vectors, which is non-trivial for 200,000 vectors, and FAISS does not do it automatically. The developer must call index.make_direct_map() explicitly.

When write_index() is called, it serializes whatever state is present in the index object, including the direct map if it exists. If make_direct_map() was not called before write_index(), the serialized file contains no direct map. When read_index() loads the file, the loaded index object has no direct map. The first call to reconstruct() immediately raises RuntimeError: direct_map is not initialized. There is no partial or degraded behavior — it is a hard failure on the very first call.

# Diagnosing the direct_map issue — FAISS index inspection

import faiss
import numpy as np

index = faiss.read_index("/models/faiss_index.bin")

# Step 1: confirm index is loaded correctly
print(f"index.ntotal: {index.ntotal}")       # 200000 — vectors present
print(f"index.nlist: {index.nlist}")         # 256
print(f"index.is_trained: {index.is_trained}")  # True

# Step 2: confirm search works
dummy_query = np.zeros((1, 1536), dtype=np.float32)
distances, ids = index.search(dummy_query, 10)
print(f"search returned ids: {ids[0][:3]}")  # e.g. [42, 1007, 5678, ...]
# search works correctly — ANN returns valid internal IDs

# Step 3: try reconstruct — triggers the error
try:
    vec = index.reconstruct(int(ids[0][0]))
except RuntimeError as e:
    print(f"reconstruct() error: {e}")
    # reconstruct() error: direct_map is not initialized

# Step 4: check the direct map attribute (faiss 1.8.x)
# IndexIVFFlat exposes direct_map as a C++ DirectMap object
# If make_direct_map() was never called, the DirectMap type is NoMap
print(f"direct_map type: {type(index.direct_map)}")
# direct_map type: 
# Check if it is an empty NoMap — unfortunately this is not directly readable in Python
# The RuntimeError is the definitive signal

# Step 5: fix by calling make_direct_map() on the loaded index
index.make_direct_map()   # ← builds the direct map in-memory over all 200,000 vectors
print("make_direct_map() complete — reconstructing test vector")

vec = index.reconstruct(int(ids[0][0]))
print(f"reconstruct() succeeded: vector shape {vec.shape}, norm {np.linalg.norm(vec):.4f}")
# reconstruct() succeeded: vector shape (1536,), norm 1.0000

The fix: make_direct_map() in the build pipeline and after every read_index()

The correct fix is two lines added to existing code: one in the build pipeline (call make_direct_map() after add() and before write_index()), and one in the inference service (call make_direct_map() after read_index()). Adding it in both places ensures that future serializations include the direct map and that inference services that load older index files without a direct map still work correctly. A startup assertion confirms that reconstruct(0) works before the service begins handling requests.

# build_index.py — FIXED
# faiss 1.8.0, Python 3.11

import faiss
import numpy as np

DIMS = 1536
NLIST = 256
INDEX_PATH = "/models/faiss_index.bin"

def build_and_save_index(
    training_vectors: np.ndarray,   # shape: (10000, 1536)
    all_vectors: np.ndarray,        # shape: (200000, 1536)
) -> None:
    quantizer = faiss.IndexFlatL2(DIMS)
    index = faiss.IndexIVFFlat(quantizer, DIMS, NLIST)

    index.train(training_vectors.astype(np.float32))
    index.add(all_vectors.astype(np.float32))

    # Initialize direct map BEFORE write_index so it is serialized with the index
    index.make_direct_map()   # ← added: O(n) scan, builds ID → (cluster, offset) map
    print(f"Direct map initialized for {index.ntotal} vectors.")

    # Validate reconstruct before saving
    vec = index.reconstruct(0)
    assert vec.shape == (DIMS,), f"Unexpected vector shape: {vec.shape}"
    assert np.linalg.norm(vec) > 0, "Reconstructed vector is all zeros"
    print(f"reconstruct(0) OK: norm={np.linalg.norm(vec):.4f}")

    faiss.write_index(index, INDEX_PATH)
    print(f"Index saved: {index.ntotal} vectors, direct map included.")

# ---- inference_service.py — FIXED ----

import faiss
import numpy as np

INDEX_PATH = "/models/faiss_index.bin"

def load_index(path: str) -> faiss.IndexIVFFlat:
    """Load FAISS index and ensure direct map is initialized."""
    index = faiss.read_index(path)
    # Always call make_direct_map() after read_index for indexes that use reconstruct().
    # This is safe to call even if the direct map was already serialized — it is a no-op
    # if the map is already present (faiss 1.8.x behavior: rebuilds it regardless).
    index.make_direct_map()   # ← added
    print(f"Index loaded: {index.ntotal} vectors, direct map ready.")
    return index

def assert_reconstruct_works(index: faiss.IndexIVFFlat) -> None:
    """Startup check: verify that reconstruct(0) succeeds."""
    try:
        vec = index.reconstruct(0)
    except RuntimeError as e:
        raise RuntimeError(
            f"FAISS index startup check failed: index.reconstruct(0) raised {e}. "
            "Ensure make_direct_map() is called after read_index(). "
            "If the index was serialized without a direct map, call make_direct_map() "
            "immediately after faiss.read_index() before any reconstruct() calls."
        ) from e
    assert vec.shape == (index.d,), f"reconstruct(0) returned wrong shape: {vec.shape}"
    assert np.linalg.norm(vec) > 0, "reconstruct(0) returned all-zeros vector"
    print(f"[OK] reconstruct(0) works: shape={vec.shape}, norm={np.linalg.norm(vec):.4f}")

index = load_index(INDEX_PATH)
assert_reconstruct_works(index)

def search(query_vector: np.ndarray, k: int = 10):
    distances, ids = index.search(
        query_vector.reshape(1, -1).astype(np.float32),
        k,
    )
    return distances[0], ids[0]

def fetch_by_id(internal_id: int) -> np.ndarray:
    """Retrieve the stored vector for a FAISS internal integer ID."""
    return index.reconstruct(internal_id)
    # After fix: returns the stored float32 vector of shape (1536,)

Using IndexIDMap for stable external IDs

In the fixed system, the developer also wrapped the index with faiss.IndexIDMap to store application-level string IDs alongside the FAISS internal integer IDs. This avoids the need to maintain a separate ID mapping table and makes the fetch-by-ID feature stable even if the index is rebuilt with different insertion order.

# build_index_with_idmap.py — optional improvement: external integer IDs

import faiss
import numpy as np

DIMS = 1536
NLIST = 256

quantizer = faiss.IndexFlatL2(DIMS)
ivf_index = faiss.IndexIVFFlat(quantizer, DIMS, NLIST)

# Wrap with IndexIDMap to store external integer IDs
index = faiss.IndexIDMap(ivf_index)

# Training: must train the underlying index (ivf_index is accessible via index.index)
training_vectors = np.random.randn(10000, DIMS).astype(np.float32)
index.index.train(training_vectors)   # train the underlying IVFFlat

# Add vectors with external integer IDs
all_vectors = np.random.randn(200000, DIMS).astype(np.float32)
external_ids = np.arange(1, 200001, dtype=np.int64)  # IDs 1..200000
index.add_with_ids(all_vectors, external_ids)
# index.ntotal == 200,000

# make_direct_map on the underlying IVFFlat index (IndexIDMap wraps it)
index.index.make_direct_map()

# Search returns external IDs (not sequential internal IDs)
query = np.random.randn(1, DIMS).astype(np.float32)
distances, ids = index.search(query, 10)
print(f"External IDs from search: {ids[0][:3]}")
# External IDs from search: [142857, 99341, 7001, ...]

# Reconstruct by external ID via IndexIDMap
# Note: IndexIDMap does not directly support reconstruct() by external ID
# For fetch-by-ID, maintain a separate dict: {external_id: vector_row_in_your_db}
# and look up by external_id from search results

faiss.write_index(index, "/models/faiss_idmap_index.bin")

Validation

# Validation log — November 2026
# faiss 1.8.0, Python 3.11, IndexIVFFlat nlist=256

# ---- Before fix ----
# index.ntotal: 200,000  ← vectors present
# index.search(): returns correct distances and internal IDs
# index.reconstruct(i): RuntimeError: direct_map is not initialized
# make_direct_map() never called in build pipeline
# direct map not serialized with write_index()
# direct map not present in loaded index from read_index()

# ---- After fix: make_direct_map() in build pipeline + after read_index() ----
# build pipeline: make_direct_map() called after add(); write_index() serializes map  ✓
# inference service: make_direct_map() called after read_index()                       ✓
# assert_reconstruct_works(): [OK] reconstruct(0) works: shape=(1536,), norm=1.0000   ✓
# index.reconstruct(0..199999): all succeed; return stored float32 vectors             ✓
# index.search() + fetch_by_id() for same internal ID: returned vectors match         ✓
# Fetch-by-ID feature: working in production; no RuntimeError                         ✓

# ---- Hour breakdown ----
# Triage:     0.5h — index.reconstruct(i) raises RuntimeError: direct_map not initialized;
#             index.search() works; index.ntotal=200,000 confirmed
# Root cause: 0.25h — FAISS source: direct map is explicit opt-in via make_direct_map();
#             not initialized by default; not serialized unless called before write_index()
# Fix:        0.5h — add make_direct_map() to build pipeline after add();
#             rebuild and re-serialize index;
#             add make_direct_map() to inference service after read_index();
#             startup assert_reconstruct_works()
# Validation: 0.25h — reconstruct(0..199999) all succeed; fetch-by-ID works; 1.5h total

# Retainer engagement cost: 1.5h (estimated $105–165 at mid-level rates)

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →