Weaviate developer on retainer: hybrid() returns 3–5 results instead of 10 — empty description property disables BM25 scoring for 8% of collection — Accessories category 87% invisible to search for 14 weeks — Weaviate developer on monthly retainer

weaviate-client 4.6.1 • Python 3.11 • Weaviate 1.25.0 Cloud • text2vec-openai vectorizer • Products collection 52,847 objects • 4,219 empty description strings • BM25 score=0.0 for empty-property objects • hybrid alpha=0.5 returns 3–5 instead of 10 • Accessories 87% affected • 14 weeks • fix: backfill [no description available] + ETL validation • 2.5h retainer engagement

Northfield Search Solutions Ltd. built a product search backend for an e-commerce client using Weaviate. The collection contained 52,847 products indexed by a text2vec-openai vectorizer with hybrid search (vector + BM25 keyword). Products were imported from a PostgreSQL catalog database via a nightly ETL pipeline. In late June 2026 the client noticed that searching for accessories — phone cases, cables, adapters — returned almost nothing. A query like “wireless headphones case” returned 3 products when the client expected to see 10. Keyword search on their previous Elasticsearch system had returned 15–20. The team assumed a Weaviate configuration issue or a model quality problem with the vectorizer and spent two weeks tuning the alpha parameter (0.3, 0.7, 0.9) and switching vectorizer models (text2vec-openai ada-002 to text-embedding-3-small). No tuning changed the result count for accessories. In October 2026 they engaged a Weaviate retainer developer.

The collection schema and query configuration at the time of the engagement:

# schema_setup.py
# Northfield Search Solutions Ltd.
# weaviate-client 4.6.1, Python 3.11, Weaviate 1.25.0 Cloud

import weaviate
from weaviate.classes.config import Configure, Property, DataType

client = weaviate.connect_to_weaviate_cloud(
    cluster_url="...",
    auth_credentials=weaviate.auth.AuthApiKey("..."),
)

products = client.collections.create(
    name="Products",
    vectorizer_config=Configure.Vectorizer.text2vec_openai(
        model="text-embedding-3-small",
    ),
    properties=[
        Property(name="name",        data_type=DataType.TEXT),
        Property(name="description", data_type=DataType.TEXT),
        Property(name="category",    data_type=DataType.TEXT),
        Property(name="sku",         data_type=DataType.TEXT,
                 skip_vectorization=True),   # SKU excluded from embedding
        Property(name="price_usd",   data_type=DataType.NUMBER,
                 skip_vectorization=True),   # numeric, not vectorized
    ],
)
# All TEXT properties are BM25-indexed by default in Weaviate 1.25.0
# description is the primary text property — most of the semantic content
# search.py — production hybrid search query
# Northfield Search Solutions Ltd.

def search_products(query: str, limit: int = 10):
    products = client.collections.get("Products")
    results = products.query.hybrid(
        query=query,
        alpha=0.5,        # 50% vector, 50% BM25
        limit=limit,
        return_metadata=MetadataQuery(score=True),
    )
    return results.objects
    # Expected: always returns `limit` objects (10)
    # Actual: returns 3–7 for queries over Accessories category
    # No error — response is HTTP 200 with fewer results than requested

Why empty description disables BM25 for those objects

Weaviate’s BM25 component indexes the text content of every non-skipped TEXT property. For each object, BM25 tokenizes the text value of each indexed property and records the term frequencies. For an object where description is the empty string "", the tokenizer produces an empty token list. An empty token list means the object has zero BM25 term frequency for any possible query term. BM25 score for that object is 0.0 on every query.

With alpha=0.5, Weaviate’s RankedFusion mode combines the vector score rank and the BM25 score rank using reciprocal rank fusion. An object that scores last in BM25 (score=0.0) receives the worst-possible BM25 rank position. Even if its vector similarity is high — it is a semantically relevant product — the combined RRF score is pulled down by the last-place BM25 rank. When the top candidates by vector similarity are predominantly empty-description products, the fusion algorithm cannot rank enough of them above the RRF threshold to fill limit=10 result slots.

The ETL pipeline that caused the empty descriptions:

# etl_pipeline.py — BUGGY VERSION
# Northfield Search Solutions Ltd.

import sqlalchemy
from sqlalchemy import text

engine = sqlalchemy.create_engine("postgresql://...")

with engine.connect() as conn:
    rows = conn.execute(text("""
        SELECT sku, name, description, category, price_usd
        FROM product_catalog
    """)).mappings().all()

# PostgreSQL: description is NULL for 4,219 products (no CMS description yet)
# SQLAlchemy .mappings() returns None for NULL columns
# Python dict comprehension: None values are kept as None
products_for_import = [dict(row) for row in rows]

# BUG: Pydantic model used for validation
from pydantic import BaseModel
from typing import Optional

class ProductRecord(BaseModel):
    sku: str
    name: str
    description: Optional[str] = ""   # ← default="" converts None to ""
    category: str
    price_usd: float

# When description=None from PostgreSQL, Pydantic assigns the default: ""
# validated_products[i].description is "" for all 4,219 NULL-description products

validated_products = [ProductRecord(**p) for p in products_for_import]

# Batch import to Weaviate
with products.batch.fixed_size(batch_size=500) as batch:
    for product in validated_products:
        batch.add_object(properties=product.model_dump())
# Weaviate accepts description="" without validation error
# BM25 tokenizer: tokenize("") → []
# BM25 score for these objects: 0.0 on all queries
# No error logged anywhere in the pipeline

Discovery: comparing hybrid vs near_text result counts

The retainer developer’s diagnostic approach was to run identical queries using both hybrid() and near_text() (pure vector search) and compare result counts:

# Diagnostic step 1: compare hybrid vs near_text result counts

products = client.collections.get("Products")

test_queries = [
    "wireless headphones case",
    "phone charging cable",
    "USB-C adapter",
    "screen protector",
    "laptop sleeve",
    "bluetooth speaker",        # Electronics — different category
    "running shoes",            # Footwear — different category
]

for query in test_queries:
    hybrid_results = products.query.hybrid(query=query, alpha=0.5, limit=10)
    vector_results = products.query.near_text(query=query, limit=10)

    print(f"Query: {query!r}")
    print(f"  hybrid result count:    {len(hybrid_results.objects)}")
    print(f"  near_text result count: {len(vector_results.objects)}")

# Output:
# Query: 'wireless headphones case'
#   hybrid result count:    3
#   near_text result count: 10
# Query: 'phone charging cable'
#   hybrid result count:    4
#   near_text result count: 10
# Query: 'USB-C adapter'
#   hybrid result count:    3
#   near_text result count: 10
# Query: 'bluetooth speaker'       ← Electronics: different category
#   hybrid result count:    10
#   near_text result count: 10
# Query: 'running shoes'            ← Footwear: different category
#   hybrid result count:    10
#   near_text result count: 10
#
# Pattern: hybrid returns fewer than limit ONLY for Accessories category queries
# near_text always returns 10 — pure vector search is unaffected
# BM25 degradation is category-specific: Accessories affected, Electronics/Footwear not
# Diagnostic step 2: count objects with empty description

from weaviate.classes.query import Filter

empty_desc_count = products.aggregate.over_all(
    filters=Filter.by_property("description").equal(""),
    total_count=True,
)
print(f"Objects with empty description: {empty_desc_count.total_count}")
# → Objects with empty description: 4,219

# Count by category
from weaviate.classes.aggregate import GroupByAggregate

by_category = products.aggregate.over_all(
    group_by=GroupByAggregate(prop="category"),
    total_count=True,
)
for group in by_category.groups:
    empty_in_cat = products.aggregate.over_all(
        filters=(
            Filter.by_property("category").equal(group.grouped_by.value) &
            Filter.by_property("description").equal("")
        ),
        total_count=True,
    )
    total_in_cat = group.total_count
    empty_rate = empty_in_cat.total_count / total_in_cat * 100
    print(f"  {group.grouped_by.value}: {empty_in_cat.total_count}/{total_in_cat} = {empty_rate:.1f}% empty")

# Output:
#   Accessories: 3,671 / 4,222 = 86.9% empty description
#   Electronics: 182 / 19,847 =  0.9% empty description
#   Footwear:    251 / 14,103 =  1.8% empty description
#   Apparel:      95 / 14,675 =  0.6% empty description
#
# 86.9% of Accessories products have description="" → BM25 score=0.0
# When hybrid query matches Accessories products, 87% of candidates have BM25=0.0
# RRF fusion cannot fill 10 result slots from candidates with near-zero hybrid scores

The fix: backfill empty descriptions and add ETL validation

The fix had two parts: immediately backfill the 4,219 empty-description objects with a non-empty placeholder, and add validation to the ETL pipeline to prevent future empty strings from reaching Weaviate.

# backfill_empty_descriptions.py — IMMEDIATE FIX
# Northfield Search Solutions Ltd.
# weaviate-client 4.6.1

from weaviate.classes.query import Filter

products = client.collections.get("Products")

# Fetch all objects with empty description
empty_desc_objects = products.query.fetch_objects(
    filters=Filter.by_property("description").equal(""),
    limit=5000,
    return_properties=["name", "category"],
)

print(f"Objects to backfill: {len(empty_desc_objects.objects)}")

# Update each object with placeholder description
# The placeholder "[no description available]" produces a non-zero BM25 token list:
# tokenize("[no description available]") → ["no", "description", "available"]
# BM25 score for these tokens against a typical product query will be low,
# but non-zero — sufficient for RRF fusion to include the object in ranking

updated = 0
for obj in empty_desc_objects.objects:
    products.data.update(
        uuid=obj.uuid,
        properties={"description": "[no description available]"},
    )
    updated += 1
    if updated % 100 == 0:
        print(f"  Updated {updated}/{len(empty_desc_objects.objects)}")

print(f"Backfill complete: {updated} objects updated")
# etl_pipeline.py — FIXED
# Northfield Search Solutions Ltd.

class ProductRecord(BaseModel):
    sku: str
    name: str
    description: Optional[str] = None   # ← None stays None, no empty string default
    category: str
    price_usd: float

    def to_weaviate_properties(self) -> dict:
        """Convert to Weaviate import dict.
        Replaces None/empty description with placeholder to avoid BM25 degradation.
        """
        return {
            "sku": self.sku,
            "name": self.name,
            # Replace None or empty string with placeholder
            # Empty string → BM25 score=0 → hybrid result count < limit
            # Placeholder → BM25 tokens ["no", "description", "available"] → non-zero
            "description": self.description if self.description else "[no description available]",
            "category": self.category,
            "price_usd": self.price_usd,
        }

# Pre-import validation
def validate_batch(products: list[ProductRecord]) -> None:
    """Validate batch before sending to Weaviate.
    Raises ValueError if any product would produce empty BM25 token list.
    """
    empty_desc = [p.sku for p in products if not p.description]
    if empty_desc:
        # This should not happen after the to_weaviate_properties() conversion,
        # but log it for ETL quality monitoring
        import logging
        logging.warning(
            f"ETL: {len(empty_desc)} products had empty description before conversion. "
            f"SKUs: {empty_desc[:10]}{'...' if len(empty_desc) > 10 else ''}"
        )

# Batch import with validation
with products.batch.fixed_size(batch_size=500) as batch:
    for product in validated_products:
        validate_batch([product])
        batch.add_object(properties=product.to_weaviate_properties())

The backfill took 4 minutes to update all 4,219 objects. After completion, the retainer developer re-ran the 20 canary queries across all categories:

# Validation: re-run canary queries post-backfill

# Query: 'wireless headphones case'
#   hybrid result count: 10  ✓  (was 3)
# Query: 'phone charging cable'
#   hybrid result count: 10  ✓  (was 4)
# Query: 'USB-C adapter'
#   hybrid result count: 10  ✓  (was 3)
# Query: 'screen protector'
#   hybrid result count: 10  ✓  (was 5)
# Query: 'bluetooth speaker'
#   hybrid result count: 10  ✓  (unchanged — never affected)
#
# All 20 canary queries return limit=10
#
# Spot check: objects.with_empty_description count = 0 (post-backfill)
# ETL pipeline validation: ran simulated NULL import → warning logged, placeholder substituted

The e-commerce client confirmed that the Accessories search results were restored within hours of the backfill completing. Products that had been invisible for 14 weeks were now appearing in hybrid search results for the first time.

The retainer developer identified two additional findings during the engagement. First, the team had been comparing their Weaviate hybrid search performance to their previous Elasticsearch implementation using result count as the primary metric. Because Elasticsearch always returns size results (filling with lower-ranked matches when needed), they interpreted the Weaviate result shortfall as a Weaviate limitation rather than a data quality issue. The retainer developer recommended adding a daily monitoring query that computes avg(len(results)) for a fixed canary query set and alerts if the average falls below limit * 0.95.

Second, the fusion_type default of RankedFusion made the empty-BM25 problem more severe than it would have been with RelativeScoreFusion. With RelativeScoreFusion, the BM25 component contributes its normalized raw score (which is 0.0 for empty-description objects) rather than its rank position. A raw BM25 score of 0.0 with RelativeScoreFusion and alpha=0.5 means the object is scored entirely on its vector component at 50% weight — it can still rank in the top-10 if its vector score is high enough. With RankedFusion, a last-place BM25 rank (score=0.0) penalizes the object’s combined rank regardless of vector score. The team switched to fusion_type=HybridFusion.RELATIVE_SCORE as a secondary mitigation, which partially restored result counts even for the remaining edge cases where the placeholder substitution had not yet propagated through the nightly ETL cycle.

Total retainer engagement: 2.5 hours (0.5h triage: reproduce hybrid vs near_text result count discrepancy; confirm near_text returns 10, hybrid returns 3–5 for Accessories queries; aggregate query: empty description count = 4,219; 0.5h root cause: trace ETL pipeline; identify Pydantic default=“” converting None to empty string; verify BM25 tokenizer behavior on empty string; confirm Accessories 87% empty-description rate; calculate that 87% empty-BM25 candidates cannot fill 10 result slots under RRF fusion; 1.0h fix: backfill 4,219 objects with placeholder; update ETL Pydantic model and add validation logging; switch fusion_type to RelativeScoreFusion as secondary mitigation; 0.5h validation: re-run 20 canary queries; confirm all return limit=10; spot-check aggregate for zero empty-description objects; document monitoring protocol).

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →