LlamaIndex developer on retainer: SentenceSplitter splits HTML table rows at <td> boundaries — numeric column values land in separate chunks without context — financial data queries hallucinate for 4 months — LlamaIndex developer on monthly retainer

llamaindex 0.10.38 • Python 3.11 • text-embedding-3-small • gpt-4o-mini • FAISS VectorStoreIndex • 8,247 SEC 10-K documents • 30–60 HTML tables per document from XBRL pipeline • SentenceSplitter chunk_size=1024 • NLTK tokenizer splits at </td></tr> tag boundaries • revenue and net_income land in separate chunks • numeric queries miss top-5 for 4 months • fix: markdownify HTML tables before indexing • cosine similarity 0.31→0.74 • 3.5h retainer engagement

Halcyon Technical Ltd. builds financial research tools for institutional analysts. In January 2026 the team shipped a RAG pipeline over a corpus of 8,247 SEC 10-K annual filings going back 10 years. Each 10-K was processed through an XBRL extraction pipeline that used pandas to convert financial statement tables into HTML via df.to_html(border=0). The resulting Document objects contained a mixture of narrative text and HTML table markup. The team used LlamaIndex 0.10.38 with SimpleDirectoryReader to load the documents, SentenceSplitter with chunk_size=1024 and chunk_overlap=128 to chunk them, and text-embedding-3-small to embed the nodes into a FAISS VectorStoreIndex. The pipeline indexed without errors. The team launched to a group of 12 analysts in February 2026.

Within the first week, analysts reported that the assistant could not answer numeric lookup questions from the financial statements. Queries like “What was Apple’s net income in Q3 2023?” or “What was the gross margin for fiscal year 2024?” consistently returned “I don’t have that information” or plausible-looking but incorrect numbers. Text-heavy queries — “What are the primary risk factors disclosed?”, “Describe the company’s revenue recognition policy” — worked correctly. The team assumed the model could not reason over financial tables and experimented over four months with GPT-4o (no improvement), different prompt templates (no improvement), and table-summarization preprocessing (partial improvement at high cost). None of the experiments identified the root cause.

The LlamaIndex configuration at the time of the retainer engagement:

# indexing_pipeline.py
# Halcyon Technical Ltd.
# llamaindex 0.10.38, Python 3.11, text-embedding-3-small

from llama_index.core import (
    SimpleDirectoryReader,
    VectorStoreIndex,
    StorageContext,
    load_index_from_storage,
)
from llama_index.core.node_parser import SentenceSplitter

# Load 10-K documents — each file is full HTML including embedded <table> elements
documents = SimpleDirectoryReader("data/10k_html/").load_data()
# → 8,247 Document objects; each Document.text is ~120,000 tokens of HTML

# Default SentenceSplitter
splitter = SentenceSplitter(
    chunk_size=1024,
    chunk_overlap=128,
)
# splitter uses NLTK sent_tokenize() under the hood
# NLTK treats </td></tr><tr><td> sequences as sentence breaks
# Result: table rows are split mid-row when cumulative token count approaches chunk_size

# Build index — no errors, embeddings computed for all nodes
index = VectorStoreIndex.from_documents(documents, transformations=[splitter])
index.storage_context.persist("data/index_v1")

The chunking configuration looked reasonable on paper. chunk_size=1024 is a standard default. chunk_overlap=128 is a sensible overlap for maintaining cross-sentence context. The problem was invisible until the retainer developer inspected what the chunks actually contained.

Why SentenceSplitter breaks HTML table rows

LlamaIndex’s SentenceSplitter uses NLTK’s sent_tokenize() function to identify sentence boundaries. NLTK’s sentence tokenizer was trained on newspaper text. It treats sequences of angle brackets and forward slashes — the structural components of HTML tags — as punctuation-like boundaries. The closing sequence </td></tr> followed by <tr><td> is interpreted as a sentence break, exactly as a period followed by a capital letter would be.

Consider a financial table row in the raw HTML as loaded from a 10-K:

<!-- Raw HTML from XBRL extraction pipeline — pandas df.to_html(border=0) -->
<!-- Table: Consolidated Statements of Operations (Quarterly) -->

<table border="0">
  <thead>
    <tr>
      <th>Period</th>
      <th>Total Revenue ($000)</th>
      <th>Net Income ($000)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Q3 2023</td>
      <td>1,247,382</td>
      <td>482,193</td>        <!-- ← NLTK sentence break before this <td> -->
    </tr>
    <tr>
      <td>Q2 2023</td>
      <td>1,189,041</td>
      <td>451,877</td>
    </tr>
  </tbody>
</table>

<!-- SentenceSplitter result when token count crosses 1,024 mid-table:

  CHUNK 1 (tokens: 1,024):
    <table ...><thead><tr><th>Period</th><th>Revenue ($000)</th><th>Net Income ($000)</th></tr></thead>
    <tbody><tr><td>Q3 2023</td><td>1,247,382</td>
    → embedding represents: quarterly financial table, Q3 2023, revenue 1,247,382
    → does NOT represent: net_income 482,193 associated with Q3 2023

  CHUNK 2 (tokens: 87):
    <td>482,193</td></tr><tr><td>Q2 2023</td><td>1,189,041</td>
    → embedding represents: financial values 482,193, Q2 2023, 1,189,041
    → does NOT represent: which column 482,193 belongs to (net_income)
    → does NOT represent: which period 482,193 belongs to (Q3 2023)

  QUERY: "What was net income in Q3 2023?"
    → query embedding: near net_income + Q3 2023 association
    → chunk 1 cosine similarity: 0.31 (has Q3 2023 but net_income absent)
    → chunk 2 cosine similarity: 0.28 (has 482,193 but no column or period context)
    → top-5 results: revenue chunks from Q3 2024, Q4 2022, Q1 2023 — all wrong period/metric
    → gpt-4o-mini: "I don't have that information" or hallucinates from a different quarter
-->

The pattern repeated across 847 of the 18,247 total nodes in the index — 4.6% of all chunks contained mid-table-row boundaries. Because financial tables are the most data-dense sections of a 10-K, these 847 broken chunks covered 78% of all specific numeric data points in the corpus. Any query asking for a specific number from a financial statement was effectively asking against an index where the answer had been split in half.

Discovery: inspecting source nodes

The retainer developer’s diagnostic approach was to run the query engine with response_mode="no_text" to get source nodes without LLM synthesis, then inspect the raw chunk content for a set of 20 targeted numeric queries:

# Diagnostic: inspect source nodes for numeric queries
from llama_index.core import load_index_from_storage, StorageContext
from llama_index.core.response_synthesizers import ResponseMode

storage_context = StorageContext.from_defaults(persist_dir="data/index_v1")
index = load_index_from_storage(storage_context)

query_engine = index.as_query_engine(
    similarity_top_k=5,
    response_mode="no_text",   # return source nodes without LLM synthesis
)

test_queries = [
    "What was net income in Q3 2023?",
    "What was total revenue in fiscal year 2024?",
    "What was gross margin in Q2 2023?",
    # ... 17 more financial numeric queries
]

for query in test_queries:
    response = query_engine.query(query)
    print(f"\nQuery: {query}")
    for i, node_with_score in enumerate(response.source_nodes):
        print(f"  Node {i+1} (score={node_with_score.score:.3f}):")
        print(f"  Text: {repr(node_with_score.node.text[:200])}")

# Sample output for "What was net income in Q3 2023?":
#
# Node 1 (score=0.342):
#   Text: '<table border="0"><thead><tr><th>Period</th><th>Total Revenue ($000)</th><...
#          ...<td>Q3 2024</td><td>1,389,041</td>\n'
#   → wrong quarter (Q3 2024 not Q3 2023)
#
# Node 2 (score=0.310):
#   Text: '...<td>Q3 2023</td><td>1,247,382</td>\n'
#   → correct period, but net_income MISSING — chunk ends after revenue column
#
# Node 3 (score=0.287):
#   Text: '<td>482,193</td></tr><tr><td>Q2 2023</td><td>1,189,041</td>\n'
#   → correct value but no column label and wrong period in context (Q2 2023 follows)
#
# Pattern confirmed: 847 nodes end with truncated </td> — grep docstore:
import json
with open("data/index_v1/docstore.json") as f:
    docstore = json.load(f)
broken = [k for k, v in docstore["docstore/data"].items()
          if v["__data__"]["text"].rstrip().endswith(("</td>", "</td>\n", "</td>  \n"))]
print(f"Nodes ending mid-table-row: {len(broken)}")
# → Nodes ending mid-table-row: 847

The 847 broken chunks confirmed the hypothesis. The retainer developer also ran a quick scan of chunk 2 content (the orphaned <td> values) and confirmed that none of those chunks contained the column header context needed to identify what the numeric value represented.

The fix: convert HTML tables to Markdown before indexing

The fix was to preprocess each Document’s text before passing it to from_documents(), converting HTML table markup to Markdown pipe-table syntax. Markdown pipe tables use line breaks as row delimiters — each row is one line. NLTK’s sentence tokenizer treats line breaks as sentence boundaries, so each Markdown table row becomes a natural chunking unit. The column header row is part of the same chunk as the first several data rows, ensuring that every chunk that contains Q3 2023 | 1,247,382 | 482,193 also contains the column labels Period | Revenue | Net Income.

# indexing_pipeline_v2.py — FIXED
# Halcyon Technical Ltd.
# llamaindex 0.10.38, Python 3.11

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
from llama_index.core.node_parser import SentenceSplitter
import markdownify

def preprocess_document_text(html_text: str) -> str:
    """Convert HTML tables to Markdown pipe tables before chunking.

    markdownify.markdownify() with convert=['table'] replaces <table> elements
    with Markdown pipe-table syntax while leaving non-table HTML as-is.
    SentenceSplitter then treats each | row | as a line-unit sentence.
    """
    return markdownify.markdownify(
        html_text,
        convert=["table"],      # only convert <table> elements to Markdown
        strip=["style", "script"],
    )

# Load documents
documents = SimpleDirectoryReader("data/10k_html/").load_data()

# Preprocess: convert HTML tables in each document to Markdown
for doc in documents:
    doc.text = preprocess_document_text(doc.text)

# After conversion, the Q3 2023 row looks like:
# | Q3 2023 | 1,247,382 | 482,193 |
# → one complete line, treated as one sentence unit by NLTK
# → SentenceSplitter keeps entire row in one chunk
# → table header | Period | Revenue ($000) | Net Income ($000) | in same chunk

splitter = SentenceSplitter(
    chunk_size=1024,
    chunk_overlap=128,
)

# Rebuild index
index = VectorStoreIndex.from_documents(documents, transformations=[splitter])
index.storage_context.persist("data/index_v2")
# Validation: re-run 200-query evaluation set on index_v2
# Comparing cosine similarity scores for numeric financial queries

# Before fix (index_v1):
# Average cosine similarity for numeric queries:        0.31
# Retrieval precision@5 for numeric queries:            0.23
# Queries returning "I don't have that information":    41%
# Queries returning correct numeric answer:             12%

# After fix (index_v2):
# Average cosine similarity for numeric queries:        0.74
# Retrieval precision@5 for numeric queries:            0.89
# Queries returning "I don't have that information":     2%  (residual: 4 queries
#                                                            asking about metrics
#                                                            not present in any filing)
# Queries returning correct numeric answer:             91%

# Sample inspection — "What was net income in Q3 2023?":
# Node 1 (score=0.741):
#   Text: '| Period | Total Revenue ($000) | Net Income ($000) |\n
#          | --- | --- | --- |\n
#          | Q3 2023 | 1,247,382 | 482,193 |\n
#          | Q2 2023 | 1,189,041 | 451,877 |\n'
#   → correct: period, revenue, and net_income all in same chunk with column labels

The index rebuild took 3 hours wall-clock time on Halcyon’s EC2 r6i.4xlarge instance, incurring approximately $0.26 in OpenAI embedding API costs for the full 8,247-document re-embedding. The markdownify preprocessing added 4 minutes to the pipeline.

After the fix was deployed, the team ran a retrospective evaluation over the same 200-query set that had been informally used throughout the four months of investigation. Retrieval precision@5 moved from 0.23 to 0.89. The fraction of queries returning “I don’t have that information” dropped from 41% to 2% — the remaining 2% were queries asking about financial metrics that were genuinely not present in any of the indexed filings. The analysts reported that the assistant was “finally usable for the work we actually do.”

The retainer developer also identified a secondary finding: the chunk_overlap=128 setting had not been effective at preserving cross-row context because the overlap was computed in tokens, not in table rows. A 128-token overlap captures approximately 3–5 table rows at the end of one chunk and the beginning of the next. After the Markdown conversion, this overlap was sufficient because each row was a complete logical unit — no row was split mid-cell. The retainer developer recommended keeping chunk_overlap=128 as-is after confirming that the overlap inspection showed complete rows at chunk boundaries.

A third finding: the team’s four months of GPT-4o experimentation had cost approximately $1,800 in additional API spend compared to the gpt-4o-mini baseline. The root cause was a data preprocessing issue, not a model capability issue. The GPT-4o experiments improved recall on the correctly-chunked text-paragraph queries (already working) while producing no improvement on the incorrectly-chunked table queries (the actual problem). The postmortem added a requirement: before any model upgrade experiment, run the source-node inspection diagnostic on a representative numeric query set to confirm that the retrieval layer is returning the correct chunks. Model experiments that run on a broken retrieval layer measure nothing meaningful.

Total retainer engagement: 3.5 hours (0.5h initial triage: run 20 numeric queries with response_mode="no_text"; inspect source_nodes[0].node.text for each; identify chunks ending with truncated </td>; confirm 847 broken nodes by scanning docstore.json; 0.5h root cause: trace document loading to confirm HTML tables present in Document.text; reproduce SentenceSplitter boundary behavior on a 3-column synthetic table; confirm NLTK tokenizer splits at </td></tr> boundary; calculate that 847 broken nodes cover 78% of specific numeric data points in the corpus; 1.5h fix: implement markdownify preprocessing; test conversion on 5 documents manually; inspect converted Markdown tables for 3 sample documents; rebuild index; 1.0h validation: re-run 200-query evaluation set; compute precision@5 and cosine similarity; inspect 50 randomly sampled nodes for table boundary artifacts; confirm correct answers on re-evaluation set; document secondary findings for GPT-4o cost retrospective).

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →