Lunary developer on retainer: LunaryCallbackHandler thread context not inherited by ThreadPoolExecutor workers — 2,000 LangChain traces per nightly batch run created as orphaned events with no parent thread — 4 months of batch monitoring data unaggregated in Lunary dashboard — Lunary developer on monthly retainer

lunary 0.7.3 • Python 3.11 • LangChain 0.2.15 • OpenAI GPT-4o • ThreadPoolExecutor(max_workers=8) • FastAPI 0.111.0 • nightly batch document processing job • LunaryCallbackHandler() in worker threads without thread_id • Python thread-local not inherited by ThreadPoolExecutor workers • 2,000 orphaned traces per nightly run • batch threads show 0 traces • 4 months unaggregated July–October 2026 • fix: explicit thread_id per worker • 2.0h retainer engagement

Birchwood Digital AI Ltd. runs a nightly batch document processing job that ingests 2,000 customer support ticket summaries, passes each through a LangChain + GPT-4o pipeline, and stores the outputs for downstream analytics. They use Lunary for LLM monitoring and observability. The intended design was straightforward: each nightly run would appear in the Lunary dashboard as a named thread — batch_YYYY-MM-DD — containing 2,000 child traces, one per document processed. After four months of running the pipeline, a data engineer reviewing the Lunary dashboard noticed something wrong. Every batch_YYYY-MM-DD thread showed zero traces. Meanwhile, thousands of traces were scattered across the dashboard as unparented events with no thread attribution. Birchwood had been running the pipeline since July 2026. By mid-October, approximately 245,000 traces across 122 nightly runs were sitting in Lunary as orphaned events. None of them were associated with any batch thread. No per-batch aggregate metrics — total tokens, average latency, model cost per run — could be computed. They brought in a Lunary retainer developer to explain why four months of batch monitoring data was unaggregated.

The nightly job was structured as a Python script that opened a ThreadPoolExecutor to process documents concurrently across eight workers. The batch thread was started in the main thread before the executor was opened, and the worker function created a fresh LunaryCallbackHandler for each document:

# jobs/nightly_batch.py
# Birchwood Digital AI Ltd.
# lunary 0.7.3, Python 3.11, LangChain 0.2.15

import lunary
from lunary.langchain import LunaryCallbackHandler
from langchain_openai import ChatOpenAI
from langchain.chains import LLMChain
from langchain.prompts import ChatPromptTemplate
from concurrent.futures import ThreadPoolExecutor, as_completed
import datetime

def process_document(doc_id: str, content: str) -> dict:
    """Process a single document — called from worker thread."""

    # Create LunaryCallbackHandler for this document
    # BUG: thread_id not passed — defaults to main thread's context (which doesn't propagate to workers)
    handler = LunaryCallbackHandler()

    llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])
    prompt = ChatPromptTemplate.from_messages([
        ("system", "Summarize this customer support ticket."),
        ("human", "{content}")
    ])
    chain = LLMChain(llm=llm, prompt=prompt, callbacks=[handler])

    result = chain.invoke({"content": content})
    return {"doc_id": doc_id, "summary": result["text"]}


def run_nightly_batch(documents: list[dict]) -> None:
    """Run nightly batch processing across 2,000 documents."""

    batch_date = datetime.date.today().isoformat()

    # Start a Lunary thread for this batch run
    lunary.start_thread(
        thread_id=f"batch_{batch_date}",
        name=f"Nightly Batch {batch_date}",
    )

    results = []
    with ThreadPoolExecutor(max_workers=8) as executor:
        futures = {
            executor.submit(process_document, doc["id"], doc["content"]): doc["id"]
            for doc in documents
        }
        for future in as_completed(futures):
            results.append(future.result())

    lunary.end_thread()

    print(f"Processed {len(results)} documents")

The bug is in Python’s threading model. lunary.start_thread() writes the batch thread_id into the main thread’s threading.local() storage. threading.local() is, by definition, per-OS-thread: each OS thread has its own isolated namespace, and values written in one thread are never visible in any other. When ThreadPoolExecutor creates worker threads — up to max_workers=8 new OS threads — those workers start with a completely empty threading.local() namespace. The batch_2026-10-15 context written by lunary.start_thread() in the main thread does not exist in any worker thread. When LunaryCallbackHandler() is instantiated inside process_document(), it reads from the worker thread’s empty threading.local() and finds no thread_id. Lunary sends the resulting trace to the API with no parent attribution. The trace is stored as an unparented event. The batch_YYYY-MM-DD thread receives nothing.

This failure mode is completely silent at the application layer. The LangChain chains complete successfully. The GPT-4o responses are correct. The result["text"] values are populated. No exception is raised anywhere in the pipeline. The only visible symptom is in the Lunary dashboard, and only if someone is actively comparing the batch thread trace counts against the expected number of documents processed per run. Birchwood’s team had not built that check into their operational monitoring, so the issue went undetected from July 2026 through mid-October 2026 — four months and roughly 122 nightly runs.

The retainer developer confirmed the diagnosis with a targeted Lunary API query that made the gap explicit:

# diagnosis: list the batch thread's traces
import lunary

# Check traces for the batch_2026-10-15 thread
client = lunary.LunaryClient(app_id="...")
thread = client.get_thread("batch_2026-10-15")
print(f"Thread traces: {len(thread.traces)}")  # Thread traces: 0

# Check orphaned traces from last night
orphaned = client.get_runs(
    type="llm",
    from_date="2026-10-15T00:00:00Z",
    to_date="2026-10-15T23:59:59Z",
    parent_id=None,  # no parent = orphaned
)
print(f"Orphaned traces: {len(orphaned)}")  # Orphaned traces: 2000

The thread showed zero traces. The same 24-hour window contained exactly 2,000 unparented traces — one for every document in the October 15 nightly run. The same query pattern held for every date going back to July. The diagnosis was confirmed: all 2,000 worker-thread traces per run were orphaned because LunaryCallbackHandler() was instantiated in worker threads that had no Lunary thread context.

The fix required three changes to the worker function and one to the batch runner. First, add batch_thread_id as an explicit parameter to process_document() so the thread identifier is passed in as a function argument rather than read from thread-local storage. Second, call lunary.set_parent(thread_id=batch_thread_id) at the top of the worker function body to initialize the worker thread’s own Lunary context. Third, pass thread_id=batch_thread_id to LunaryCallbackHandler() so the handler is explicitly bound to the batch thread. In the batch runner, thread the batch_thread_id string through executor.submit():

# Fixed version — explicit thread_id passed to each worker

def process_document(doc_id: str, content: str, batch_thread_id: str) -> dict:
    """Process a single document — called from worker thread."""

    # Set the batch thread context in this worker thread
    # Thread-local context must be initialized in each worker, not inherited from main
    lunary.set_parent(thread_id=batch_thread_id)

    handler = LunaryCallbackHandler(thread_id=batch_thread_id)

    llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])
    prompt = ChatPromptTemplate.from_messages([
        ("system", "Summarize this customer support ticket."),
        ("human", "{content}")
    ])
    chain = LLMChain(llm=llm, prompt=prompt, callbacks=[handler])

    result = chain.invoke({"content": content})
    return {"doc_id": doc_id, "summary": result["text"]}


def run_nightly_batch(documents: list[dict]) -> None:
    """Run nightly batch processing across 2,000 documents."""

    batch_date = datetime.date.today().isoformat()
    batch_thread_id = f"batch_{batch_date}"

    lunary.start_thread(
        thread_id=batch_thread_id,
        name=f"Nightly Batch {batch_date}",
    )

    results = []
    with ThreadPoolExecutor(max_workers=8) as executor:
        futures = {
            executor.submit(process_document, doc["id"], doc["content"], batch_thread_id): doc["id"]
            for doc in documents
        }
        for future in as_completed(futures):
            results.append(future.result())

    lunary.end_thread()

    print(f"Processed {len(results)} documents")

After deploying the fix, the October 17 nightly run produced a batch_2026-10-17 thread in the Lunary dashboard containing all 2,000 traces. For the first time, the team could see per-batch aggregate metrics: total tokens consumed, average LangChain chain latency per document, model cost per run, and the distribution of trace durations across the eight worker threads. The thread-level view also immediately surfaced that the eight workers were not evenly loaded — workers assigned longer tickets were taking 2.1× as long as workers assigned shorter tickets, a parallelism imbalance that had been completely invisible when all traces were orphaned.

The retroactive analysis was particularly useful. Because GPT-4o’s responses had been correct throughout the four-month period — the bug was purely an observability failure, not a functional one — the team re-ran a stratified sample of 300 historical documents from July, August, and September through the fixed pipeline to generate comparable metrics. The ROUGE-L scores for the generated summaries were consistent at 0.79 across all three months, confirming that summary quality had been stable throughout the period despite the monitoring gap. The batch processing pipeline had been working correctly all along; it simply had not been observable in Lunary.

The retainer developer also identified a secondary issue in the original code: lunary.end_thread() was called unconditionally after the as_completed() loop, but if any future raised an exception, the exception would propagate out of future.result(), unwind the stack, and bypass lunary.end_thread() entirely. This would leave the batch thread in Lunary in an “open” state indefinitely. The fix is a try/finally block:

    try:
        for future in as_completed(futures):
            results.append(future.result())
    finally:
        lunary.end_thread()

With try/finally, lunary.end_thread() is called regardless of whether a future succeeds or raises. The batch thread is always closed cleanly in Lunary. For production batch jobs where individual document failures are expected — malformed inputs, transient API errors, rate limit exceptions — this guarantees that thread-level metrics are recorded even for partial runs. The team updated the batch runner to catch per-document exceptions inside the loop body and record them to a failed-documents list rather than propagating them, so a single bad document would not abort the remaining 1,999.

The postmortem defined two new operational requirements. First, all batch jobs that use Lunary threads must include a post-run assertion that checks thread.traces count against the number of documents submitted. If the count is less than 95% of expected, the job must emit a WARN log line so the monitoring alert fires. Second, any code path that creates a LunaryCallbackHandler in a function that may be called from a non-main OS thread must explicitly pass thread_id as an argument; instantiating LunaryCallbackHandler() without thread_id is prohibited in the team’s LLM observability code review checklist. The rule is enforced by a pre-commit hook that greps for bare LunaryCallbackHandler() calls in files under the jobs/ directory.

Total retainer engagement: 2.0 hours (0.5h initial triage: query Lunary API for batch thread trace counts using client.get_thread() for the most recent five batch dates, confirm each returns 0 traces, use client.get_runs(parent_id=None) to confirm 2,000 unparented traces in the same nightly time windows, establish that the pattern holds across all dates back to July 2026; 0.5h root cause: examine process_document() and run_nightly_batch() source, identify LunaryCallbackHandler() instantiated without thread_id in worker thread bodies, confirm that lunary.start_thread() uses threading.local() internally and that ThreadPoolExecutor workers are independent OS threads with no inherited thread-local state; 0.75h fix: add batch_thread_id parameter to process_document(), thread it through executor.submit() calls, add lunary.set_parent(thread_id=batch_thread_id) at top of worker function, add thread_id=batch_thread_id to LunaryCallbackHandler() constructor, wrap futures loop in try/finally with lunary.end_thread() in finally block, deploy and run nightly job manually against a 50-document test batch; 0.25h validation: confirm test batch thread shows 50 child traces in Lunary dashboard, confirm no orphaned traces in same time window, run retroactive sample of 300 historical documents to confirm ROUGE-L stability at 0.79 across July–September).

Related retainer posts

• Langfuse developer on retainer — flush thread dies on gunicorn fork, 25.7% trace delivery rate for 7 days
• TruLens developer on retainer — metadata-polluted context extractor, 0.42 groundedness score for 6 months
• Helicone developer on retainer — per-user UUID in cache key, 2.1% cache hit rate instead of expected 60%