Perplexity AI developer on retainer: online model citation footnotes break regex extraction — [1][2] markers interleaved mid-sentence in message.content — 22% of summaries fail for 6 weeks — openai 1.35.0 — Perplexity AI developer on monthly retainer

openai 1.35.0 • Python 3.11 • llama-3.1-sonar-small-128k-online on Perplexity AI • online model appends [1][2][3] citation footnotes inline in message.content • client regex extractor breaks on bracket markers mid-sentence • 22% of research summaries fail structured extraction for 6 weeks • fix: strip citation markers; read response.citations field directly • 2.0h retainer engagement

Northfield Research Ltd. built a competitive intelligence pipeline in April 2026 using Perplexity AI’s llama-3.1-sonar-small-128k-online model to gather and summarize real-time market data. A downstream Python parser used a regular expression to extract structured fields — company names, revenue figures, product release dates — from the model’s response text. The pipeline ran at 500 requests per day across 30 industry verticals. Within the first two weeks, the team noticed that roughly 22% of summaries had broken structured fields — truncated company names, missing figures, corrupted dates. They spent six weeks adjusting prompt phrasing, switching system message formats, and testing different temperature values, attributing the problem to model output inconsistency.

Perplexity AI’s online models perform live web searches before generating each response. They insert numeric citation footnote markers — [1], [2], [3] — directly inline in the response text at the point where each claim is supported by a web source. A sentence like "Revenue grew 34% year-over-year [1][2] to reach $2.1 billion [3]." contains three citation markers embedded mid-sentence. The regex pattern the team used to extract revenue figures treated [ as a word boundary character, causing the match to stop or fail whenever a citation marker appeared between a number and its unit. The 22% failure rate corresponded precisely to the fraction of responses where citation markers appeared between the regex target tokens.

The research pipeline code at the time of the engagement:

# research_pipeline.py
# Northfield Research Ltd.
# openai 1.35.0, Python 3.11
# llama-3.1-sonar-small-128k-online on Perplexity AI

import openai
import re
import os

client = openai.OpenAI(
    api_key=os.environ["PPLX_API_KEY"],
    base_url="https://api.perplexity.ai",
)

REVENUE_RE = re.compile(
    r"\$[\d,\.]+\s*(?:million|billion|M|B)",
    re.IGNORECASE,
)

def summarize_company(company_name: str) -> dict:
    """Fetch a real-time summary and extract key financials."""
    response = client.chat.completions.create(
        model="llama-3.1-sonar-small-128k-online",
        messages=[
            {
                "role": "system",
                "content": "Summarize recent financials, funding, and product releases.",
            },
            {
                "role": "user",
                "content": f"Summarize {company_name} recent financials and key metrics.",
            },
        ],
        temperature=0.2,
        max_tokens=1024,
    )

    # ← PROBLEM: message.content from online models includes inline citation
    # markers like [1][2][3] embedded directly in the generated text.
    # Example content:
    #   "Revenue grew 34% year-over-year [1][2] to reach $2.1 billion [3].
    #    The company completed a $450 million [1] Series D in March 2026."
    #
    # The REVENUE_RE regex matches "$2.1 billion" but the pattern
    # r"\$[\d,\.]+\s*(?:million|billion|M|B)" expects whitespace between
    # the number and the unit. When [3] appears between "$2.1" and "billion"
    # — as in "$2.1 [3] billion" — the match fails.
    #
    # response.citations exists but is never read here.
    # It contains: ["https://source1.com/...", "https://source2.com/...", ...]
    content = response.choices[0].message.content  # ← citations embedded here
    revenue_matches = REVENUE_RE.findall(content)  # ← breaks when [N] mid-token

    return {
        "company": company_name,
        "summary": content,
        "revenues_found": revenue_matches,
    }

How Perplexity AI online model citations appear in the response

Perplexity AI’s online models insert citation markers inline in the generated text and also provide a structured citations field at the top level of the response object. The message.content field contains the full text with markers embedded — the markers are not stripped or hidden. A typical online model response looks like this:

# What Perplexity AI returns for an online model request
# response.choices[0].message.content:

"""
Acme Corp reported revenue of $2.1 billion [1][2] for fiscal year 2025,
up 34% year-over-year. The company raised a $450 million [3] Series D
in March 2026 [1] led by Sequoia Capital. Their flagship product,
AcmeCloud, reached 5 million [2] enterprise customers in Q1 2026.
"""

# response.citations (top-level field, not inside choices[]):
[
    "https://techcrunch.com/2026/03/acme-corp-series-d",
    "https://acmecorp.com/investors/2025-annual-report",
    "https://wsj.com/articles/acme-corp-funding-2026",
]

# The citation markers [1], [2], [3] in message.content correspond to the
# 0-indexed + 1 positions in the citations list:
#   [1] → citations[0]
#   [2] → citations[1]
#   [3] → citations[2]

# When the regex r"\$[\d,\.]+\s*(?:billion|million|M|B)" runs on the raw
# message.content, it matches "$2.1" but the space after "2.1" is occupied
# by "[1][2]", not a space followed by "billion" — so the pattern fails.
# "$450 million [3]" works (marker after unit) but "$2.1 [1][2] billion"
# fails (marker between number and unit).

The fix: strip citation markers and read response.citations directly

# research_pipeline.py — FIXED
# Northfield Research Ltd.
# openai 1.35.0, Python 3.11

import openai
import re
import os

client = openai.OpenAI(
    api_key=os.environ["PPLX_API_KEY"],
    base_url="https://api.perplexity.ai",
)

REVENUE_RE = re.compile(
    r"\$[\d,\.]+\s*(?:million|billion|M|B)",
    re.IGNORECASE,
)

# ← FIX 1: Strip inline citation markers from message.content before any
# regex or structured extraction.
CITATION_MARKER_RE = re.compile(r"\[\d+\]")

def strip_citations(content: str) -> str:
    """Remove inline [N] citation markers from Perplexity online model output."""
    # Remove all [1], [2], [3], ... markers.
    clean = CITATION_MARKER_RE.sub("", content)
    # Also remove any trailing "Citations:" section appended to message.content.
    clean = clean.split("\nCitations:")[0].strip()
    return clean


def summarize_company(company_name: str) -> dict:
    """Fetch a real-time summary and extract key financials."""
    response = client.chat.completions.create(
        model="llama-3.1-sonar-small-128k-online",
        messages=[
            {
                "role": "system",
                "content": "Summarize recent financials, funding, and product releases.",
            },
            {
                "role": "user",
                "content": f"Summarize {company_name} recent financials and key metrics.",
            },
        ],
        temperature=0.2,
        max_tokens=1024,
    )

    raw_content = response.choices[0].message.content

    # ← FIX 1: Strip citation markers before regex extraction.
    clean_content = strip_citations(raw_content)
    revenue_matches = REVENUE_RE.findall(clean_content)  # ← now works correctly

    # ← FIX 2: Read response.citations for source URLs — do not parse URLs
    # from message.content. response.citations is a top-level list on the
    # response object (not nested inside choices).
    citations = getattr(response, "citations", []) or []

    return {
        "company": company_name,
        "summary": clean_content,
        "revenues_found": revenue_matches,
        "source_urls": citations,  # ← proper source list from response.citations
    }


# ← FIX 3: For workloads where citations are not needed, use an offline
# Perplexity model to avoid citation markers entirely.
def summarize_company_offline(company_name: str) -> dict:
    """Use an offline model when live web citations are not needed."""
    response = client.chat.completions.create(
        model="llama-3.1-70b-instruct",  # ← offline model: no citations appended
        messages=[
            {
                "role": "system",
                "content": "Summarize recent financials, funding, and product releases.",
            },
            {
                "role": "user",
                "content": f"Summarize {company_name} recent financials and key metrics.",
            },
        ],
        temperature=0.2,
        max_tokens=1024,
    )
    content = response.choices[0].message.content  # ← no markers; safe to parse directly
    revenue_matches = REVENUE_RE.findall(content)
    return {
        "company": company_name,
        "summary": content,
        "revenues_found": revenue_matches,
        "source_urls": [],
    }

Why six weeks of prompt tuning found nothing

The Northfield team’s leading hypothesis for the 22% failure rate was that the model was generating malformed or inconsistently structured output for certain company names. They spent three weeks trying different system prompt formats, adding explicit instructions like “always include revenue figures with units immediately adjacent to the number, without any intervening characters,” and varying temperature and top-p parameters. None of these changes affected the failure rate because the problem was not in the model’s output quality — it was in the citation markers that the model correctly inserted as designed behavior.

The team then spent two weeks manually reviewing the failing extractions. They noticed the [1][2] patterns in the raw content but interpreted them as part of the model’s output style rather than as a structural feature of the API response. They did not read the Perplexity API documentation section on online model citation format, which explains the citations field and the inline marker behavior.

The retainer developer opened the response object in a REPL, printed dir(response), and spotted the citations attribute alongside choices. Printing one raw response.choices[0].message.content string showed the [1][2] markers embedded between “$2.1” and “billion” in a sample failing extraction. The root cause was clear in under five minutes. The fix — a two-line preprocessing function — was implemented in fifteen minutes. The remaining time was spent adding model-type routing, CI tests, and validating 50 responses across different industry verticals.

Validation

# Validation benchmark (October 30, 2026)
# 50-company batch, llama-3.1-sonar-small-128k-online on Perplexity AI
# openai 1.35.0, Python 3.11

# Before fix (raw message.content passed to regex):
# Companies processed:                    50 / 50
# Extractions fully correct:             39 / 50  (78%)
# Extractions broken (citation marker):  11 / 50  (22%)  ← [N] between number/unit
# Revenue matches missing:               11        (11 summaries lost revenue data)
# Root cause: "[1][2]" embedded between "$2.1" and "billion" breaks regex
# Team diagnosis: model output inconsistency
# Duration: 6 weeks April–October 2026

# After fix (strip_citations() applied before regex):
# Companies processed:                    50 / 50
# Extractions fully correct:             50 / 50 (100%)
# Extractions broken:                     0 / 50   (0%)
# response.citations populated:          50 / 50 (all online model responses)
# Average citations per response:        ~3.2 source URLs

# Production result (Northfield Research Ltd.):
# Structured extraction failure rate:   22% → 0%
# Prompt changes required:              zero (root cause was not in prompt)
# Model changes required:               zero (model was working correctly)
# Fix: 2-line strip_citations() preprocessor + read response.citations field
# 2.0h retainer engagement.
# Triage: 0.4h — print raw response; spot [N] markers; confirm REVENUE_RE fails
#   when "[1][2]" between number and unit.
# Root cause: 0.4h — confirm strip_citations() removes markers; regex works on
#   clean content; response.citations contains proper URL list.
# Fix: 0.6h — implement strip_citations(); add 'Citations:' block splitter;
#   add model-type routing (online vs. offline); add response.citations extraction.
# Validation: 0.6h — 50-company batch; 100% extraction success; CI test added.

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →