Fireworks AI developer on retainer: FireFunction-v2 injects full conversation history into context function argument — args[“context”] contains 50+ messages — 25× cost overrun for 8 weeks — fireworks-ai 0.15.0 — Fireworks developer on monthly retainer

fireworks-ai 0.15.0 • Python 3.11 • accounts/fireworks/models/firefunction-v2 • 4 functions in tools • get_related_products schema includes a “context” parameter • FireFunction-v2 injects full conversation history into “context” when schema defines it • 52 message dicts in args[“context”] instead of 1–3 product IDs • downstream iterates all 52 items • 25× cost overrun for 8 weeks • fix: remove “context” from function schema • 2.5h retainer engagement

Eastbrook AI Ltd. built a product recommendation assistant using FireFunction-v2 in August 2026. The assistant let customers browse a product catalog and used function calling to retrieve product details, check inventory, and generate personalized recommendations. The system returned recommendations successfully, but the recommendation quality was unexpectedly poor in long browsing sessions: the model appeared to ignore the products the user had looked at during the session. No errors were logged. A cost review in October 2026 also flagged a 25× higher-than-expected inference cost on the get_related_products function call. They engaged a Fireworks retainer developer to investigate both symptoms simultaneously.

The function calling setup at the time of the engagement:

# recommender.py
# Eastbrook AI Ltd.
# fireworks-ai 0.15.0, Python 3.11

import json
from fireworks.client import Fireworks

client = Fireworks(api_key="FIREWORKS_API_KEY")

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "get_product_details",
            "description": "Get the full details of a product by its ID.",
            "parameters": {
                "type": "object",
                "properties": {
                    "product_id": {"type": "string"},
                },
                "required": ["product_id"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "get_related_products",
            "description": "Get products related to a given product, filtered by items the user has already viewed.",
            "parameters": {
                "type": "object",
                "properties": {
                    "product_id": {"type": "string"},
                    "context": {
                        # ← PROBLEM: The parameter is named "context".
                        # FireFunction-v2 has a special behavior: when ANY function
                        # in the tools list has a parameter named "context",
                        # FireFunction-v2 injects the FULL conversation history
                        # (all messages[] entries) into that parameter.
                        # The injection overrides whatever the user passes and
                        # ignores the schema's type and description constraints.
                        # Intent: pass 1-3 product_ids from this session.
                        # What actually arrives: 52 message dicts (full history).
                        "type": "array",
                        "items": {"type": "string"},
                        "description": "Product IDs the user viewed in this session.",
                        "default": [],
                    },
                    "limit": {"type": "integer", "default": 5},
                },
                "required": ["product_id"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "check_inventory",
            "description": "Check the inventory level for a product.",
            "parameters": {
                "type": "object",
                "properties": {
                    "product_id": {"type": "string"},
                },
                "required": ["product_id"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "get_user_history",
            "description": "Get the purchase history for a user.",
            "parameters": {
                "type": "object",
                "properties": {
                    "user_id": {"type": "string"},
                },
                "required": ["user_id"],
            },
        },
    },
]

def get_recommendations(messages: list, product_id: str) -> list[str]:
    response = client.chat.completions.create(
        model="accounts/fireworks/models/firefunction-v2",
        messages=messages,  # 52 messages after a long browsing session
        tools=TOOLS,
        tool_choice="auto",
    )

    msg = response.choices[0].message
    if msg.tool_calls:
        call = msg.tool_calls[0]
        args = json.loads(call.function.arguments)
        # args["context"] is expected to be ["prod_001", "prod_007", "prod_023"]
        # BUT: args["context"] is actually a list of 52 message dicts:
        # [
        #   {"role": "user", "content": "show me running shoes"},
        #   {"role": "assistant", "content": "Here are some options..."},
        #   ...52 items...
        # ]
        #
        # TypeError: string indices must be integers
        # (when downstream code does: for product_id in args["context"]: db.get(product_id))
        # Caught by generic except → return fallback top-5 popular products
        related = recommend_engine.get_related(
            product_id=args["product_id"],
            context_products=args.get("context", []),  # ← gets 52 message dicts
            limit=args.get("limit", 5),
        )
        return related
    return []

How FireFunction-v2 context injection works

FireFunction-v2 is a specialized function calling model trained by Fireworks AI on conversational function use. It includes an automatic context-passing feature: when the model determines that a function needs access to the conversation history — for example, to filter recommendations by what the user has already discussed — it injects the full messages list as the value of any parameter named context.

This injection is triggered purely by parameter naming. If any function in the tools list has a parameter with the key "context", FireFunction-v2 will populate it with the complete conversation history as a list of {"role": str, "content": str} dicts. The schema’s type, description, and any other constraints are ignored for the injection. The resulting function_call.arguments JSON is well-formed (valid JSON, no error), which means json.loads() succeeds and the bug only surfaces in downstream code that processes the argument value.

# Demonstrating FireFunction-v2 context injection
# fireworks-ai 0.15.0, Python 3.11

import json
from fireworks.client import Fireworks

client = Fireworks(api_key="FIREWORKS_API_KEY")

# 52-message conversation history (26 browsing turns)
long_conversation = [
    {"role": "user", "content": "show me running shoes"},
    {"role": "assistant", "content": "Here are some options in the running shoes category..."},
    {"role": "user", "content": "what about trail running shoes?"},
    {"role": "assistant", "content": "For trail running, these models are popular..."},
    # ... 48 more messages ...
]

# Test 1: function schema WITH "context" parameter
response_with_context_param = client.chat.completions.create(
    model="accounts/fireworks/models/firefunction-v2",
    messages=long_conversation + [
        {"role": "user", "content": "find shoes similar to prod_trail_001"}
    ],
    tools=TOOLS,  # ← includes "context" parameter in get_related_products
    tool_choice={"type": "function", "function": {"name": "get_related_products"}},
)

args = json.loads(response_with_context_param.choices[0].message.tool_calls[0].function.arguments)
print("product_id:", args["product_id"])
print("context type:", type(args["context"]))
print("context length:", len(args["context"]))
print("context[0]:", args["context"][0])
# Output:
# product_id: prod_trail_001
# context type: 
# context length: 52              ← full conversation history injected
# context[0]: {'role': 'user', 'content': 'show me running shoes'}
# ← NOT the product IDs the user viewed. The conversation history.

# Token cost for this function call:
# Injected context tokens: 52 messages × 80 tokens/msg = 4,160 extra tokens
# Expected cost without injection: ~180 tokens
# Actual cost: ~4,340 tokens ← 24x overrun

# Test 2: same schema but WITHOUT "context" parameter
TOOLS_NO_CONTEXT = [
    # ... same as TOOLS but "context" renamed to "recent_product_ids" ...
]

response_no_context_param = client.chat.completions.create(
    model="accounts/fireworks/models/firefunction-v2",
    messages=long_conversation + [
        {"role": "user", "content": "find shoes similar to prod_trail_001"}
    ],
    tools=TOOLS_NO_CONTEXT,
    tool_choice={"type": "function", "function": {"name": "get_related_products"}},
)

args2 = json.loads(response_no_context_param.choices[0].message.tool_calls[0].function.arguments)
print("product_id:", args2["product_id"])
print("recent_product_ids:", args2.get("recent_product_ids"))
# Output:
# product_id: prod_trail_001
# recent_product_ids: None   ← model doesn't auto-populate without injection trigger
#                              (pass explicitly or remove the parameter entirely)

Why the overrun went undetected for 8 weeks

The cost overrun was not obvious for three reasons. First, the per-request cost increase was small in absolute terms: each injected context call cost approximately $0.0021 extra (4,160 tokens at FireFunction-v2 pricing), spread across 200 daily calls. The total monthly overrun was roughly $12 — large as a multiplier (25×) but below the threshold that would trigger an automatic billing alert. Second, the function call succeeded and returned a result. The TypeError from iterating message dicts instead of product ID strings was caught by a generic except Exception handler that fell back to top-5 popular products. The team saw the fallback behavior as a recommendation quality issue, not as a code error. Third, there was no per-function token tracking in the observability setup. The team tracked total daily token usage, not the breakdown by function call. The overrun was buried in the daily total.

The engagement diagnosis started by logging the raw function_call.arguments JSON for all get_related_products calls. The first logged payload was 6.2 KB. Expected size for 3 product IDs and a limit parameter: under 100 bytes.

The fix: remove “context” from the function parameter schema

# recommender.py — FIXED
# Eastbrook AI Ltd.
# fireworks-ai 0.15.0, Python 3.11

import json
from fireworks.client import Fireworks

client = Fireworks(api_key="FIREWORKS_API_KEY")

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "get_product_details",
            "description": "Get the full details of a product by its ID.",
            "parameters": {
                "type": "object",
                "properties": {
                    "product_id": {"type": "string"},
                },
                "required": ["product_id"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "get_related_products",
            "description": "Get products related to a given product.",
            "parameters": {
                "type": "object",
                "properties": {
                    "product_id": {"type": "string"},
                    # ← FIX: "context" parameter REMOVED entirely.
                    # FireFunction-v2 only injects conversation history
                    # when a parameter is named exactly "context".
                    # Removing this parameter stops the injection.
                    # The recommendation engine receives product_id and limit;
                    # recent_product_ids is passed explicitly by the application
                    # layer from session state, not via function call arguments.
                    "limit": {"type": "integer", "default": 5},
                },
                "required": ["product_id"],
            },
        },
    },
    # ... check_inventory and get_user_history unchanged ...
]

def get_recommendations(
    messages: list,
    product_id: str,
    session_product_ids: list[str],  # ← application passes session context explicitly
) -> list[str]:
    response = client.chat.completions.create(
        model="accounts/fireworks/models/firefunction-v2",
        messages=messages,
        tools=TOOLS,
        tool_choice="auto",
    )

    msg = response.choices[0].message
    if msg.tool_calls:
        call = msg.tool_calls[0]
        if call.function.name == "get_related_products":
            args = json.loads(call.function.arguments)
            # args now contains only: {"product_id": "...", "limit": 5}
            # No "context" key. No conversation history injection.
            related = recommend_engine.get_related(
                product_id=args["product_id"],
                context_products=session_product_ids,  # ← passed from application layer
                limit=args.get("limit", 5),
            )
            return related
    return []

Adding argument size monitoring to catch future anomalies

# Per-request argument size monitoring
# fireworks-ai 0.15.0, Python 3.11

import json
import logging

logger = logging.getLogger(__name__)

MAX_ARGS_BYTES = 2_000  # alert threshold for function_call.arguments

def dispatch_tool_call(call) -> dict:
    raw_args = call.function.arguments
    args_size = len(raw_args.encode("utf-8"))

    if args_size > MAX_ARGS_BYTES:
        logger.warning(
            "Large function_call.arguments: function=%s size=%d bytes",
            call.function.name,
            args_size,
        )
        # Alert: check for FireFunction-v2 context injection.
        # If any function schema has a "context" parameter, this is
        # likely the conversation history being injected.

    args = json.loads(raw_args)

    # Schema validation: assert no conversation history in args
    for key, value in args.items():
        if isinstance(value, list) and len(value) > 10:
            if value and isinstance(value[0], dict) and "role" in value[0]:
                logger.error(
                    "FireFunction-v2 context injection detected: "
                    "function=%s parameter=%s length=%d",
                    call.function.name, key, len(value),
                )
                # This indicates a "context" parameter is still in the schema.
                raise ValueError(
                    f"Unexpected conversation history in {key}. "
                    f"Remove 'context' from the function parameter schema."
                )

    return args

Validation

# Validation benchmark (October 29 2026)
# 100 get_related_products calls from October 28 production sessions
# fireworks-ai 0.15.0, Python 3.11, accounts/fireworks/models/firefunction-v2

# Before fix ("context" parameter in get_related_products schema):
# function_call.arguments size:  p50=6,200 bytes  p95=9,800 bytes
# args["context"] type:          list[dict] in 100% of calls (message history injected)
# args["context"] length:        p50=52  p95=87 messages
# TypeError on downstream iterate: 100% (message dicts, not product ID strings)
# Fallback to popular products:  100%
# Recommendation quality (CTR):  2.1%  (vs 4.8% for non-context-injection calls)
# Per-call token cost (avg):     4,520 tokens  (180 expected)
# Per-call cost overrun:         25.1×

# After fix ("context" parameter removed from schema):
# function_call.arguments size:  p50=82 bytes  p95=105 bytes  ← expected
# args keys:                     {"product_id": "...", "limit": N}  — no "context"
# TypeError on downstream:       0 calls
# Fallback rate:                 0%  (correct recommendations returned)
# Recommendation CTR:            4.9%  (matches expected quality)
# Per-call token cost (avg):     175 tokens
# Cost overrun:                  0×

# Cost impact (Eastbrook AI Ltd.):
# Daily savings: 200 calls × (4,520 - 175) tokens × $0.0005/1K tokens = $0.43/day
# 8-week accumulated overrun: 56 days × $0.43 = $24.08 (actual excess paid)
# Future monthly savings: ~$13/month
# 2.5h retainer engagement.

Related retainer posts

Track retainer hours without a client portal

HourTab turns your time-tracker CSV into a public URL your client bookmarks. No login. No portal. CSV in, URL out.

See pricing →