Blog › ICP guides

Data analytics engineer on retainer: tracking BI dashboard advisory and demonstrating KPI framework value between dashboard releases and metric rollouts

July 24, 2026 · ~19 min read

The revenue dashboard has a tile labeled “Revenue.” The sales pipeline dashboard also has a tile labeled “Revenue.” In the Monday morning executive review, the CFO quotes $4.2M for the quarter. The VP Sales quotes $3.4M for the same quarter. Neither number is wrong on its own terms. The CFO is reading gross bookings from the executive summary dashboard. The VP Sales is reading net revenue from the sales pipeline dashboard. The two numbers differ by 23% because the company has an 18% average refund rate that one dashboard applies and the other does not. Both tiles are labeled “Revenue.” Neither label is technically incorrect — gross bookings is a legitimate revenue metric and so is net revenue after refunds. The problem is that a room full of executives is using the same word to mean two different things and no one in the room has any reason to suspect it, because both tiles look identical on the dashboard surface and neither tooltip, description, nor footnote distinguishes them.

The only person who can find this problem is someone who traces both dashboard tiles back to their underlying SQL definitions in the BI tool and reads the queries side by side. The executive summary dashboard’s Revenue explore is querying the bookings fact table and summing gross_booking_amount before any refund records have been applied. The sales pipeline dashboard’s Revenue explore is querying the recognized_revenue table, which applies a standard 18% refund deduction to gross bookings before summing. The two queries are pulling from different source tables with different refund treatment logic, and the divergence is completely invisible from the dashboard surface. You have to open the SQL editor, examine the FROM clause, trace the join conditions, and compare the field definitions line by line. This is the diagnostic work of a data analytics engineer operating on retainer — not building new dashboards, but governing the metric definitions that existing dashboards already rely on, before the misalignment propagates into a board presentation where the CFO and the VP Sales quote revenue figures that differ by 23% and both believe their number is correct.

This is not a rare scenario. It is the default state of any analytics environment that has accumulated dashboards across multiple teams without a metric governance layer. The finance team builds a revenue dashboard in one sprint. The sales team builds a pipeline dashboard in another. Both teams define “Revenue” in the way that makes sense for their analytical context. Neither team knows the other dashboard exists. The BI tool stores both definitions in separate explores with no cross-reference. A year later, the company has seven dashboards each with a “Revenue” metric, three distinct SQL definitions behind those seven tiles, and an executive team that has stopped trusting any of the numbers because the numbers never agree. A data analytics engineer on retainer is the governance layer that prevents this accumulation by reviewing metric definitions before dashboards are published, tracing divergent definitions to their source when they are identified, and establishing the canonical metric definitions in the semantic layer that subsequent dashboards are required to reference rather than redefine.

The BI dashboard release and the KPI framework rollout are the visible analytics events. A new revenue dashboard shipped to the executive team is visible. A company-wide KPI framework launched to all department heads is visible. What is not visible is the continuous advisory between those milestones: the metric definition governance that catches the inconsistency before it reaches the board meeting, the KPI framework advisory that distinguishes the leading indicators from the lagging ones before an entire go-to-market team orients its weekly rhythm around a metric that tells them what already happened rather than what is likely to happen next, the data visualization advisory that replaces an 11-slice pie chart with a ranked bar chart that answers the question in three seconds rather than thirty, the self-service analytics governance that keeps the certified dataset tier meaningful rather than a list of every table anyone has ever marked as “official,” and the data quality monitoring advisory that ensures the 9am Monday executive review is looking at data from this morning rather than data from Friday. All of that advisory is invisible to the data director and the CDO without a work log that connects the ongoing governance to the analytical reliability and decision quality it produces.

BI dashboard design advisory

BI dashboard design advisory is not about aesthetics. It is about whether the numbers on the dashboard mean what the viewer believes they mean, whether the filters produce the results the viewer expects when they apply them, and whether the drill-down path from a summary metric to the underlying rows that compose it resolves correctly without the denominator silently changing at an intermediate level. These are structural correctness questions that require the data analytics engineer to read the SQL definitions behind the dashboard surface rather than evaluate the visual presentation.

Metric definition governance

Metric definition inconsistency is the most common failure mode in multi-dashboard analytics environments, and it is the failure mode most likely to cause organizational damage before it is detected. The revenue scenario described in the opening is the canonical form: two legitimate metric definitions sharing a label, diverging in value, and producing executive disagreement that erodes trust in the analytics function rather than in the metric definition governance that failed to establish a canonical definition. But the pattern appears across every business metric category. Conversion rate defined as sessions-to-signups in the marketing dashboard and as signups-to-activated-users in the product dashboard. Churn rate defined as the number of accounts who canceled divided by the end-of-period account count in the finance dashboard and as the number of accounts who canceled divided by the beginning-of-period account count in the customer success dashboard. Monthly active users defined as any user with at least one login event in the growth dashboard and as any user with at least one core feature interaction in the product dashboard. Each pair of definitions is defensible. The combination produces a 15–30% divergence in the reported metric value depending on the period and the user behavior distribution, and both definitions are labeled identically on the dashboard surface.

Metric definition governance on retainer covers the systematic review of metric definitions across the dashboard environment: identifying metrics with the same label but different SQL definitions, tracing each definition to its source table and filter conditions, documenting the business rationale for each definition variant, and recommending either a canonical definition to replace the variants or a naming convention that makes the definitional difference explicit on the dashboard surface. The governance session that identifies the revenue definition inconsistency and recommends a canonical “Gross Bookings” and “Net Revenue” labeling convention across all dashboards prevents the next six months of board-meeting confusion. The governance session that confirms all seven uses of “Monthly Active Users” across the company’s dashboards are pulling from the same semantic layer metric definition confirms the consistency that the analytics environment depends on. Both sessions are governance work; only one produces a finding that is easy to describe to the data director.

The metric definition governance function also covers the ongoing review of new metric definitions before they are published to dashboards accessible to business stakeholders. A new dashboard built by the business development team that defines “Partnership Revenue” as gross bookings from accounts flagged with a partner source tag — without accounting for the 22% of partner-attributed accounts that were also touched by a direct sales rep and are already included in the sales team’s revenue metric — will produce an attribution sum that exceeds total company revenue by the amount of the overlap. That calculation error is invisible on the dashboard surface; it requires the data analytics engineer to read the SQL definition, trace the source table, identify the overlap in the partner-source join logic, and recommend a mutual exclusion rule before the dashboard is published. Metric definition governance is the function that makes that review happen before the business development team presents inflated partnership revenue numbers to the board.

Filter interaction design

Filter interaction design is the second most common structural failure in BI dashboards, and it is the failure mode that produces the most user confusion per occurrence because the numbers change in ways the viewer cannot predict or explain. A filter interaction design problem manifests when the viewer applies a filter that they believe narrows the dataset and the metric value moves in a direction that contradicts their mental model of the data. The most common variant: a cascading filter sequence where the order of filter application changes the result. A date range filter applied before a region filter and a region filter applied before a date range filter produce identical-looking filter states on the dashboard surface but different result sets when the underlying SQL applies filter conditions in a different sequence at the query layer.

Cascading filter order advisory covers the review of filter application sequences in the BI tool’s query logic: identifying filter conditions that are applied at the row level before aggregation versus filter conditions that are applied after aggregation as a HAVING clause; advising on the filter order that produces the result the viewer intends rather than the result the query engine produces by default; and documenting the intended filter semantics for complex filter combinations so that the dashboard maintainer can verify the behavior when the dashboard is modified. A dashboard where “Filter by Region” is intended to show only transactions from accounts headquartered in that region but is actually filtering by the billing address on the transaction record — which differs from the account headquarters for multinational accounts — produces a subtly incorrect regional breakdown that the viewer cannot detect from the filter behavior alone.

Cross-filtering scope advisory covers the review of how filtering one visualization in a dashboard affects the other visualizations on the same dashboard canvas. A cross-filter that is scoped correctly in the product sales breakdown chart but is not propagated to the customer retention chart on the same dashboard page produces a state where the viewer believes they are looking at retention for the filtered product segment but are actually looking at retention for all segments. The user who applies a filter to examine the behavior of enterprise accounts in the product chart and reads the adjacent retention chart as enterprise-account retention is making a decision based on data that has not been filtered to the scope they believe they are examining. Cross-filtering scope review identifies mismatches between the viewer’s expected filter propagation and the BI tool’s actual cross-filter implementation, and recommends either expanding the cross-filter scope or adding a visible filter state indicator that makes the scope difference explicit.

Drill-down hierarchy design

Drill-down hierarchy design is the advisory function that ensures the path from a summary metric to the underlying row detail resolves consistently at each level. The denominator problem is the canonical failure: a conversion rate at the summary level is defined as events divided by sessions; the drill-down to the regional level shows conversion rates for each region; the sum of the regional numerators equals the total events; but the sum of the regional denominators does not equal the total sessions because some sessions span multiple regions in the attribution model. The total conversion rate and the regional conversion rates are computed correctly at their respective levels, but they do not reconcile because the denominator definition changes between levels of the hierarchy without any indication on the dashboard surface that this is happening.

Drill-down hierarchy advisory covers the review of the metric definition at each level of the drill-down path: tracing the numerator and denominator definitions at the summary level, the intermediate category level, and the row level to verify they are consistent or explicitly distinguishable; identifying denominators that are scoped at a different grain than the numerator when the hierarchy changes the unit of analysis; and advising on the labeling and documentation changes that would make the scope difference visible to the viewer before they attempt to reconcile summary-level and row-level figures that are computed on different denominators. A well-designed drill-down hierarchy either maintains a consistent denominator definition across all levels or makes the denominator scope explicit at each level with a note that prevents the viewer from attempting to reconcile figures that are intentionally computed on different bases.

The drill-down from a summary revenue tile to the individual transaction rows also requires attention to the aggregation logic that holds at the summary level but breaks at the row level. A revenue metric that applies currency conversion using a monthly average exchange rate at the summary level but displays transaction amounts in the original transaction currency at the row level produces a summary figure that cannot be reconciled to the sum of the row-level amounts by the viewer who does not know the monthly average exchange rate for each currency. The data analytics engineer reviewing a drill-down hierarchy identifies these reconciliation gaps before the dashboard is published and recommends either consistent currency treatment across levels or a visible exchange rate reference that allows the viewer to perform the reconciliation themselves.

KPI framework advisory

A KPI framework is only as useful as the distinction it maintains between leading and lagging indicators, the decomposition tree that attributes variance to its component drivers, and the north-star metric that connects each team’s individual performance signals to the company-level outcome. A KPI framework that conflates leading and lagging indicators gives teams a false sense of forward visibility. A framework that lacks a metric decomposition tree leaves teams unable to diagnose where in the funnel or pricing equation their performance variance is originating. A framework where individual team metrics optimize independently without connecting to a north-star metric produces local optima that are collectively inconsistent with the company’s strategic direction. The KPI framework rollout is the visible event; the ongoing framework advisory that maintains these distinctions as the business evolves and new metrics are added is the retainer function.

Lagging vs. leading indicator selection

Revenue is a lagging indicator. It reflects what the business has already produced, closed, and recognized. By the time revenue is visible in a dashboard, the sales motions, marketing investments, product activations, and retention interventions that determined that revenue outcome have already occurred and cannot be changed for that period. A go-to-market team whose primary KPI is monthly revenue is flying the airplane by looking at where it has been rather than at where it is going. The team can tell you the revenue for the month that just ended with high precision. They cannot tell you, from revenue alone, what revenue will be in three months — because revenue at month three is determined by pipeline coverage ratio, average sales cycle length, and deal close rate at month zero, and none of those forward-looking signals appear in a revenue KPI.

Pipeline coverage ratio is a leading indicator. A pipeline coverage ratio of 3.2x means the team has $3.20 in qualified pipeline for every $1.00 of quota in the forecast period. If the team’s historical close rate is 33%, a 3.2x coverage ratio suggests approximately 106% quota attainment before discounting for slippage. That calculation is available at the start of the forecast period, not the end. A sales team that monitors pipeline coverage ratio weekly — and acts on coverage ratio below 2.5x by accelerating prospecting or pulling forward deals from the next quarter’s pipeline — has the forward visibility to intervene before the revenue shortfall materializes. A sales team that monitors only monthly revenue has no intervention point: by the time the shortfall is visible in the revenue KPI, the forecast period is already closed.

KPI framework advisory covers the audit of each team’s indicator set for the lagging-to-leading balance that gives the team actionable forward visibility: identifying teams that are using lagging indicators as proxies for leading ones (using closed ARR as a weekly performance signal rather than pipeline velocity, stage progression rate, or days since last customer activity); recommending leading indicator substitutes or complements that give the same team the forward visibility that lagging indicators cannot provide; and advising on the leading indicator refresh cadence (weekly pipeline coverage ratio versus monthly revenue) that aligns with the team’s actual decision-making frequency. The advisory also covers the common failure of selecting a leading indicator that is not actually predictive of the lagging outcome — using the number of outbound emails sent as a leading indicator for closed deals when the team’s email sequence has a 0.3% reply rate and a 0.1% conversion rate, making outbound email volume a vanity metric rather than a predictor.

Metric decomposition tree design

Revenue equals price times volume. That decomposition seems trivial until the revenue KPI shows a 12% quarter-over-quarter decline and no one can determine whether the decline is driven by fewer customers (volume), lower average contract value (price), or a mix shift toward smaller customers that lowers the average without either the price schedule or the customer count moving in isolation. A revenue KPI without a decomposition tree is a number that tells the team the outcome has changed without telling them where in the equation to look for the explanation. A revenue KPI with a decomposition tree that separates average contract value from customer count from mix by segment allows the team to identify in three minutes that the 12% revenue decline is entirely explained by a 15% decline in enterprise segment customer count while mid-market revenue is flat and SMB revenue is up 8% — and that the root cause is therefore an enterprise churn problem, not a pricing problem or a mid-market pipeline problem.

Metric decomposition tree design covers the advisory on the multiplicative or additive decomposition structure that makes variance attributable to its component drivers: decomposing revenue into price, volume, and mix so that the combination of the three components equals the total and the variance in each component can be measured independently; decomposing customer acquisition cost into spend divided by acquired customers so that CAC increases can be attributed to increased spend, decreased conversion, or both; decomposing net revenue retention into expansion revenue rate minus contraction rate minus churn rate so that NRR changes can be attributed to upsell motion, downgrade frequency, and customer retention independently. The decomposition design advisory also covers the common failure of decomposing into components that are not multiplicatively consistent — where the product of the decomposed components does not equal the metric total and the team cannot reconcile the decomposition to the top-line number.

The metric decomposition tree also serves as the diagnosis tool for variance in the KPI framework. When the north-star metric moves, the decomposition tree is the instrument that attributes the movement to its contributing drivers without requiring an ad hoc analysis from the data team. A well-designed decomposition tree makes the variance attribution available in the dashboard itself: the revenue tile decomposes to an ACV tile and a customer count tile; the ACV tile decomposes to an enterprise ACV tile, a mid-market ACV tile, and an SMB ACV tile; the customer count tile decomposes to new customer count, churned customer count, and net new. The executive who sees revenue decline can follow the decomposition path to the contributing driver in three clicks rather than waiting two days for the data team to run a custom analysis.

North-star metric alignment

Each team optimizing its own metric without connecting to the north-star metric is the organizational failure that a KPI framework is designed to prevent. The growth team optimizes for new user signups. The product team optimizes for feature adoption rate. The customer success team optimizes for renewal rate. The go-to-market team optimizes for new ARR. Each team is meeting its individual target, but the company’s north-star metric — net revenue retention, or engaged accounts, or time-to-value for new customers — is flat or declining because the individual team optimizations are not compositionally consistent with the north-star outcome. The growth team is acquiring users who do not activate. The product team is shipping features that are adopted by users who are not in the renewing segment. The customer success team is retaining customers who are not expanding. The go-to-market team is closing deals that churn at twelve months.

North-star metric alignment advisory covers the review of each team’s KPI set against the company’s north-star metric: identifying team metrics whose local optimization is inconsistent with or actively opposed to the north-star direction (a growth team whose signup volume KPI rewards acquiring unqualified users that the customer success team must then churn at month three); recommending metric modifications that connect the team metric to the north-star outcome (replacing signup count with activated user count, or with users who reach the product’s core value moment within the first seven days, as a growth KPI that is predictive of the north-star); and advising on the OKR structure that makes the connection between team-level key results and the north-star metric explicit in the company’s performance framework rather than assumed.

The north-star metric alignment review also identifies the common case where the north-star metric itself is misspecified. A company that adopts daily active users as its north-star metric for a B2B workflow product is selecting a north-star borrowed from consumer social media that does not map to the value delivery mechanism of a B2B tool, where the core value is delivered through weekly or monthly workflow outputs rather than daily engagement patterns. The data analytics engineer advising on north-star metric selection traces the proposed metric to the company’s actual retention and expansion drivers — not through benchmark comparison but through the historical correlation between the proposed metric and the outcomes the company is trying to maximize — and recommends a north-star metric whose maximization is mechanistically connected to the revenue and retention outcomes the company is actually managing.

Data visualization advisory

Data visualization advisory is the discipline that ensures each chart type, color encoding, and annotation design serves the analytical question rather than the visual presentation preference of the dashboard author. The failure modes in data visualization are not primarily aesthetic — they are analytical: chart types that require the viewer to perform visual arithmetic that the chart design could perform instead, color encodings that suggest a distinction the data does not contain or suppress a distinction the data does contain, and the absence of annotations that would explain a visible feature of the data so that the viewer does not diagnose a level shift as a trend or attribute a seasonal pattern to a product decision.

Chart type selection

A pie chart with eleven slices does not communicate anything useful. The human visual system is not equipped to accurately perceive angular area for slices below roughly 5% of the total, and eleven slices on a revenue breakdown chart means that at least six of the slices are below the perception threshold where the viewer can meaningfully compare them. The viewer spends eleven seconds matching slice colors to legend entries and leaves the chart without being able to articulate which segment is the third largest, how much the bottom five segments collectively represent, or whether the largest segment is growing or shrinking. A horizontal bar chart of the same eleven segments, sorted by revenue descending, communicates the ranking instantly, allows accurate comparison of adjacent bars through bar length rather than angular area, and makes the total contribution of the bottom four segments visible in the two seconds it takes to read that they add up to 6% combined. The insight that was invisible in the pie chart — that the bottom four regions are collectively immaterial to the revenue total and are consuming reporting space and executive attention disproportionate to their economic contribution — is the first thing the viewer takes away from the bar chart.

Chart type selection advisory covers the review of each chart type against the analytical question it is answering: evaluating whether a pie or donut chart should be a bar chart (part-to-whole with more than four or five components, or where ranking and comparison matter); whether a line chart should be a bar chart (time-series data where the comparison is between discrete periods rather than continuous trend); whether a scatter plot should be a connected dot plot (when the x-axis is categorical rather than continuous and the dots represent ordered comparisons within a category); and whether a table should be a chart (when the viewer is being asked to detect a pattern that is visible in a chart and invisible in a column of numbers). The advisory session does not evaluate the chart for visual appeal but for the time it takes the viewer to extract the finding the chart is designed to communicate, and whether a different chart type would reduce that extraction time from minutes to seconds.

The most consequential chart type selection failures appear in executive dashboards where the analytical finding is critical but the chart design requires the viewer to work to extract it. A funnel chart displayed as a stacked bar chart rather than a waterfall or sequential funnel chart requires the viewer to visually subtract each stage from the previous stage to identify where conversion is dropping — mental arithmetic that takes thirty seconds per stage and is error-prone for five or more funnel stages. A time-series chart showing daily data with a seven-day rolling average overlaid is immediately useful; a chart showing only the daily data without the rolling average is dominated by day-of-week seasonality that masks the underlying trend the viewer is trying to detect. Chart type selection advisory identifies these mismatches and recommends the chart design that delivers the finding to the viewer rather than requiring the viewer to work for it.

Color encoding for categorical and quantitative variables

Color encoding problems in BI dashboards fall into two failure modes that pull in opposite directions. The first is using a sequential or diverging color scale on a categorical variable, which implies a continuous ordering or a meaningful midpoint in a variable that has neither. A regional sales breakdown where North America is the darkest blue, EMEA is a medium blue, APAC is a light blue, and LATAM is nearly white implies that North America and LATAM are at opposite ends of some continuous dimension when the reality is that the color reflects sales volume and the implied ordering is an artifact of the color scale rather than a property of the categories. The viewer reads the color gradient as a qualitative ranking by importance or priority rather than as a quantitative intensity encoding for a single variable, and builds a mental model of the regions that is shaped by the color assignment rather than the data.

The second failure mode is using a diverging color scale on a metric that has no natural zero point or meaningful midpoint. A diverging red-to-green scale on a customer satisfaction score that ranges from 3.8 to 4.7 implies that scores below the midpoint (4.25) are negative and scores above the midpoint are positive, when the actual business context is that all scores in the 3.8–4.7 range are acceptable and the scale midpoint is an artifact of the color scheme rather than a business-defined acceptable threshold. The correct encoding is a sequential scale from light to dark (all values are positive, and darker means more) or a sequential scale with a threshold color break at the minimum acceptable score (distinguishing acceptable from unacceptable without implying a continuous good-to-bad dimension where none exists). A diverging scale is appropriate when the data has a meaningful zero point or a policy-defined threshold that distinguishes acceptable from unacceptable, and the color encoding should reflect the policy threshold rather than the statistical midpoint of the observed range.

Color encoding advisory also covers the use of color to distinguish categories in multi-series charts, where the failure mode is using colors that are perceptually similar for categories that require clear distinction (two shades of blue for two revenue streams that viewers need to track independently) or using more than eight colors on a single chart, which exceeds the number of distinct hues the viewer can reliably match to legend entries without consciously checking. The data analytics engineer reviewing a multi-series line chart with nine series advises on reducing the series to the four or five most analytically relevant, grouping the remaining series as “Other,” or redesigning the chart as a small multiples layout that gives each series its own panel and eliminates the color discrimination problem entirely.

Annotation and callout design

Annotation and callout design is the difference between a time-series chart that shows a visible change in a metric and a time-series chart that explains what caused the visible change. A DAU chart that shows a step-change increase on a specific date — where DAU rises from 42,000 to 58,000 between one week and the next and stays at the higher level — will be read by every viewer who sees it for the first time as an unexplained trend change that requires investigation. If the step change is explained by the launch of a mobile app feature that brought a new user cohort into the daily engagement surface, an annotation at the launch date — “Mobile feature launch, March 14” — converts an apparent data anomaly into a documented causal event that any viewer can interpret correctly without a separate analysis. Without the annotation, every new viewer who encounters the chart must either investigate the step change themselves or form an incorrect mental model of the metric’s behavior.

The critical annotation use case for data quality monitoring is marking the date of a data pipeline event or collection methodology change that caused a visible change in the metric. A checkout completion rate chart that shows a 34% day-over-day drop on a specific date will be read by the product team as a potential product regression requiring immediate investigation. If the drop is caused by a tracking tag misconfiguration that stopped firing the checkout_complete event for 34% of completions — a data collection artifact rather than a product behavior change — the product team may spend two to three days investigating a product regression that does not exist before the data team identifies the tracking tag as the source. An annotation at the drop date — “Tracking tag misconfiguration identified and resolved, data gap covers Nov 3–Nov 5” — prevents the misdiagnosis entirely and signals to every viewer that the visible feature is a data artifact rather than a product signal.

Annotation design advisory covers the review of time-series charts for visible features that require explanation: identifying step changes, level shifts, trend breaks, and data gaps that are attributable to specific events (product launches, policy changes, data pipeline issues, collection methodology changes, external market events); recommending the annotation content (event label, date, brief description), placement (vertical reference line at the event date versus shaded region for a multi-day event or data gap), and priority (annotating the three or four most analytically significant events on a chart rather than annotating every event that touches the metric, which produces a chart with sixteen annotations that the viewer stops reading). The annotation policy also covers who has the authority to add annotations to shared dashboards — a data governance question that determines whether the annotation layer reflects the data team’s validated interpretation or accumulates contradictory notes from anyone with dashboard edit access.

Self-service analytics advisory

Self-service analytics — the ability for business users to query, filter, and explore data without submitting requests to the data team — is the operational goal of most enterprise analytics environments. The infrastructure that makes self-service analytics safe and productive consists of three governance layers: a data catalog that tells business users which datasets are safe to query and what the fields mean, a row-level security design that ensures each user sees only the data their role and organizational position authorize them to see, and a certified dataset governance process that maintains the distinction between trusted, curated datasets and raw tables that require data team expertise to use correctly. Without these three governance layers, self-service analytics produces a situation where business users are querying raw tables with ambiguous field names, joining datasets at the wrong grain, and publishing dashboards based on uncertified data that produces incorrect metric calculations.

Data catalog documentation

A data catalog that lists every table in the data warehouse without distinguishing certified datasets from raw tables, without documenting what the fields in each table mean, and without identifying the join keys that connect related tables gives the business user a map of the warehouse without a legend. The business user who wants to analyze customer retention will find ten tables with “customer” in the name, three tables with “subscription” in the name, and no documentation indicating which table is the canonical source for customer retention analysis, what the difference is between subscription_id and contract_id, or which date field represents the subscription start date versus the billing period start date. The typical response is to either ask the data team for help (negating the self-service objective) or to pick a table, join it to another table using a field name that seems like it should be the join key, and produce a dashboard based on a join that produces a fanout or a many-to-one mismatch.

Data catalog documentation advisory covers the design of the catalog documentation structure: recommending the documentation fields that provide the information business users need to select the right dataset and use it correctly (dataset description, owner, refresh cadence, intended use case, known limitations, certified status, related datasets); advising on metric definition documentation that captures the business logic embedded in the metric calculation rather than the SQL expression (defining “Monthly Active Users” as “distinct users with at least one core feature interaction in the calendar month” rather than documenting the SQL as “COUNT(DISTINCT user_id) WHERE event_type IN ('core_feature_click', 'core_feature_submit')”); and advising on join key documentation that identifies the relationship between related datasets (the join between the accounts table and the subscriptions table is one-to-many on account_id, with one account able to have multiple concurrent subscriptions, so joining without aggregating the subscription table first will produce one row per subscription rather than one row per account).

The data catalog documentation advisory also covers the ongoing documentation maintenance process that prevents catalog entries from becoming outdated as the underlying datasets evolve. A catalog entry for the daily_sessions table that accurately describes the table structure and field definitions at the time of initial documentation but is not updated when the ingestion pipeline is modified to include a new event type, or when the session attribution methodology is changed to use a 30-minute rather than a 45-minute inactivity timeout, becomes a documentation liability: business users are reading documentation that no longer accurately describes the table they are querying. The advisory covers both the initial documentation design and the documentation update process that keeps the catalog accurate as the analytics environment evolves.

Row-level security design

Row-level security is the mechanism that ensures each business user’s self-service queries return only the rows their role and organizational position authorize them to see. A sales manager whose BI tool access is governed by row-level security sees only their team’s pipeline in the pipeline dashboard, not the full company pipeline. A regional director sees the pipeline for their region. The VP of Sales sees the full company pipeline. The same dashboard, the same query, the same metric definitions — but the result set filtered at the row level by the viewer’s identity and organizational role. Row-level security that is designed and implemented correctly is invisible to the authorized viewer: they see exactly the data their role entitles them to and have no visibility into the existence of rows they are not authorized to see.

Row-level security design advisory covers the review of the security model design: identifying the organizational dimensions that require row-level filtering (sales territory, business unit, legal entity, cost center, product line, customer segment); advising on the security model architecture that maps each dimension to the authorized viewer population (a territory-based security model that joins user identity to a territory assignment table that defines each user’s authorized territory set); and reviewing the implementation for the common failure modes in row-level security design. The most common failure mode is a security model that enforces row-level security on the primary fact table but allows the viewer to join to a reference table that exposes restricted rows through a dimension attribute — a sales manager who cannot see another team’s pipeline rows directly but can see the account names of all pipeline accounts in a dropdown filter that is populated from the accounts table without the row-level security filter applied.

The row-level security design advisory also covers the performance implications of row-level security implementation. A row-level security model implemented as a subquery that re-evaluates the authorization logic for every query against the fact table — joining the user identity to the territory assignment table and then filtering the fact table on the matching territory set, for every query, for every user, without caching — may be correct from a security design perspective but produce query times that are three to five times longer than the same query without row-level security. The advisory covers the security model design that maintains both the access control correctness and the query performance the business users need for self-service analytics to be faster than asking the data team for a custom analysis.

Certified dataset governance

The certified dataset tier is the mechanism that maintains the distinction between the datasets business users should use for self-service analytics and the raw tables, staging tables, and intermediate transformation outputs that require data team expertise to use correctly. A certified dataset has been reviewed for documentation completeness, field definition accuracy, join key correctness, known limitations disclosure, and metric calculation logic validation. A raw table has none of those guarantees. The certified tier is only meaningful if there is a defined process for what review a dataset must pass before it can be certified, who is authorized to certify it, and how certified status is revoked when the underlying dataset changes in a way that invalidates the certification criteria.

Certified dataset governance advisory covers the design of the certification process: recommending the review checklist that a dataset must pass before certification (documentation fields completed, field definitions accurate and current, join keys documented and validated, known limitations explicitly disclosed, at least one data quality check configured for the dataset, business owner identified and confirmed); advising on the certification authority structure (whether certification is approved by a data governance committee, a designated data steward for each business domain, or the data analytics engineer on retainer performing a pre-certification review); and advising on the certification maintenance process (who receives notification when a certified dataset’s underlying table schema changes, and what review the dataset must pass before the certification is maintained or revoked).

The governance advisory also covers the policy for dataset publication access — who has the authority to publish a dataset to the certified tier, and what prevents a business user with dashboard edit access from marking their own ad hoc dataset as certified without the review process. In most BI tool implementations, the certified tier is enforced by a permission level that restricts the “certify” action to a specific user role; the governance advisory covers the correct permission configuration for the BI tool in use and the process for handling certification requests that come from outside the authorized certification group. The certified tier that any user can populate is not a governance tier; it is a visual label that has accumulated data team trust without the data team’s involvement.

Data quality monitoring advisory

Data quality monitoring for BI-facing tables is the detection layer that catches data problems before they propagate to dashboards and produce the executive meeting where someone asks why the revenue figure is different from last week and the answer is that six hours of data is missing from the daily sessions table due to a timezone offset bug in the ingestion pipeline. The three functions of data quality monitoring advisory in a BI context are: upstream data quality checks that validate the BI-facing tables before dashboards are refreshed against them, anomaly detection for key metric time series that distinguishes a real product or business event from a data collection artifact, and data freshness alerting that ensures the dashboards being used in time-sensitive reporting are reflecting current data rather than data from a pipeline run that completed eight hours late.

Upstream data quality checks for BI-facing tables

The daily_sessions table missing six hours of data from a timezone offset bug is a canonical upstream data quality failure. The ingestion pipeline is collecting session events from mobile clients across multiple timezones. The Pacific-region mobile client is sending event timestamps in local time without a timezone offset indicator. The ingestion pipeline is interpreting those timestamps as UTC and storing them accordingly. Events that occurred between 4pm and midnight Pacific time (which is midnight to 8am UTC the following day) are being bucketed into the following UTC day, creating a six-hour gap in Pacific-region session data for the calendar day and a six-hour inflated session count for the following calendar day. The daily_sessions table appears to be refreshing successfully; the pipeline has no errors; the table has rows for every day. The only visible symptom is a small dip in the Pacific-region DAU chart that looks like it might be a product regression, a sampling artifact, or a data collection issue — and takes three days to diagnose to the timezone offset bug because no upstream data quality check is asserting that the hourly session count distribution for Pacific-region events matches the expected distribution for that timezone.

Upstream data quality checks for BI-facing tables cover the advisory on the data quality assertions that should be configured to run before the BI tool’s scheduled dashboard refresh: row count assertions that flag the absence of rows for time periods that should have data (no rows for a calendar day in a table that should have daily data is a data quality failure, not a business event); distribution assertions that flag unexpected changes in the breakdown of a categorical variable (an 80% drop in Pacific-region session rows on a day with a normal East Coast session volume is not a product event; it is a pipeline artifact); completeness assertions that flag null rates for fields that should be non-null (a session_duration_seconds field that is null for 34% of rows on a specific date when it is null for less than 1% of rows on surrounding dates is a data collection failure, not user behavior); and cross-table consistency assertions that verify that the row counts and aggregate totals in BI-facing tables are consistent with the raw source tables they are derived from.

The upstream data quality advisory also covers the configuration of check timing relative to the BI tool’s dashboard refresh schedule. A data quality check that runs at 6am for a pipeline that delivers data at 7am provides no protection for the 7am dashboard refresh; it will flag the absence of data as a quality failure before the data has arrived. A check that is scheduled at 8am for a pipeline that is supposed to deliver by 8am but is consistently delivering at 11am will silently pass at 8am and allow the BI dashboards to refresh against stale data without triggering any alert. The check timing advisory covers the scheduling of upstream quality checks in the actual delivery window of the pipeline rather than the intended delivery window, so that the checks are validating the data that will actually be used in the next dashboard refresh rather than checking for data that has not yet arrived.

Anomaly detection for key metric time series

A 34% day-over-day drop in checkout completion rate is either a serious product regression that requires an all-hands engineering investigation or a tracking tag misconfiguration that temporarily stopped recording checkout completion events. The product team’s first response is to assume the former and pull the engineering team off their current sprint to investigate. If the monitoring system has anomaly detection configured for the checkout completion rate that checks the event volume behind the metric — not just the metric value but the count of checkout_complete events that produced the metric denominator — the investigation starts in a different place. A 34% drop in the checkout completion rate accompanied by a 34% drop in checkout_complete event volume with no change in checkout_initiated event volume is a tracking tag misconfiguration: the completions are happening, but the events are not being recorded. A 34% drop in the checkout completion rate accompanied by no change in event volume is a product regression: the events are being recorded, but fewer of them are completions.

Anomaly detection advisory covers the design of the detection layer for key business metrics: recommending the metric and sub-metric monitoring configuration that allows the investigation team to distinguish business events from data collection artifacts without a full data forensics investigation; advising on the detection thresholds for each metric that balance sensitivity (catching real anomalies quickly) against specificity (not producing false alerts that the team learns to ignore); and recommending the alert routing design that sends metric anomaly alerts to the right audience (a 20% drop in DAU is a data team alert before it is a product team alert; the data team should confirm the anomaly is not a collection artifact before the product team begins investigating a potential product regression). The anomaly detection design also covers the seasonality adjustment that prevents day-of-week patterns (lower DAU on weekends) from triggering false alerts on Monday when Sunday’s DAU is lower than Saturday’s by the normal weekend decay amount.

Anomaly detection for BI contexts also requires sensitivity to the pipeline delivery patterns that produce apparent metric anomalies. A key metric that drops sharply at 7am and recovers to normal levels by 9am may not be a product regression or a tracking failure; it may be the morning pipeline delivery window where the dashboard is showing data that is 12 hours old rather than 1 hour old, and the apparent drop is the absence of the most recent 12 hours of activity rather than a decline in the underlying rate. Distinguishing a data freshness anomaly from a metric anomaly requires the anomaly detection system to be aware of the pipeline delivery schedule and to suppress metric anomaly alerts during known delivery windows — or alternatively to classify the apparent metric drop as a freshness event rather than a metric event and route the alert accordingly.

Data freshness alerting

The executive review that starts at 9am on Monday and is looking at Friday’s data is a scenario that occurs in every analytics environment that has a pipeline delivery SLA without a freshness verification check at the dashboard level. The pipeline that was supposed to deliver data by 8am was delivering at 11am for the past six weeks because a configuration change to the pipeline scheduling system silently extended the execution window from 4 hours to 8 hours without anyone noticing — the pipeline was still completing successfully, all quality checks were passing, and the data was accurate when it arrived. The symptom was not visible in any monitoring system; it was only visible to the executive who noticed on Tuesday morning that the “as of” timestamp on the Monday morning dashboard said “Friday, 11:47pm” rather than “Monday, 7:58am.” The Monday 9am executive review had been conducted on Friday’s data for six weeks without anyone realizing it.

Data freshness alerting advisory covers the design of the freshness monitoring layer that detects pipeline delivery delays before business-critical reporting windows: recommending a freshness check for each BI-facing table that asserts the maximum timestamp of the most recent data is within the expected freshness window (the daily_sessions table should have data through yesterday 11:59pm by 8:00am UTC; an alert should fire if the maximum event date in the table is more than 26 hours old at 8:00am UTC); advising on the alert routing that sends freshness failures to the data team before the reporting window rather than during the executive review; and recommending the freshness timestamp display that makes the data currency visible to dashboard viewers directly in the dashboard interface (an “as of” timestamp on each dashboard tile, or a dashboard-level freshness indicator that shows the oldest data timestamp across all tiles on the page).

The freshness alerting advisory also covers the priority tiering of freshness alerts by the business impact of stale data for each metric. A freshness failure on the real-time fraud detection dashboard has a different urgency profile than a freshness failure on the monthly executive KPI dashboard. The freshness alert design should reflect the business impact of each pipeline’s delivery delay: a real-time operations dashboard with a 15-minute data freshness requirement should trigger an alert after 20 minutes of delay; an executive summary dashboard with a 24-hour reporting cadence should trigger an alert after 2 hours of delay, not 20 minutes. The alert thresholds and routing should be calibrated to the business consequence of staleness for each reporting surface rather than applied uniformly across all pipelines at a single default threshold.

Frequently asked questions

What does a data analytics engineer on retainer typically do?

A data analytics engineer or BI engineer on monthly retainer typically provides BI dashboard design advisory (metric definition governance, filter interaction design review, and drill-down hierarchy design), KPI framework advisory (lagging vs. leading indicator selection, metric decomposition tree design, and north-star metric alignment), data visualization advisory (chart type selection, color encoding review, and annotation design for level shifts and policy changes), self-service analytics advisory (data catalog documentation design, row-level security design, and certified dataset governance), and data quality monitoring advisory (upstream data quality checks for BI-facing tables, anomaly detection for key metric time series, and data freshness alerting). The BI dashboard release and the KPI framework rollout are the visible milestones; the continuous advisory between those events is the ongoing retainer function.

What data analytics work is most commonly underlogged?

The five most consistently underlogged categories are: metric definition reviews that confirmed consistency across dashboards (tracing every dashboard tile labeled “Revenue” to its underlying SQL definition and confirming all seven instances are pulling from the same source table with the same refund treatment required the same review work as finding the 23% divergence, whether or not a divergence was found); data visualization reviews that approved proposed charts (evaluating chart type, color encoding, and axis scaling and concluding the chart is analytically sound required the same evaluation as finding a truncated y-axis); KPI framework advisory sessions that confirmed a sound leading indicator balance; data quality monitoring reviews with no anomalies detected (reviewing anomaly detection results and confirming all metrics are within normal range still required performing the review); and certified dataset governance sessions that approved a dataset for certification (reviewing documentation completeness, join key accuracy, and data quality check configuration and approving certification required the same governance work as a review that identified missing documentation before approving).

What should a data analytics engineer retainer agreement include?

Data analytics engineer retainer agreements should address: BI tool and data warehouse access (the consultant needs the SQL layer, the semantic layer, the data catalog, and the row-level security configuration to conduct meaningful advisory); scope boundary between advisory and implementation (building dashboards, writing SQL metric definitions, and configuring row-level security policies are implementation activities requiring separate authorization); data access and confidentiality requirements for production analytical data (revenue, compensation, and customer-level data frequently appear in BI environments; define the data classification levels the consultant is authorized to access); certification authority for datasets and metrics (define whether the consultant has approval authority for certified tier publication or feeds a governance committee review); and a shared work log visible to data leadership that documents the ongoing BI advisory, metric governance, data visualization reviews, and data quality monitoring work the retainer produces.

What are typical retainer rates for data analytics engineers and BI engineers?

Data analytics engineer and BI consultant retainer rates vary with experience level and analytics environment complexity. Mid-level BI engineers (3–6 years, one BI platform specialism): $95–$160/hr, with monthly retainers of 10–20 hours ($1,100–$3,200/mo) covering dashboard design review, metric governance, and data quality monitoring. Senior data analytics engineers (6–10 years, multi-tool or enterprise analytics environments): $150–$240/hr, with monthly retainers of 15–30 hours ($2,800–$7,000/mo) covering BI architecture, metric governance, self-service analytics design, and data quality monitoring strategy. Principal BI architects and analytics platform leads (10+ years, enterprise analytics platform design): $200–$380/hr, with monthly retainers of 20–40 hours ($5,000–$14,000/mo) covering analytics platform strategy, metric governance design, and BI practice leadership. Advisory-only retainers covering dashboard design review, metric governance, KPI framework advisory, and data quality monitoring design are typically priced at a lower rate than implementation retainers that include hands-on dashboard building, SQL writing, or BI tool configuration.

How should data analytics engineer retainer hours be logged?

Log entries should capture the analytics advisory function (BI dashboard design, KPI framework advisory, data visualization review, self-service analytics governance, data quality monitoring), the specific dashboard, metric, or dataset reviewed, the activity performed, and the recommendation or finding. Effective format: [analytics function] + [artifact reviewed] + [activity performed] + [recommendation or finding]. Example: “BI dashboard design advisory — Revenue metric definition governance: traced the Revenue metric in the executive summary dashboard and the Revenue metric in the sales pipeline dashboard to their underlying SQL definitions; identified that the executive summary explore sums gross_booking_amount from the bookings table while the sales pipeline explore sums net_revenue from recognized_revenue which applies an 18% refund deduction; definitions diverge by 23% on any period with representative refund volume; recommended creating a canonical Gross Bookings and Net Revenue metric in the LookML semantic layer and retiring both divergent explores: 3.5 hours.” Log every session, including metric definition reviews that confirmed consistency and data quality reviews that found no anomalies. The review that confirmed all seven Revenue definitions are consistent required the same review work as finding the divergence; logging only sessions that produced a finding systematically understates the governance volume and misrepresents the retainer’s value.

Make your data analytics retainer work visible to clients. HourTab turns a time-tracker CSV into a public retainer-hours URL your client can bookmark — no login required, no portal to maintain. Your data director or CDO sees exactly what the monthly advisory produced: the metric governance sessions, the KPI framework reviews, the data quality monitoring work, the visualization advisory. See how HourTab works →