How to Verify an AI-Generated Data Answer Before It Reaches the Board

The SQL is the first thing to check, and it is not enough on its own. If you are asking how to check whether an AI-generated number is correct, the answer is a short, repeatable pass. Before a figure reaches a board deck, confirm eight things: the interpretation matches your intent, the metric matches your business definition, the right tables were read, joins did not multiply rows, filters excluded the right rows, time zones and currencies line up, the result passes a sanity check, and any warnings were read.

Who signs off on the number

Executives who present numbers they did not compute themselves and want a defensible answer to “where did this figure come from?”. Finance teams who reconcile AI-generated metrics against the ledger before anything goes external. Analysts who are asked to sign off on a conversational-analytics answer and need a checklist that is faster than redoing the work from scratch — but no less rigorous.

Why reading the query is necessary but not sufficient

A verifiable answer is one that ships with its evidence — the exact query that ran, the tables it read, and what came back. That property is the foundation, and what verifiable conversational analytics means makes the case for it. But evidence only helps if you know what to check. A query can be syntactically perfect, run without error, and still compute the wrong thing: it can read an ambiguous question differently than you meant, use a defensible-but-wrong metric definition, or quietly multiply rows through a join. How text becomes SQL explains why the same question can honestly produce different queries; this guide is about deciding whether the query you got is the one your board number should stand on.

How to verify an AI-generated SQL answer in eight steps

Work through these in order. Steps one through six read the query and its sources; steps seven and eight read the result and the run record.

Confirm the interpretation matches your intent. Read your original question next to the executed query and check that they describe the same measurement. “Revenue last quarter” can mean the previous calendar quarter or the trailing ninety days, and “top customers” can mean by revenue or by order count — the query commits to one reading, and it must be yours.
Check the metric definition. Decide whether the figure should be gross or net, and whether returns, refunds, discounts, taxes, and shipping are in or out — then check that the query’s arithmetic and filters agree. A technically correct query can still measure the wrong thing here.
Check the data scope. Look at which tables the query read and confirm they are the canonical ones, not a staging copy or an abandoned _v2. Then check completeness and freshness: whether the period is fully loaded, and whether the current month is partial and should be excluded or labeled.
Check the joins for fan-out. Joining an order-level amount to line items multiplies it once per line, silently inflating any sum built on top. Confirm that amounts are summed at the grain where they live, and that row counts do not balloon across a join.
Check the filters. Look for status filters (completed vs. cancelled vs. refunded), test rows, internal accounts, and deleted records. Whether these are included or excluded is a business decision, and the query’s WHERE clause is where that decision is visible.
Check time zone and currency. A day boundary in UTC is not a day boundary in your reporting time zone, and month-end figures can shift when the cutoff moves. If the data spans currencies, confirm amounts were converted before being summed rather than added raw.
Sanity-check the execution result. Compare the row count and the magnitude of the answer against expectation: a customer table you know holds thousands of rows should not return twelve, and a quarterly revenue figure should be in the neighborhood of previous quarters. A result that is wildly off is a prompt to revisit steps three through six, not a rounding quirk.
Read the exceptions and warnings. Check whether the result was truncated by an automatic row limit, whether the query failed and was revised and retried, and whether any step of the run flagged partial data. If the row count equals the row limit, treat the result as truncated: re-ask the question as an explicit aggregate so the total is computed in the database rather than over a capped slice.

A worked example: the refund trap

The example below uses a fictional retailer’s demo schema — customers, orders, order_items — and every figure in it is invented for illustration, not output from a live system.

The question is: “What was our revenue in Q2 2026?” Here are two queries a model could reasonably generate. Both are valid SQL, both are reads, both run without error — and they answer different questions.

Variant A sums every order in the quarter, whatever its status:

SELECT SUM(oi.quantity * oi.unit_price) AS revenue_q2
FROM orders AS o
JOIN order_items AS oi ON oi.order_id = o.order_id
WHERE o.ordered_at >= DATE '2026-04-01'
  AND o.ordered_at < DATE '2026-07-01';

Variant B counts only completed orders, excluding refunded and cancelled ones:

SELECT SUM(oi.quantity * oi.unit_price) AS revenue_q2
FROM orders AS o
JOIN order_items AS oi ON oi.order_id = o.order_id
WHERE o.status = 'completed'
  AND o.ordered_at >= DATE '2026-04-01'
  AND o.ordered_at < DATE '2026-07-01';

On the demo data, Variant A returns 1,284,650 and Variant B returns 1,209,410 — a gap of 75,240, which is exactly the refunded and cancelled orders. Neither query is broken; the difference is definitional. If finance defines revenue as completed orders net of refunds and the deck carries Variant A’s figure, the number is wrong in a way no validator can catch — only step two of the workflow catches it, and only if someone runs it.

The same schema, with a join that inflates the total

Step four is harder to spot than step two, because the query still reads correctly. Variant C asks the same question, but sums a column that lives on orders rather than on order_items:

SELECT SUM(o.order_total) AS revenue_q2
FROM orders AS o
JOIN order_items AS oi ON oi.order_id = o.order_id
WHERE o.status = 'completed'
  AND o.ordered_at >= DATE '2026-04-01'
  AND o.ordered_at < DATE '2026-07-01';

The join produces one row per item, so an order with four line items contributes its total four times. On the demo data Variant C returns 4,031,180 — more than triple Variant B — and the row count tells you why: the aggregate is built from 7,190 item rows against 2,180 orders, an average of 3.3 line items each. Whenever a sum is taken over a column at a coarser grain than the join produces, the figure inflates by roughly that average.

Two checks catch it. Compare the row count against the grain you expect: 7,190 rows for 2,180 orders means every order was counted several times. And sanity-check the magnitude — a quarter that triples year on year is a query bug far more often than it is a business result.

Red flags that should stop a number at the door

Red flag	What it suggests
An answer with no visible query	You cannot run any of steps one through six; the figure is unverifiable by construction
A row count wildly off expectation	Wrong table, over-aggressive filter, or a join that dropped or multiplied rows
A metric name with no definition	”Revenue” or “active users” without gross/net, statuses, or windows spelled out — step two never happened
A suspiciously round number	Possible truncation by a row limit, a capped result, or an aggregate over partial data
A join that multiplies rows	Order-level amounts summed per line item — the classic fan-out inflation from step four

Any one of these is a reason to pause; two or more is a reason to rerun the question and re-verify from the top.

Where the evidence comes from

The workflow looks long, but with the right evidence attached to the answer, most steps are a short read rather than an investigation.

Evidence shipped with the answer	Steps it answers
The exact query that ran	1, 2, 4, 5, 6 — interpretation, definition, joins, filters, and time handling are all visible in the SQL text
The source tables	3 — you see immediately whether the canonical tables were read
The row count	4 and 7 — fan-out shows up as an implausible count; so does an over-filtered result
The runtime	7 — a supporting signal that the query scanned roughly the data you expected
The recorded run, step by step	8 — retries, row-limit truncation, and warnings live in the run record, plus the ability to reopen and re-check any of it later

The pattern is worth naming: steps one through seven are answerable in a single sitting from four pieces of evidence, provided the tool ships them with every answer instead of on request.

How much verification a number needs

Not every number deserves the same scrutiny. For an operational question — “how many orders came in yesterday?” — a glance at the query and a magnitude check (steps one and seven) is a proportionate review. For a number headed into a board deck, an investor update, or an external filing, run all eight steps, and have a second person review the query and evidence independently before the figure ships. The escalation judgment — which numbers are high-stakes, what the canonical definitions are, when a figure needs a second pair of eyes — stays with people. Verification tooling shortens the checking; it does not replace the analysts and finance owners who decide what correct means.

Reading the evidence panel

Datarelix puts the inputs for the workflow above on every answer by default: the query that ran, the tables it read, the row count, and the runtime — what each of the four tells you. Every run is recorded step by step and reopenable from history, so step eight and any later re-check work from the record rather than from memory. When a verified answer becomes a dashboard, it refreshes from the same query logic, so the definition you verified is the definition that keeps running. The mechanics of reading a result and its evidence are in the queries guide.

What the eight steps do not catch

This workflow verifies what was measured, not whether the underlying data is right — if refunds land in the warehouse three days late, a perfectly verified query still reports a number that will drift. It also cannot supply your definitions: whether revenue is net of refunds is a finance decision, and the workflow only checks that the query agrees with a definition someone has already made. The second-reviewer step needs one practical caveat: a reviewer works from the query and evidence you hand them, pasted into review notes or a ticket, because accounts are isolated (account isolation). And for genuinely novel, high-stakes analysis with heavy modeling assumptions, verification of a single query is not enough; that work belongs with an analyst end to end, with conversational tools serving as one input.

About the worked example

Product statements in this guide describe Datarelix as shipped; the security page is the canonical statement of the validation, credential-handling, isolation, and run-recording boundaries referenced here. All SQL examples and every numeric figure use a fictional retailer’s demo schema (customers, orders, order_items) and are illustrative inventions, not output captured from a live system. No external citations, benchmarks, or customer examples are used.

Turn this into a standing habit

Turn the eight steps into a standing evaluation habit: the the AI analytics security checklist extends this per-answer workflow into the questions to ask of any tool before you connect it to a real database.