A brass automaton analyst rereads an enormous glowing ledger while a long queue of identical question cards waits and a small fare meter runs on the desk ← Notebook

AI · Data · Enterprise Architecture

Stop Paying AI to Answer the Same Question Twice

Conversational analytics is moving from the demo to the Monday meeting. The economics change when the tenth person asks the same question.

Drew BruceOctober 202610 minute read

There is a moment in every conversational analytics demonstration when somebody types ‘What was gross revenue by region last week?’ into a box, a chart appears, and the room goes quiet in the way rooms do when they are mentally reallocating next year's reporting budget.

It is a genuinely good moment. Business users asking a data warehouse questions in plain English, without waiting three weeks for a report request to emerge from a queue, is a real improvement.

The question nobody asks during the demo is what happens when four hundred people ask the same thing every Monday.

That is not a demo question. It is an operating question. And it is arriving faster than most of us have planned for.

The demo has left the building

Google made Conversational Analytics in BigQuery generally available on 1 July 2026. The agent grounds its answers in Knowledge Catalog (business glossaries, profile scans and what Google calls context bundles), along with your own instructions and a set of verified queries. It can be published into Gemini Enterprise, Data Studio or your own applications through an API.

In other words, it is no longer something you show the executive team. It is something you give to everyone.

The pricing has also arrived. Google's Data Cloud agents move to consumption pricing of US$3 per million input tokens and US$20 per million output tokens, which is roughly A$4.30 and A$28.65 at the current exchange rate. There is a free trial running until 31 December 2026.

So the first proper bill arrives in January, when everybody is in the ideal frame of mind to read it.

Every question pays the full fare

In a purely ad-hoc setup, every question sets off the same sequence of events.

The agent reads the context it needs: table schemas, column descriptions, glossary terms, instructions and examples. It writes the SQL. BigQuery runs the SQL. The agent reads the results and explains them. If you ask a follow-up, much of that context goes round again, along with the conversation so far.

Every time.

It is a bit like employing a brilliant analyst who has no memory and insists on rereading the entire data dictionary before answering any question, including the one you asked yesterday. And the day before. And, now that you mention it, every Monday since July.

None of this is a design flaw. Conversational analytics is very good at questions nobody has asked before, and those questions genuinely need the reading, the reasoning and the drafting.

The problem is that most questions are not new. They are the same few dozen questions, asked by different people, in slightly different words, on a fairly predictable schedule.

The 10x stress test

A useful way to think about the economics is to imagine usage growing tenfold, from 1,000 questions a month to 10,000, and ask what grows with it.

There are four lines on the bill, and they behave quite differently.

Meter What happens at 10x Why
Model tokens Grows in line with questions Every ad-hoc question pays to read the context and write the SQL again.
Warehouse scans Grows in line with questions Every query still has to run, and generated SQL is not always efficient about it.
Catalog and metadata Barely moves Glossaries and descriptions grow with your data, not with how often people ask about it.
People Rises, then plateaus Analysts spend real time checking the common questions early on. Once those are covered, the effort settles.

The first two scale with how often people ask. The last two scale with how many different things they ask about. That difference is where the design opportunity lives.

The meter you can see is not the one that is running

Most of the conversation about AI costs focuses on tokens, because tokens are new, interesting and itemised on an invoice with a fresh product name.

So let me put some numbers on it, in Australian dollars. They are illustrative assumptions, not benchmarks, but they are the kind of numbers worth testing against your own environment.

Say an ad-hoc question reads about 12,000 tokens of context and results and writes about 1,500 tokens of SQL and explanation. At Google's published rates, that is roughly 9.5 cents per question. At 10,000 questions a month, around A$950.

Not nothing. Not the sort of figure that gets anybody called into an office either.

A small brass meter glows on a desk while a colossal meter turns in a data warehouse turbine hall behind it
One meter gets watched. The other one gets paid.

Now say the generated SQL forgets a partition filter on a large table and scans 200 GB instead of 20. At BigQuery's on-demand rate of US$6.25 per TiB (about A$8.95), that one query costs about A$1.75. At 10,000 questions a month, it is around A$17,500.

The token bill is the meter you can see. The warehouse is the meter that is quietly running in the background.

This will vary considerably. With small, well-partitioned tables, tokens may well be the larger cost. With large, raw, unpartitioned tables, scans will dwarf everything else. The point is not that one meter always wins. The point is that if you only watch one, you are budgeting with one eye shut.

There is also a small detail in how BigQuery caching works that I find quietly delightful. BigQuery does not charge for results served from its cache, but only when the query text is identical. Google's documentation is quite specific: a change in whitespace or a comment is enough to miss the cache.

A model that writes the same logic slightly differently each time has, in effect, opted out of the discount.

The Tuesday join problem

The cost that worries me most does not appear on any invoice.

Large language models are probabilistic. Ask the same question on Tuesday and Thursday and you may get two slightly different pieces of SQL. One uses an inner join. The other uses a left join. One counts cancelled orders. The other does not.

Both answers look authoritative. Both come with a nicely formatted chart.

Two printed reports showing the same bar chart with different bar heights lie side by side on a dark boardroom table
Same question. Same data. Two different answers, both beautifully formatted.

Eventually, two executives bring two different revenue numbers to the same meeting. The cost of that is not measured in tokens. It is measured in the forty minutes spent establishing which number is real, while everybody else in the room discovers an urgent reason to check their phone.

Consistency is not a nice-to-have in reporting. It is most of the point.

Author, validate, reuse

The sustainable pattern is not complicated. It is just slightly less exciting than the demo.

  1. Author. A business user or analyst asks the question in plain language. The agent drafts the SQL using the glossary and metadata in Knowledge Catalog. This is where the model earns its tokens.
  2. Validate. An analyst checks the logic, the edge cases, the performance and whether it should be answered at all.
  3. Save. The checked query goes back in as a verified query (Google's renamed ‘golden queries’), with parameters for things like date ranges and regions.
  4. Reuse. The next person who asks a matching question gets the deterministic, checked SQL rather than a fresh interpretation.

Google's documentation describes verified queries as deterministic SQL that runs when it matches a user's question, and the API flags when one has been used. That gives you consistency, a tuned query and the same SQL text every time, which helps with that cache.

There is a detail here worth getting right, partly because I initially got it wrong.

A verified query is not a zero-token answer. Google's documentation does not spell out what a match costs, but the question still goes to the agent, which has to work out that it matches and usually still explains the result. What you stop paying for is the SQL being invented from scratch. You do not stop paying for the question being understood.

A cheaper conversation is still a conversation.

The genuinely zero-token path is graduation. When the same question is asked every Monday by forty people, it is no longer really a question. It is a report. It belongs in a view, a dashboard or a scheduled job, where it runs once and everybody reads the result.

So I find it useful to think in three lanes:

A night-time railway junction where one track winds into fog, one passes a lit signal box and one runs straight into a bright station
Explore the new. Recognise the familiar. Graduate the recurring out of the chat window.

The aim is not to eliminate the first lane. That is where conversational analytics is most useful. The aim is to stop the third lane from being handled by the first.

The crossover

Here is the slightly awkward part, and I think it is the most important one.

Using the same illustrative numbers, plus a modest allowance for analyst time to check and maintain verified queries, the curated approach costs more at 1,000 questions a month. Analyst hours outweigh the savings. At around 3,000 questions, the lines cross. At 10,000, the curated approach costs less than half as much as the ad-hoc one, and the gap keeps widening.

That is not a flaw in the argument. It is the argument.

At low volume, discipline looks like overhead. By the time volume makes it obviously worthwhile, four hundred people have formed habits, a dozen slightly different revenue definitions are circulating in slide decks and somebody has already built a dashboard on top of one of them.

Retrofitting governance onto established habits is a much harder job than designing for the volume before it arrives.

Try it with your own assumptions below. The interesting number is not either total. It is where the lines cross.

Interactive model

Ad hoc versus author, validate, reuse

Change the assumptions to match your environment. Everything runs in your browser and nothing is sent anywhere. Figures are monthly, in Australian dollars. Google publishes its prices in US dollars, so the model converts them at the exchange rate below.

The model compares today's volume with ten times today's volume.

Ad hoc today-
Curated today-
Ad hoc at 10x-
Curated at 10x-

Line chart of monthly cost against questions per month for both approaches.
Where the money goes
MeterAd hoc todayCurated todayAd hoc at 10xCurated at 10x
Ad-hoc questions
Curated questions
People and prices

How it works: ad-hoc questions pay for tokens and a full scan. Verified-query matches pay a smaller token cost plus a tuned scan. Graduated reports pay only a tuned scan. Cache hits and materialised results are treated as free. Analyst time is a fixed monthly cost that does not grow with volume, which flatters the curated approach at very high volumes and penalises it at low ones. The defaults are illustrative assumptions, not benchmarks. The free BigQuery tier and the agent free trial are ignored. The default exchange rate is the Reserve Bank of Australia rate for 9 October 2026.

What I would do before the 10x arrives

Treat generated SQL as a draft. Do not point a user-facing agent at raw, unpartitioned production tables and hope. Point it at curated views and partitioned tables. Set a per-query byte cap: the Conversational Analytics API supports a maximum billed bytes setting, runs a dry run first, and fails the query without charge if it would exceed the limit. Add project and per-user daily query quotas where you use on-demand billing.

Invest in the catalog. Business descriptions, approved metric definitions, join paths and glossary terms are a largely fixed cost that improves every answer. Good metadata is the difference between SQL that knows about your partitions and SQL that discovers them the expensive way. Google also advises against defining the same glossary terms in two places, which is sensible advice for almost everything in life.

Decide when a question graduates. Agree a simple rule, such as any question asked by more than a set number of people or more than a set number of times a month, and review the most frequent questions regularly. If you cannot see which questions are being repeated, that is the first thing to fix.

Count all three meters. Track cost per trusted answer, not just tokens per question. Include warehouse scans and analyst time. On that note, Google's current guidance is that tracking costs for individual agents through BigQuery labels is not supported, and that you should isolate each agent in its own project if you need per-agent limits. Your cloud platform team may have feelings about this. It is better to discover those feelings before go-live.

The question worth paying for

Conversational analytics is excellent at the question nobody has asked before. That is where the reasoning, and the money, should go.

The question everybody asks every Monday deserves something less impressive and more reliable: an answer that has already been checked.

Paying AI to answer the same question twice is a design choice.

Source Notes