What an AI Analyst Needs From Your Semantic Layer

An AI analyst that works on a semantic layer answers a question in two steps. First it chooses a governed metric. The previous article was about this step: the analyst finds the metric in the layer's catalogue, or learns that none exists. Then it fills in the details the question asks for: which platform, which country, which dates. This article is about the second step. It gets less attention, and it goes wrong more quietly. Every parameter of the query must be grounded.

Querying the semantic layer#

The semantic layer used in the experiment defines app_opens, the fields available for filtering and grouping, and their values. The metric's definition always excludes internal accounts.

We give the agent a tool, query_metric, that queries this layer. The agent supplies the metric and parameters; the layer generates SQL using the metric's definition.

The agent must set metric. It sets four optional parameters when the question requires them: filters, group_by, period and time_grain. The tool accepts the layer's dimension names and values, so the agent must translate the user's words into these arguments.

query_metric(
  metric     = app_opens,
  filters    = {platform: ios},
  group_by   = [country_name],
  period     = 2026-Q2,
  time_grain = week,
)

These are the dimensions and values available when querying app_opens in our experiment:

The SQL can execute correctly even when an argument misreads the question. That is the failure this experiment tests.

Where it goes wrong#

Take one question: how many app opens came from iPhone users in Germany in summer 2026?

The metric is easy: app_opens. The parameters are not. "iPhone" is not a value of any dimension. "Summer" is not a period the tool accepts. Nobody said anything about internal accounts. Here are two calls an analyst could make:

The first call drops "iPhone" because no value matches it. It runs, returns a clean number, and is two and a half times too large. Nothing fails: the metric is governed and the SQL is correct. The error is in a parameter. The second call is right, and it is still not the whole answer: the data ends on 12 July, so the answer must also say that summer is only partly covered.

The metric was governed. The answer was not.

Every parameter of the query must be grounded#

Go back to the iPhone question. A careful analyst does not reach for the nearest call. They take the question apart and settle one parameter at a time, from the easiest to the hardest. Each parameter is grounded in one of two ways: by a rule the data team wrote down, or by a dimension search that finds the value among the values of a dimension.

Germany is the easy one. The dimension search finds it as an exact match: it is one of the eight values of country_name, so the parameter is country_name = Germany. There is nothing to interpret.

Summer needs a rule. No dimension has a value called summer, but the company calendar defines it as June to August. That definition only works north of the equator: for Brazil the same word means December to February, and the analyst would ask which months are meant. The layer records that Germany is in the north, so the rule applies and gives 1 June to 31 August 2026. One catch: the data stops on 12 July. The analyst answers for the six weeks it has and says that summer is only partly covered.

"iPhone" is where judgement comes in. No rule mentions it, and the dimension search finds no exact match. Any analyst would read it as iOS, and platform = ios is the closest value on the menu. It is a little broader than the question, since iOS also runs on iPads, which is why the answer spells out the choice. The layer adds a second catch: platform is the device the account signed up on. Someone who signed up on an iPhone and now opens the app on Android still counts as ios. The answer says that too.

The last parameter is one nobody said. The question never mentions internal accounts, but the definition of app_opens in the layer excludes them. That does not change the call. It does change the number, so the answer names it.

Now every parameter has a source the analyst can point to, and every assumption is written next to the number.

query_metric(
  metric  = app_opens,
  filters = {
    platform: ios,
    country_name: Germany,
  },
  start   = 2026-06-01,
  end     = 2026-07-12,
)

That is what grounded means. A parameter is grounded by a rule or by a dimension search, and a dimension search ends in one of two ways:

AI analyst judgement is allowed, because a senior analyst uses judgement every day. It is used only when no rule and no exact match settles the parameter, and it is always stated, because it is the step most likely to be wrong. The analyst never guesses silently.

How a dimension search works#

On the test layer a dimension search is easy. Every dimension has a short list, so the candidates are simply all of its values, and the model chooses among about 74 options. Real layers are not like that. Cities, products, customers and campaigns have thousands of values, and a question with thousands of options is slow, expensive and unreliable.

So a dimension search has two parts, with one contract between them. Candidate search narrows the values to a short list. Resolution decides. Code settles an exact match on its own. The model is asked only when no candidate matches exactly, and only about the candidates.

We cannot know how a user will name a value. The same city arrives as "Munich", "Muncih", "München", "MUC" or "Munich, Germany", and each form needs a different method. So a dimension declares several search methods, in priority order, and code runs all of them. They take milliseconds and need no model. An exact match from a high-priority method ends the search. Two different exact matches are a conflict, and the analyst asks. Otherwise the candidates from every method go to the model together, in one judgement.

In the semantic layer, the declaration sits next to the values it searches:

- name: city
  config:
    meta:
      search: [aliases, hierarchy, words, meaning]   # in priority order
      aliases: city_alias                             # Bombay, Big Apple, München
      complete: true

Two rules keep the design safe.

Every candidate carries its context. A large dimension has namesakes. When we built a test city dimension of 12,409 cities, six of the foreign capitals we picked as test cases were already in it: Rome, Moscow, Lima, Dhaka, Vienna and Amsterdam, as smaller towns in the eight countries (Rome, Georgia; Lima, Ohio). An exact match on "Rome" would answer about Rome, Georgia, and the answer would look grounded. Shown as "Rome, Georgia, United States", the candidate can be judged, and the answer can state it.

"Not in the data" needs an exhaustive search. A short list from a similarity search always contains something, and a low score proves nothing. In the city test, the scores of absent cities overlapped the scores of misspelled present ones. The analyst may say a value is not in the data only when the dimension is declared complete, no exhaustive method found the phrase, and the model judges that it names a value missing from the list. Every other miss becomes a question.

Does it work?#

We built this in one experiment, changing the semantic layer and the analyst step by step. Each version was tested on fresh questions written outside the work: one AI model wrote the requests and the right outcome for each, another wrote three wordings of every request, neither saw our code, and we committed our predictions before each run. We tracked three numbers:

We did not get there in one step. The experiment's findings record every change; five of them moved the numbers:

How to measure your own semantic layer#

Before an AI analyst goes in front of your team, you want to know three things: where it will be right, where it will ask, and where it will give a wrong number with nothing said. A pass or fail per test question does not tell you that, and it does not tell you what to fix. So we measure the semantic layer itself, one parameter at a time: every parameter a metric accepts should have a source the analyst can find. The method checks that against the questions your people actually ask.

The result is a grid. Each row is a metric, and each column is a parameter it accepts. Each cell counts what happened to the test questions that used that parameter: grounded, grounded on a stated assumption, asked back, or a silent error. The colour shows the worst outcome, so a single silent error turns a cell red. The last column shows, per metric, how many parameter checks were grounded. The counts are illustrative, modelled on our real run.

Read the red cells first: they are wrong numbers nobody noticed. Then the yellow ones: questions the analyst could have answered if the layer held one more value, meaning or rule.

What this means for your semantic layer#

I'd start with a few questions your team asks regularly. For each one, can the analyst find the right values, apply the right rules and explain its choices? A metric can be well defined and still return the wrong number if "iPhone users" is misinterpreted or "summer" uses the wrong dates.

Four things will help make your semantic layer ready for an AI analyst:

1. The terms people use. If people ask about "mobile", model it as a value they can query. Include aliases such as "Deutschland" for Germany.

2. Write down the definitions and rules. For example: platform is the device an account registered on; "summer" means June to August in the company calendar; app_opens excludes internal accounts. The analyst needs these facts to choose the parameters and explain the number.

3. Help the analyst look up values. Specify how to search each dimension. For cities, return the state and country alongside the name, so "Rome" does not silently become Rome, Georgia. Also say whether the list contains every city in your data. Finding no match in a short list of suggestions does not prove that the city is absent from the full list.

   I'm working on an architectural approach that makes search part of the semantic layer: each dimension defines how to find its values, and the agent uses that definition. Follow me for updates as I develop it.

4. Make evaluation a regular practice. Keep a set of real questions and rerun it whenever the layer or agent changes. Measure grounding for each parameter, rather than giving the whole question one pass or fail. That shows exactly what needs fixing: the country filter, the date range, the grouping or another part of the query.

The analyst should use those values and rules first. When it needs judgement, it should say how it read the question. If it cannot settle the meaning, it should ask.

Use the readiness grid to choose what to fix first: silent errors, then questions the analyst could answer if the layer supplied the missing meaning or rule.