Skip to main content
Preview An analytics agent answers business questions, such as “What was average order value last month?”, by writing SQL against your warehouse. Whether its answers are right depends on the context it reads first. This guide uses the /atlan-analytics command to build that agent on your metrics and saved queries, measure it on questions with known answers, and improve it. This is where Metrics Studio and the rest of AI Registry meet. The metrics and saved queries, the agent record, each eval run, and the trace of every answer are all kept in the same workspace, so you can see which context change moved the score.

​
How it works

  1. Build. An interview settles who the agent is for, what they ask, and which tables it may use. It reuses the metrics and saved queries already in the workspace, adds what’s missing, and asks you to settle definitions people disagree on. Then it registers the agent.
  2. Eval set. You choose real questions with trusted answers. The answers are kept away from the agent.
  3. Baseline. Each question goes to a fresh copy of the agent, and a grader compares its result with the known answer. The run is recorded in Atlan as an eval, with a trace per question.
  4. Improve. Each round looks at what went wrong, makes one change, usually a new metric or saved-query revision, and keeps it only if the score on held-out questions goes up. Every change is a new revision, so you can see what changed and what it did to the score.

​
Before you begin

  • Metrics Studio app: An administrator has installed it for your organization. The /atlan-analytics command appears only when it’s installed. See Install and manage apps.
  • Workspace role: Builder or Admin in the workspace that will hold the agent and its context.
  • A client with the Atlan plugin: Claude Code with the Atlan plugin, or any MCP client connected to MCP Gateway. See Connect client to MCP Gateway.
  • A Snowflake connection in your client: The build, eval, and improve modes run all SQL through a Snowflake MCP connector you’re signed in to. MCP Gateway runs none.
  • Python: python3, for the bundled eval script.

​
Choose a mode

With no argument, /atlan-analytics checks your sign-in and connection, resumes a project saved in your folder if there is one, and otherwise asks whether to build or improve. You can add a project name, for example /atlan-analytics improve store-analytics. In an MCP client without the plugin, use the MCP prompt atlan-analytics that MCP Gateway serves.

​
Build an agent

1

Start the build

In your project folder, run:
The command asks one question at a time and stops after each phase so you can review what it made.
2

Settle purpose and scope

Say who the agent is for and what they ask, for example “store managers asking about orders and revenue”. Then confirm the tables it may use. Keeping scope to five tables or fewer keeps answers accurate.
3

Approve the context

The command reuses metrics and saved queries already in the workspace, drafts the ones that are missing, and shows them to you before it publishes them. Where two definitions of the same number disagree, it asks which one wins.
4

Choose eval questions

Pick real questions with answers you trust. The command keeps the answers in your project folder, away from the agent, and asks whether to add the folder to .gitignore.
5

Get a baseline

The command registers the agent, runs a smoke test, and then scores it on the eval set. The score is your baseline.

​
Improve it

Run /atlan-analytics improve. Each round:
  1. Reads the questions the agent got wrong and the traces of those answers.
  2. Proposes one change, usually a new metric revision or saved query.
  3. Scores the agent again on held-out questions it didn’t learn from.
  4. Keeps the change only if the score goes up.
Because each change is a new revision, a change that helps can be released for every agent that reads the metric, and one that doesn’t can be left unreleased.

​
Verify

  • In Metrics Studio, the workspace lists the agent’s metrics and saved queries. Released ones show Live in the Release column.
  • In the workspace’s agents, the agent appears with its eval runs. See Evals.
  • Each eval question has a trace showing which metrics and saved queries the agent read. See Traces.

​
Where the command keeps its state

The command saves its progress in your project folder, under .atlan-analytics/<project>/ for build, improve, and eval, or .atlan-metrics/<scope>/ for metrics mode. Run the same command again in that folder to resume.