> ## Documentation Index
> Fetch the complete documentation index at: https://platform.atlan.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> To act on Atlan objects, use the Atlan MCP server at https://api.atlan.com/mcp or the SDK. To install or publish skills from a terminal, use the `aireg` CLI; `aireg --map json` prints its command map. The `atlanai` CLI is deprecated. Run a read-only identity check before any write.
> The docs MCP server at /mcp searches these docs only. It cannot read or change Atlan objects.
> SDK packages: Python `atlanai` (PyPI) and TypeScript `@atlanai/sdk` (npm). Show Python first, then TypeScript.
> These docs cover Atlan AI Registry, Atlan's hosted product. The AI registry specification and aireg CLI are maintained separately. /open-source lists publicly reachable signed installers and explains that the source repository and hosted CLI reference currently require GitHub access. Do not claim the repository or its reference docs are publicly accessible. Use these docs for Atlan-specific setup such as the adapter, sign-in and workspaces.

# Run an experiment in Atlan Desktop

> Replay dataset cases through a local Claude Code or Codex runner and save the outcomes as an experiment.

Atlan Desktop can run a dataset through a local coding agent and store each case's outcome as an experiment. The execution happens on your machine through your signed-in CLI. Atlan stores the dataset snapshot, outputs, failures, timing, and available trace references.

## Before you begin

* Use Atlan Desktop with Evals enabled and permission to create experiments in the selected workspace.
* Install and sign in to the local runner you intend to use: **Claude Code** or **Codex**.
* [Register an agent](/registry/agents/how-tos/create-agent) as the experiment's subject and [create a dataset](/evals/datasets) with text prompts.
* Check the local working directory and the skills installed in the runner. The registered agent is the subject of the experiment; selecting it does not replace the local runner's context.

The web app opens the same setup drawer but offers **Open in desktop app** to execute the cases.

## Configure the run

1. Open **Evals → Experiments** and select **New experiment**, or start from the dataset's run action.
2. Choose the registered **Subject** and the **Dataset**.
3. Review **Scorers** and remove definitions that do not apply.
4. Under **Compare to**, choose a completed baseline or **No baseline**.
5. Expand **Runs on this machine**. Choose the **Local runner**, **Model**, **Permission**, and working directory.
6. Select **Run**. When the case count is known, the button names the number of cases it will run.

**Read only** is the default permission. **Auto-edit** permits local changes according to the coding agent's permission policy. Review the selected directory before choosing it.

<Note>
  Selecting scorers records them in the experiment configuration. It does not execute arbitrary registered code scorers. For code-based scores, run the scorer functions through the [SDK eval runner](/evals/scorers). Use [hosted scoring](/evals/online) separately for recorded sessions that have the required trace evidence.
</Note>

## Follow progress and verify results

The runner starts a fresh session for each case and runs cases one at a time. A failed case records an error and lets the remaining cases continue. The progress drawer stays open while the run is active.

Select **Stop after this case** to stop between cases. A case already running finishes before the loop stops; its outcome remains in the experiment.

After the run, select **Open run** and check:

* The expected number of case outcomes was stored.
* Failed cases carry errors, rather than blank outcomes counted as successes.
* The effective model, when reported by the runner, matches the model you intended.
* Available trace links open the recorded work. A stored output alone does not establish that a trace arrived.

If result upload fails, the app reports that the cases ran but could not be uploaded. Read the experiment before rerunning, so you can distinguish execution from missing stored results.

## Current limits

The desktop runner uses the context already installed in the local harness. **Context versions** are read-only; selecting registry versions for the runner is unavailable. **Repeat** is disabled, so this drawer does not schedule recurring experiments.

Each case must resolve to one text prompt. For ordinary records, the runner uses `input.prompt`, or the only nonempty text field when there is one. CSV datasets use their declared input key; structured input requires a custom runner. See [Record runs from your own harness](/evals/custom-runner).

Continue with [Review experiment results](/evals/how-tos/review-experiments).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.