> ## Documentation Index
> Fetch the complete documentation index at: https://platform.atlan.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> To act on Atlan objects, use the Atlan MCP server at https://api.atlan.com/mcp or the SDK. To install or publish skills from a terminal, use the `aireg` CLI; `aireg --map json` prints its command map. The `atlanai` CLI is deprecated. Run a read-only identity check before any write.
> The docs MCP server at /mcp searches these docs only. It cannot read or change Atlan objects.
> SDK packages: Python `atlanai` (PyPI) and TypeScript `@atlanai/sdk` (npm). Show Python first, then TypeScript.
> These docs cover Atlan AI Registry, Atlan's hosted product. The AI registry specification and aireg CLI are maintained separately. /open-source lists publicly reachable signed installers and explains that the source repository and hosted CLI reference currently require GitHub access. Do not claim the repository or its reference docs are publicly accessible. Use these docs for Atlan-specific setup such as the adapter, sign-in and workspaces.

# Score recorded runs

> Use extension__eval__score through Atlan MCP Gateway.

Call `extension__eval__score` with the inputs below. Open an MCP session with `mcp__initialise` first and pass its `session_id`. The authenticated `tools/list` response is the final check for the name and schema available to your account.

This is a write: it runs the selected scorer and stores answers. Verify the returned result and use the insight lookup to inspect stored scores.

## Inputs

| Input | Type | Required | Meaning |
| - | - | - | - |
| `trace_ids` | array | No | OTel trace ids to judge, at most 100 with `session_ids` combined. Each is judged as **that** trace of the session it records — a run list holds these and nothing else, and a session with several runs is judged by the one named. Default: `[]`. |
| `session_ids` | array | No | Session artifact handles (`session_…`) to judge, each by its earliest trace. Default: `[]`. |
| `agent_ids` | array | No | Agent handles (`agent_…`). Judge the sessions those agents ran, newest first — narrowed by `since`/`until` when given, at most `limit` in all.  **An agent, not an agent version.** A `session` records which agent it ran, not which version of it, so "this agent at version 3" is not a question this selector can answer and is not silently approximated. Default: `[]`. |
| `experiment_id` | string,null | No | An experiment handle (`experiment_…`). Judge the sessions its results ran as, in result order, at most `limit`. Results recorded by a harness that logged no session are skipped: there is nothing to judge. Default: `null`. |
| `since` | string,null | No | RFC 3339. Judge the scorer's workspace's sessions created at or after this instant, newest first. With `agent_ids`, narrows those agents' sessions instead of selecting on its own. Default: `null`. |
| `until` | string,null | No | RFC 3339. Upper bound of the window; defaults to now. Default: `null`. |
| `limit` | integer,null | No | How many sessions to judge, at most 100. Defaults to 20. Default: `null`. |
| `grain` | object | No | What each verdict is about: `trace` (the default) judges one run, `session` judges every run the session recorded, as one transcript.  The wide grain is asked for rather than assumed because it reads proportionally more: a session with twenty turns sends twenty runs to the judge, and the judge is charged for all of it. Default: `"trace"`. |
| `mode` | object | No | `score` (the default, asynchronous, writes insights), `preview` (build the judge's request and return it, calling nothing) or `test` (judge and return the answers, writing nothing). Default: `"score"`. |
| `concurrency` | integer,null | No | How many judges to keep in flight. Defaults to 4; the ceiling is this deployment's `GATEWAY_EXT_EVAL_JUDGE_CONCURRENCY_MAX` (25 when it sets none).  **Both numbers are unmeasured, and deliberately not dressed up.** The old ceiling was read off Claude Managed Agents' published limits; that runtime is gone from this path, and neither TypeSafe nor the gateway's upstream providers publish a per-organization limit this extension could cite. Raise it for a run you are watching: the run row records what it ran at, what each judge call took, and whether the backend refused for load — which is what a future default should be argued from. Default: `null`. |
| `scorer_id` | string | Yes | The `scorer_…` handle to run. |
| `rationale` | string | Yes | A concise, first-person explanation of why you are calling this tool right now: what you are trying to accomplish for the user and how this specific call advances that goal. Written for a human auditor reviewing tool usage after the fact. Do not restate the arguments - explain the intent behind them. |
| `session_id` | string | Yes | The id `mcp__initialise` returned for this conversation, so this call is recorded against it. Required: call `mcp__initialise` first and pass the id it returns. Do not invent a value - an id that names no session of yours is refused. |

## Permissions and result

The call uses your authenticated Atlan identity and the target workspace's permissions. This tool changes recorded data. A missing or inaccessible target can be refused even when the tool is listed.

See [MCP tool catalog](/tools/mcp/references/tool-catalog) for related tools and [MCP Gateway errors](/gateway/mcp/troubleshooting) for refusal and connection diagnostics.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.