> ## Documentation Index
> Fetch the complete documentation index at: https://platform.atlan.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> To act on Atlan objects, use the Atlan MCP server at https://api.atlan.com/mcp or the SDK. To install or publish skills from a terminal, use the `aireg` CLI; `aireg --map json` prints its command map. The `atlanai` CLI is deprecated. Run a read-only identity check before any write.
> The docs MCP server at /mcp searches these docs only. It cannot read or change Atlan objects.
> SDK packages: Python `atlanai` (PyPI) and TypeScript `@atlanai/sdk` (npm). Show Python first, then TypeScript.
> These docs cover Atlan AI Registry, Atlan's hosted product. The AI registry specification and aireg CLI are maintained separately. /open-source lists publicly reachable signed installers and explains that the source repository and hosted CLI reference currently require GitHub access. Do not claim the repository or its reference docs are publicly accessible. Use these docs for Atlan-specific setup such as the adapter, sign-in and workspaces.

# Route and fall back across deployments

> Spread calls for one model name across several deployments by weight, retry on transient failures, and keep calls in allowed regions.

Give one model name several deployments, so calls keep working when one provider is slow or down. You set this up once on the model in Model Gateway studio; callers keep sending the same model name.

```mermaid theme={null}
flowchart LR
  C["Call names<br/>openai/&lt;model&gt;"] --> G["Model Gateway"]
  G -->|"weight 3"| A["openai-2/&lt;model&gt;"]
  G -->|"weight 1"| B["azure-eu/&lt;model&gt;"]
  A -. "transient failure:<br/>next attempt" .-> B
```

## How a pooled call is routed

A model with a **Deployment pool** routes every call that names it across the pool's deployments. Each deployment is another enabled model of the same modality in the same workspace.

1. The gateway orders the deployments by weight, at random, without repeats. A deployment with weight 3 is picked first three times as often as one with weight 1.
2. It skips deployments that are off, can't serve the request, are outside the allowed regions, or are cooling down after a recent failure.
3. It calls the first remaining deployment. If that call fails with a rate limit, overload, timeout, transport error, or `5xx`, and no output has reached you yet, it tries the next deployment.
4. It stops when a deployment answers, when it has made the maximum number of attempts, or when the request deadline passes.

Once a model has a pool, calls naming it go only to the pool's deployments. A pooled model can't be a deployment in another pool.

The gateway does not retry an authentication failure, an invalid request, a content refusal, or a reply it can't decode. Once a reply has started streaming to you, a failure ends the stream with an error instead of starting a second answer.

### Recovery settings

Recovery applies to every model, with or without a pool. Without a pool, a retry goes back to the same deployment after its cooldown.

| Setting | Default | Allowed |
| - | - | - |
| **Attempts** (including the first) | 3 | 1 to 8. `1` turns retries off. |
| **Deadline, ms** for the whole request, including waits and streaming | 180,000 | Up to 600,000 |
| **Backoff, ms** between attempts, growing from the first value to the second | 100 to 2,000 | Longest at most 30,000 |
| **Cooldown, ms** a failed deployment is skipped | 5,000 | Up to 120,000 |

A provider's `Retry-After` is respected within the deadline. A fallback never resets the deadline.

### Region restriction

Each deployment can name the region it serves from, such as `us-east-1`. A pool can list **Allowed regions**, and then routes only to deployments whose serving region is on that list. A deployment with no serving region is skipped by a pool that lists regions.

A caller can narrow further, per request, with the `x-gateway-model-regions` header:

```bash theme={null}
curl https://api.atlan.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $ATLAN_API_KEY" \
  -H "x-gateway-model-regions: eu-west-1,eu-central-1" \
  -H "Content-Type: application/json" \
  -d '{"model": "<provider>/<model>", "messages": [{"role": "user", "content": "Hello"}]}'
```

The header narrows and never widens. A call is served only from a region on both the pool's list and the header. For a model without a pool, the header requires the model's own serving region to be listed. The header takes up to 32 comma-separated region names, each 1 to 64 letters, digits, `-`, or `_`.

## Before you begin

* You're an organization admin. Only organization admins can change a model's settings.
* The models you want as deployments are on, have the same modality, and live in the same workspace as the model you're configuring.

<Steps>
  <Step title="Open the model's settings">
    In Model Gateway studio, open **Models**, select the model your callers name, and open the **Settings** tab.
  </Step>

  <Step title="Set each deployment's serving region">
    Skip this step if you don't restrict regions. For each model you'll add to the pool, open its **Settings**, find **Serving region**, choose **Override**, and enter the **Region**, such as `us-east-1`. Select **Save changes**.
  </Step>

  <Step title="Add the deployment pool">
    Back on the model your callers name, find **Deployment pool** and switch from **Direct (default)** to **Pool**. For each row, choose a **Deployment** and enter a **Weight** from 1 to 10,000. Select **Add deployment** for more rows, up to 16. To restrict regions, enter **Allowed regions**, comma-separated, such as `us-east-1, us-west-2`.
  </Step>

  <Step title="Adjust recovery">
    Optional. Under **Recovery**, switch from **Default** to **Override** and set **Attempts**, **Deadline, ms**, **Backoff, ms** with its maximum, and **Cooldown, ms**.
  </Step>

  <Step title="Save the model">
    Select **Save changes**. The bar above it lists what will change before you save.
  </Step>
</Steps>

## Verify

1. On the model's **Overview** tab, **Routing and health** reads, for example, `Weighted across 2 deployments, only in us-east-1.` and `Recovery is overridden: …` when you changed recovery.
2. Make a call that names the model. It answers as before; nothing changes for the caller.
3. Open **Calls** and select the call. The drawer's subtitle shows the number of attempts, and **Served by** shows the provider that answered.

If no deployment qualifies, for example when every deployment is outside the regions you asked for, the call returns `400` with `No authorized deployment meets the route constraints.`

## Related

* [How Model Gateway works](/gateway/model/concepts/how-it-works): where routing sits in a request.
* [Connect model providers](/gateway/model/how-tos/connect-providers): add the providers your deployments come from.
* [Model API compatibility](/gateway/model/references/compatibility#headers): every request and response header.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.