Custom LLM proxy

Route an agent's model calls through a gateway you run instead of Ellipsis's provider accounts. Prompts and completions leave through infrastructure you control, so your gateway can log, filter, or fulfill traffic on its own provider contracts. This feature is enabled per account on request; the Models page tells you how to ask.

When to use

• Compliance: your policy requires LLM traffic to exit through an endpoint you audit.
• Logging: you want your own record of every prompt and completion.
• Your own provider deals: your gateway fronts a provider contract you already pay for (an LLM router such as LiteLLM, OpenRouter, or a Cloudflare AI Gateway).

Unlike your Anthropic key and Bedrock, which route every session once enabled, a proxy routes nothing by default. You may register several, and each agent opts in individually. That is the point: move one agent's traffic without touching the rest.

Configuration

Register the gateway on the Models page of the dashboard, under LLM proxy. You provide:

• Name: a label for the dashboard.
• Base URL: the gateway's address. Ellipsis appends /v1/messages, and pasting the full endpoint from your provider's docs works too (a trailing /v1/messages or /v1 is stripped).
• Credential (optional): what your gateway expects. Ellipsis sends it as both x-api-key and Authorization: Bearer. Leave it blank for a gateway that authenticates by network position. The credential is write-only: no API response or dashboard surface returns it (the table shows the last 4 characters), and it never enters a sandbox; Ellipsis attaches it server-side on the way out.
• Model map (optional): Ellipsis model id to your gateway's model id, for gateways that name models differently (anthropic/claude-opus-5, an alias you defined). Unmapped models pass through unchanged, which is right for a plain reverse proxy.

Save & verify runs a live streamed completion through the gateway; routing turns on only when that succeeds. If a verified gateway later breaks, re-running Verify disables it rather than leaving sessions to fail mid-run. Any edit disables routing until the next successful verify, and changing the base URL requires re-entering the credential.

The table shows each proxy's id (proxycfg_...). Point an agent at it:

llm:
  proxy: proxycfg_8f2k1x9q

That agent's sessions now send every model call through your gateway. Agents without the field keep their current routing. The id is scoped to your account, so a config naming another account's proxy fails the session at start; a deleted or disabled proxy also fails the session at start, with the reason recorded. Ellipsis never falls back to its own provider accounts silently.

With any proxy enabled, the token line of your Ellipsis bill drops to zero; your gateway's provider bills for the tokens it serves. Sandbox compute and the platform fee are unchanged, and session cost breakdowns still show real token spend so spend limits keep working. A model Ellipsis has no rate for is priced at the Claude Fable 5 rate for spend-limit purposes, an estimate that keeps the cap meaningful; your gateway's provider bills the real price.

Constraints

Your endpoint must be a working Anthropic Messages API. Concretely:

• POST {base_url}/v1/messages accepting the Anthropic request shape, over https on a public address (port 443 or 8443). No query string or fragment in the base URL.
• Streaming: responses stream as Anthropic-shaped server-sent events. Agent traffic is almost entirely streaming, and gateways that buffer or rewrite SSE framing fail verification.
• A usage block on every completion. Ellipsis prices each call from reported usage to enforce spend limits, so a gateway that omits it cannot be enabled.
• Serving the models your agents name. Verification probes with your mapped model when a map exists, otherwise with the default agent model (claude-opus-5). If your gateway serves different model ids, add the model map rather than expecting a translation layer.

Verification checks all of these with one streamed completion and reports exactly which one failed.