Models & providers

A model is just a string. You hand Agent something like "claude-sonnet-4" and AgentRoute figures out which provider to talk to, which base URL to hit, and which API key to use. One OpenRouter key works for every hosted model, so most of the time there is nothing else to configure.

from agentroute import Agent
 
agent = Agent(name="assistant", model="claude-sonnet-4")
result = agent.run("Summarize the plot of Hamlet in one sentence.")
print(result)

This page explains how that string is resolved, how vendor prefixes are inferred, where your API key comes from, and how to point at a local Ollama server or any OpenAI-compatible endpoint. The exact signatures live in the models reference.

How a model string is resolved

Agent passes your model to resolve_model, which turns it into a concrete provider connection. Resolution follows three rules, checked in order.

# base_url -> https://my-gateway.example.com/v1, model_id -> my-model
agent = Agent(name="a", model="https://my-gateway.example.com/v1/my-model")

agent = Agent(name="a", model="ollama/llama3.2")

agent = Agent(name="a", model="claude-sonnet-4")        # -> anthropic/claude-sonnet-4
agent = Agent(name="b", model="anthropic/claude-sonnet-4")  # used verbatim

Vendor inference for bare names

When you pass a bare name with no /, AgentRoute infers the OpenRouter vendor prefix from how the name starts. This lets you write "gpt-4o" instead of "openai/gpt-4o".

If a bare name does not match any of these prefixes, it is sent to OpenRouter unchanged. When in doubt, write the full vendor/model slug and skip inference entirely.

API-key resolution

For any provider that needs auth, AgentRoute looks for a key in a fixed order and uses the first one it finds.

agent = Agent(name="a", model="claude-sonnet-4", api_key="sk-or-...")

export AGENTROUTE_API_KEY="sk-or-..."

The config file

Config reads ~/.agentroute/config, a simple key=value file. Blank lines and lines starting with # are ignored. The OpenRouter key uses the bare api_key name; other provider kinds use a <kind>_api_key name, and each kind may also set a <kind>_base_url override.

# OpenRouter catch-all key
api_key=sk-or-...
 
# Optional per-provider overrides
openai_api_key=sk-...
openai_base_url=https://api.openai.com/v1

The provider kind is derived from the model string during resolution — hosted models all resolve to the openrouter kind, so for the common case you only ever set AGENTROUTE_API_KEY or the api_key= line above. URL and ollama/ strings skip the Config lookup entirely.

Using Ollama

Run a model locally with no key. Make sure Ollama is running and the model is pulled (ollama pull llama3.2), then prefix the name with ollama/.

from agentroute import Agent
 
agent = Agent(name="local", model="ollama/llama3.2")
result = agent.run("Give me three names for a Rust web framework.")
print(result)

To point at a remote Ollama host or a non-default port, pass base_url explicitly — it overrides the http://localhost:11434/v1 default.

agent = Agent(
    name="local",
    model="ollama/llama3.2",
    base_url="http://gpu-box.lan:11434/v1",
)

Using a custom endpoint

Any OpenAI-compatible server works. You can encode the URL in the model string, or keep the string clean and pass base_url (and api_key, if the endpoint needs one) as separate arguments. The explicit base_url argument always wins over a URL embedded in the model string.

from agentroute import Agent
 
# Endpoint and key passed as arguments
agent = Agent(
    name="gateway",
    model="my-model",
    base_url="https://my-gateway.example.com/v1",
    api_key="sk-...",
)
result = agent.run("Ping?")
print(result)

The message and request types

You rarely touch these directly — the agentic loop builds them for you — but they define the contract between AgentRoute and a provider. A custom Model implementation receives an LLMRequest and returns an LLMResponse.

• Message — one OpenAI-compatible turn: a role ("system", "user", "assistant", or "tool"), optional content, and tool-call fields (tool_calls, tool_call_id, name).
• LLMRequest — what gets sent: the model id, a list[Message], optional tools schemas, temperature, max_tokens, and an extra dict[str, Any] for provider-specific options.
• LLMResponse — what comes back: content, any tool_calls, a usage dict, and the resolved model.

The Model protocol is just three async methods — complete, complete_stream, and close — so anything that satisfies them can be passed wherever a model is expected, including straight into Agent(model=...).

from agentroute import Agent, resolve_model
 
# Resolve once, reuse across agents
model = resolve_model("claude-sonnet-4")
agent = Agent(name="assistant", model=model)

See the models reference for full signatures, defaults, and the streaming LLMChunk type.

Next steps