Skip to content

Models and settings

aidy calls language models through the AI SDK’s provider packages. You give it a chain: endpoints tried in order, so a provider that fails, is out of credit or is rate-limited falls through to the next. The configured chain is the default; admins can replace it at run time without a redeploy.

createAidy({
// ...
models: {
chain: [
{
id: 'qwen',
provider: 'openai-compatible',
model: 'qwen3-max',
baseURL: 'https://dashscope.aliyuncs.com/compatible-mode/v1',
apiKey: process.env.DASHSCOPE_API_KEY,
providerOptions: { 'openai-compatible': { enable_thinking: false } },
},
{
id: 'claude',
provider: 'anthropic',
model: 'claude-sonnet-4-5',
apiKey: process.env.ANTHROPIC_API_KEY,
vision: true,
},
],
},
});

Each endpoint:

Field Meaning
id Required and unique in the chain, 1 to 64 characters. Settings keep an entry’s API key by it.
provider openai-compatible, openai or anthropic.
model The provider’s model id.
baseURL Required for openai-compatible; optional for the others (a proxy or gateway).
apiKey The provider key. Without it, the openai and anthropic packages read OPENAI_API_KEY and ANTHROPIC_API_KEY.
vision The model accepts images. Turns with images go to vision endpoints first (Vision).
temperature Sent with every call to this endpoint; omitted when unset.
maxOutputTokens Sent with every call to this endpoint.
cooldownSeconds { quota?, rateLimit?, error? }: how long a failing endpoint rests (Fall-through).
providerOptions Passed to the provider, keyed by provider name.
  • openai-compatible covers any endpoint that speaks the OpenAI chat API: Qwen through DashScope (https://dashscope.aliyuncs.com/compatible-mode/v1), DeepSeek, vLLM, a LiteLLM gateway, and others. It needs baseURL.
  • openai is OpenAI itself.
  • anthropic is Claude.

providerOptions reaches the provider unchanged. For Qwen’s reasoning models, { 'openai-compatible': { enable_thinking: false } } turns off thinking, which keeps tool-calling turns fast. aidy never sends raw provider reasoning to the client.

When aidy is embedded, a chain entry may also be any AI SDK language model, for example Workers AI through the agents package:

import { createAI } from 'agents/models/ai-sdk';
createAidy({
// ...
models: {
chain: [{ id: 'workers-ai', languageModel: createAI({ binding: env.AI })('@cf/zai-org/glm-4.7-flash') }],
},
});

A custom entry is { id, languageModel } plus the same vision, temperature, maxOutputTokens and cooldownSeconds; id names it in settings. Custom entries can only come from code; the settings API accepts provider endpoints only.

For each model step, aidy calls the first endpoint in the chain that is not resting:

  • Timeouts, 408, 5xx, network errors and unusable answers (an unknown tool, an empty answer) are retried on the same endpoint, at most 4 attempts.
  • Any other failure, such as 400, 401, 403, 404, a quota answer or 429, and the last of those retries, moves the step to the next endpoint, and the failing endpoint rests:
Failure Rest
Quota-like: 402, or a status or body matching quota, insufficient, exhausted or a provider quota code such as 401008 6 hours
429 (rate limited) 60 seconds
Anything else 5 minutes

An unusable answer moves on without resting the endpoint. Override the durations per endpoint:

{ id: 'qwen', provider: 'openai-compatible', /* … */ cooldownSeconds: { quota: 3600, rateLimit: 30, error: 120 } }

A resting endpoint is skipped. When none is left, the run ends with RUN_ERROR code model_unavailable. Clients see each failed attempt as an aidy.model.retry event, with the endpoint it went to and fallback: true when the next attempt goes to another endpoint (the dock shows “Model busy, retrying”); partial text from a failed attempt is dropped with aidy.message.discard. Event details are in the event reference.

There is one chain per aidy instance, shared by runs and host jobs. Rests are kept in its memory (a process, or a Workers isolate).

A turn whose messages include images goes to endpoints with vision: true first, in chain order. When no vision endpoint can answer, the others get the text alone, with a note in place of each image; without any vision endpoint aidy never fetches the images. Put a vision model somewhere in the chain if users attach screenshots; a text-only model first keeps ordinary turns cheap. How images reach the model is in AG-UI: user messages.

Any signed-in user can read the chain in effect. Users whose identity has the scope aidy:admin can replace it at run time.

GET /aidy/v0/settings
{
"models": {
"chain": [
{
"id": "qwen",
"provider": "openai-compatible",
"model": "qwen3-max",
"baseURL": "https://dashscope.aliyuncs.com/compatible-mode/v1",
"hasApiKey": true
}
]
}
}

GET never returns API keys, only hasApiKey. A custom entry appears as { id, provider: "custom", model, vision?, hasApiKey: false }, where model is the language model’s id.

PUT /aidy/v0/settings
Content-Type: application/json
{
"models": {
"chain": [
{ "id": "qwen", "provider": "openai-compatible", "model": "qwen3-max", "baseURL": "https://dashscope.aliyuncs.com/compatible-mode/v1" },
{ "id": "gpt", "provider": "openai", "model": "gpt-5", "apiKey": "sk-…", "vision": true }
]
}
}
  • PUT needs aidy:admin and replaces the whole chain. The chain must have at least one entry, and unknown fields are rejected.
  • An endpoint sent without apiKey keeps the key of the entry with the same id. Send a key only to set or change it.
  • provider must be openai-compatible, openai or anthropic: a custom entry cannot be sent back, so saving replaces it.
  • Saved settings take precedence over the configured models from then on, custom entries included. The response is the new chain, in the GET shape.
DELETE /aidy/v0/settings

DELETE needs aidy:admin. It forgets the saved settings, so the configured chain applies again, and answers it in the GET shape.

Saved settings are encrypted with AES-GCM under a key derived from the instance secret (32 or more bytes) before they reach the store. To rotate it, pass a list: the first secret seals, every one unseals, and settings sealed with an older one are sealed again with the first when read.

createAidy({ secret: [process.env.AIDY_SECRET!, process.env.AIDY_PREVIOUS_SECRET!], /* … */ });

aidy-server reads the same pair from AIDY_SECRET and AIDY_PREVIOUS_SECRET. Settings sealed with a secret aidy no longer has answer 503 settings_unreadable; an admin can then DELETE them. See Deploy.

@aidy-dev/react ships the admin form:

import { ModelSettings } from '@aidy-dev/react';
export function AdminModelsPage() {
return <ModelSettings />;
}

It lists the chain, lets an admin add, remove and reorder endpoints, pick the provider, and set the model, base URL, key, temperature, max output tokens and image support. Keys are write-only: the form shows whether one is stored, and an empty key field keeps it. New rows get an id model-N. Models configured in code (provider custom) are shown read-only; saving replaces them, and the form says so. Reset to defaults asks for confirmation, then calls DELETE /aidy/v0/settings and shows the configured chain. Render it inside AidyProvider, on a page only admins reach; the server enforces aidy:admin either way. See React.