Documentation
Loading…
Provider routing
How a request chooses a provider, and the preferences you can set per request.
Base URL for this deployment: https://www.aiteraiworld.com/api/v1
How routing works
- The model id is resolved, including aliases, and the key's model allowlist is checked.
- Every endpoint for the model is considered. Disabled providers, providers excluded by your preferences, endpoints that cannot honour requested parameters and endpoints above your price ceiling are dropped.
- Your own provider credentials (BYOK) come first, then explicit
order, then the provider that served your last request for this model, then the configured sort. - Credit is reserved for the worst case, the first candidate is called, and failures before the first byte fall through to the next candidate. Unused credit is released when the record settles.
The provider object
json
{
"model": "author/model",
"messages": [{ "role": "user", "content": "Hello" }],
"provider": {
"order": ["aiter-server"],
"ignore": [],
"allow_fallbacks": true,
"sort": "price",
"max_price": { "prompt": 1, "completion": 4 },
"require_parameters": true,
"data_collection": "deny"
}
}| Preference | Meaning |
|---|---|
only | Consider only these provider ids. |
ignore | Never use these provider ids. |
order | Try providers in this order before the default sort. |
allow_fallbacks | Enabled by default. When set to false, only the first candidate is tried. |
sort | price, latency (default) or throughput. |
max_price | Ceilings in USD per million tokens, per price component (prompt, completion, cached, reasoning). |
require_parameters | Skip endpoints that do not support every sampling or tool parameter in the request. |
data_collection | deny restricts routing to providers that do not train on prompts. |
cache | Opt in to the exact-match response cache; see Caching and idempotency. |
Provider ids are shown on the providers page and on each model page. A 503 no_eligible_provider response means your preferences excluded every endpoint.