Cloud provider keys
Cloud models (OpenAI, Anthropic, HuggingFace and other providers) need a provider API key. From 5.0.196 the key is kept on the controllers, and the node that runs the cloud role reads it from there. This page says how to set a key, what happens when the controllers cannot be reached, and how to move keys that older releases kept on a cloud node.
How keys work
- A superadmin adds a cloud backend in the admin console (cloud dashboard) or with
POST /api/v1/cloud/backendsand itsapi_key. An administrator who is not a superadmin gets403 superadmin_required. - The key goes to the controllers' credential store, sealed, and every controller holds the same copy. The cloud node keeps only a reference and reads the key into memory every 30 seconds.
- The API never returns a key.
GET /api/v1/cloud/backendsshows where it comes from,key_source(store,local_legacyornone), and its state,key_state(ok,stale,unreachableormissing). - If the controllers cannot be reached, a running cloud node keeps the last key it read. After a restart without them, the backend answers
503 backend_unavailable(reasoncredential_store_unreachable, retryable) until a controller answers. The node never falls back to an environment variable or a file.
Keys from before 5.0.196
Older releases kept the key on the cloud node, sealed in its own store or in an environment variable named by api_key_env. Those keys keep working after the update and show key_source: local_legacy. To move them, a superadmin runs POST /api/v1/cloud/keys/import once on each cloud node. It moves the keys into the credential store, removes the sealed copy from the node and stops reading the variable. A key already in the store is not overwritten.
GET /api/v1/cloud/backends/unconfigured-keys (administrators) lists provider-key variables in a node's environment, by name only, never the value. key_without_backend: the variable is set but no backend uses it; add the backend in the admin console, which stores its key on the controllers, then remove the variable. env_no_longer_read: the key is in the store now and the variable is no longer read; remove it, and delete backup copies of the environment file that still contain it. The list changes nothing by itself.
Check a backend
GET /api/v1/cloud/backendson the cloud node:key_source: storeandkey_state: okis the healthy answer.- Send a short test request (
max_tokens: 1) for one of the backend's models through your normal entry point. - An error from the provider itself (for example missing credit, or a model the provider does not offer) proves the key reached the provider.
503 backend_unavailablewithcredential_store_unreachablemeans the node could not read it from the controllers.
Whose key is used
Today there are no per-tenant or per-user provider keys. Every user who can use a cloud backend uses the key the administrator entered, and the usage is billed to that provider account.