Model types
Eldric serves more than chat models. This page lists every kind of model the shipped release can serve, the file it comes in, the route that answers and what it refuses. Each section names what we read in the source or measured.
Chat (large language models)
A chat model is a GGUF file in the node's model directory, run by the llama.cpp engine built into Eldric. Clients call POST /v1/chat/completions; with "stream": true the answer arrives token by token as server-sent events. How the files get there, how much context a model gets and what happens when memory runs out is on Local models. Cloud models answer on the same route, see Cloud models.
Tool calling
Whether a local model can call tools depends on its chat template, the part of the GGUF that says how a conversation is laid out. Eldric reads the template when it lists the model and recognises these tool formats: qwen_xml, hermes_json, harmony, llama3_json, mistral_nemo, mistral_v7, mistral_v7_call_id and gemma4. The model's row in /v1/models carries the result as tool_format and tool_call_supported.
A template that teaches none of them gets tool_format: "none". If a request offers tools to such a model, Eldric refuses it with 422 model_cannot_tool_call and the message "this model's chat template teaches no tool-call format, so the offered tools cannot reach it; choose a model that can call tools". Until 5.0.194 the tools were dropped without a word and the model answered in text as if it had looked something up. In the 5.0.194 release check on a node with a 12 GB card, llama3-8b-Q4_0 was refused this way and llama3.1-8b returned a real tool call.
A request that sets "tool_choice": "none" counts as offering no tools and is not refused.
Vision
A vision model is two GGUF files: the language model and its projector, the part that turns an image into something the model can read. Eldric pairs them by name only: call the projector <model>.mmproj.gguf or mmproj-<model>.gguf. A projector that matches no model is named in the log with the names it would accept. A paired model shows vision_supported: true in /v1/models. If its projector cannot be loaded, the whole model fails to load rather than loading blind.
Images go into an ordinary chat message as image parts. The image is read on the same node, in the same process; nothing is sent to a separate reader. What can come back:
| Answer | Meaning |
|---|---|
422 model_cannot_see_image | The model has no projector, or this build has no image support. |
image_not_decodable, image_too_large, images_too_large, too_many_images | The image could not be read or is over the limits. |
remote_image_not_fetched, image_url_not_fetchable | An image given as a URL was not fetched. |
image_tokenize_failed | The projector could not turn the image into input for the model. |
The same projector can also turn an image into a vector without a chat, see encode below.
Embeddings
POST /v1/embeddings turns text into vectors, which is what knowledge bases and memory search on. A local GGUF answers it through the engine. Eldric uses the pooling the model declares (for example CLS for bge-m3, mean for e5) and returns each vector scaled to length 1, so cosine similarity is a plain dot product. Each input is embedded on its own, with up to 4096 tokens.
Two things to know. A model behind a local server such as Ollama that cannot chat is marked no_chat_backend and is served for embeddings only (from 5.0.188). From 5.0.196 Eldric reads from a GGUF file what the model is for. An encoder-only model, or one whose file names a pooling type for embeddings, is kind: "embedding"; a model built for reranking is kind: "rerank"; both have supports_chat: false. The chat's model picker and the agent designer leave them out. A client that names such a model in /v1/chat/completions directly is not refused yet, and does not get a useful answer. How Eldric uses embeddings for knowledge is on How Eldric knows things.
Retrieve, forecast, policy, encode
Four smaller workloads run in every Eldric node without a language model. GET /api/v1/xlstm/status shows, per workload, what serves it on the node you ask.
| Workload | Route | What it does | Served by |
|---|---|---|---|
| retrieve | /api/v1/retrieve/{load,store,query} | Stores vectors in a compressed memory and recalls the closest; each answer gives its recall time. Loading and storing are for administrators. | native-matrixmemory |
| forecast | POST /api/v1/forecast/predict | Continues a series from a forecaster file you name. | a native forecaster |
| policy | POST /api/v1/policy/execute | Turns an observation into an action with an N-slot model (see World models). | nslot-cpu-dense |
| encode | POST /api/v1/encode/embed | Turns an image into a vector with a vision projector. | native-clip, or native_encoder_not_in_build |
Encode needs a projector named in ELDRIC_INFERENCE_ENCODE_MMPROJ; without one it answers 503 no_vision_tower_configured, and the status says so. Choosing another projector in the request is for administrators.
Forecasting your data
A user can attach a file of readings, such as an electricity meter export, and ask in the chat for a forecast. The chat calls POST /api/v1/forecast/by-reference with the attachment, and Eldric reads the file itself: CSV or other text, .xlsx or .ods. .xls and Apple Numbers files are refused with a request to export as .xlsx or CSV. Only the caller's own file is read.
The forecaster is a .exmm file (see EXMM) that an administrator names in the node's forecast_model setting, next to the limits file that comes with the model. Without one the answer is 503 forecaster_unavailable. The steps, the answers and the refusals are in the help topic inference.forecast-your-data; continuous series that grow over time are in inference.forecast-streams.
World models
A world model is a small file that steps a simulation forward on the processor, distilled from a simulator. Eldric reads it as an N-slot file, .ensl; the older name .nsl loads too, because Eldric checks the file's content, not its name. The world-model routes are POST /api/v1/worldmodel/rollout, /serve, /seismic and /seismic/simulate. The model store keeps .ensl files in <data>/worldmodel.
The router's tool index
Before a chat message reaches a model, the router looks it up in an index of what each tool is for and offers the model only the tools whose purpose matches. That keeps the list short enough for a model to choose from. The index is a file the router loads at start; it is not a language model. How this fits into a whole chat turn is on How Eldric reasons.
From 5.0.182 no separate classifier model ships for the router; a node logs "no legacy .enrn on this node (expected since 5.0.182)". An administrator can still load one with PUT /api/v1/router/xlstm-model, and the router then uses it only for its own classify routes when no index is loaded.