Task episodes

Guide · Users and administrators · Applies to 5.0.194

An episode is a record of something Eldric tried: the task, the steps it took, how it ended and how that ending was checked. Eldric keeps episodes so it can recall what worked and what failed the next time a similar task comes up. This page explains what is recorded, how you confirm whether a task worked, how experiments are recorded, and how Eldric learns methods from episodes.

What is recorded

An episode is recorded when Eldric carries out a task with tools. That covers a chat turn that ran tools, a workflow run (also one started by a schedule), a sandbox run and a concluded experiment. An agent run is not recorded: agent tasks read past episodes before they are planned, but do not write one.

A chat answer that used no tool is not an episode. If such an answer did not help, use the thumbs down under the message; that goes to a separate feedback list.

Each step keeps the tool, whether it succeeded and how long it took. Argument values are kept only where the tool declares them as a fixed choice, a number or a yes/no. Free text such as a search query is replaced by a placeholder, so your content is not copied into the record.

"Did it work?"

The outcome Eldric records for a chat turn is what the tools reported. A tool that returned no error counts as a success, even when your problem is not solved. Only you know whether it was, so the chat asks.

• Under a reply that recorded an episode, the web chat shows Did it work? with Yes, No and Not sure. No lets you add a one-line reason of up to 200 characters.
• The × closes the question without recording anything.
• You are asked once per reply. An answer given on another device counts too.
• Your answer is stored beside the recorded outcome and never replaces it. Not sure is kept but counts neither way.
• Only the person who ran the turn can answer, signed in as themselves.

Seeing your episodes

Through the API, GET /api/v1/episodes lists them and GET /api/v1/episodes/<id> returns one with its steps.

Experiments

Concluding an A/B experiment records it as an episode. Experiments are an administrator feature. What the episode says is decided by the analysis you ran before concluding, not by the winner you pick.

1. Create the experiment with two variants: POST /api/v1/experiments.
2. Assign users and record events as before.
3. Analyse it: POST /api/v1/experiments/<id>/analyze with {"metric": "binary" | "continuous", "metric_key": "..."}. A yes/no result is tested with a chi-square test, a measured value with Welch's t-test. The analysis is stored, and the latest one counts.
4. Conclude it: POST /api/v1/experiments/<id>/conclude with {"winner_variant": "..."}.

What the stored analysis found	Outcome recorded
One variant is better at p < 0.05, and it is the winner you concluded (or you named none)	Worked, checked by the test
One variant is better, but you concluded the other one	Failed at the conclusion, checked by the test. It is kept as a warning.
No significant difference	Unknown: no difference found is not a failure
No test could run (not exactly two variants, or fewer than two observations each), or you concluded without analysing	Unknown, not checked

The episode has one step per variant, with its number of observations and its success rate or mean, one step for the test with its p value and the winner it found, and a last step for the winner you concluded. An unknown outcome is kept as history: it is never offered by recall and never counts towards a learned method.

How Eldric learns from episodes

When a node is idle, Eldric's learning cycle reads the episodes of each kind of task and looks for a way of doing it that keeps succeeding. Experiments with the same name and metric count as the same kind of task.

Only checked episodes count: those confirmed by a tool result, a check or a test, or by a person answering Did it work?. A model saying it succeeded counts for nothing. So do unknown outcomes, including an experiment concluded without analysing it or with no significant difference.

A method found this way is a candidate. It is offered only when you ask for something similar; see Methods that worked.

Who can see episodes

• An episode you started is private to you.
• Every signed-in user, viewer included, can read their own private episodes. Episodes shared with the tenant are visible from the developer role up.
• An episode recorded with a service key or a token, rather than by a person, belongs to the tenant. Administrators read it.
• An experiment's episode belongs to the administrator who concluded it.

For administrators

• GET|PUT /api/v1/episodes/settings holds the tenant's auto_record (on by default) and retention_days (180 by default, 1 to 3650).
• With auto_record off, nothing is recorded for the tenant, experiments included.
• Deleting an episode (DELETE /api/v1/episodes/<id>) also removes it from every method learned from it. Deleting an account removes that user's episodes; see Your data.