Test cases
Test cases are questions your agent must answer well, each with what a good answer needs. Run them against the live agent to check a change before you share it, or have them run after every new version, so an answer you fixed stays fixed.
Open the agent’s page → Test cases. Owners and builders add, edit and run them; everyone in the workspace can see them and their results.
Add a test case
Section titled “Add a test case”Click Test case, type the question, and choose what a good answer needs:
- Must contain: text the answer must include, for example 14 days.
- Must not contain: text that must not appear, for example guarantee.
- Judged by a rubric: what a good answer does, in your own words, for example “Explains the 14-day return window politely and offers to start a return.” An AI judge rates the answer against it, the same way as quality checks.
For must contain and must not contain, upper and lower case don’t matter, and neither do Arabic diacritics or the different forms of alef. So سياسة الارجاع matches سِيَاسَة الإرجاع.
An agent can have up to 20 test cases.
You can also turn a real question into a test case from the Inbox: under an answer, click Add as test case.
Run them
Section titled “Run them”- Run all asks the live version of the agent every question, each in its own conversation, and checks every answer. Before it starts, backcrew shows an estimate of what the run will cost.
- Run the test cases after each deploy: turn this on to run them automatically whenever a new version goes live (an update, or a rollback to an earlier version). If the workspace has no credits, the run doesn’t start, and the agent’s activity says why.
A rubric case passes only when the judge rates the answer Met. Partly met fails, with the judge’s reason, so you can see what to fix.
If the credits run out part-way, the run stops, and the cases already checked keep their results.
Read the results
Section titled “Read the results”Each case shows its latest result: passed or failed, the agent’s answer, and why. Each version keeps its own score, for example v3: 5/5 · v2: 4/5, so you can see whether a change made things better or worse. To go back to a version that scored better, see Versions and rollback.
What it costs
Section titled “What it costs”- Each answer is charged like any other request to the agent.
- Each rubric rating costs 2 credits, the same as a quality check. Must contain and must not contain are checked for free.
See Credits.
The audit log records who added, changed, removed or ran test cases, without the questions or what a good answer needs.