Start with familiar questions
Use questions your customers or colleagues actually ask. Include exceptions: when should the agent stop or involve a person?
Testing agents
Check whether your agent answers correctly.
Save test questions with the answers and, optionally, tool steps you expect. Run them again after changing knowledge or instructions. In Qosie, these checks are called evaluations.
How it works · example
Does the answer about returns still hold up?
Your test question
“Can I return an item after 40 days?” Save the expected answer based on your return policy.
What Qosie does
The agent answers the test question. The evaluation compares its response with the expected answer and configured criteria.
What you review
Inspect the score and answer. If they differ from what you expected, check the source or instruction before using the change more widely.
Example: add your own return policy and test cases. A test score helps with review, but does not guarantee every future answer.
Repeat after a change
Test the same questions when you change instructions, the knowledge base or available tools. Spot when a previously correct answer no longer holds up.
Bring a few frequently asked questions. We can show you how to turn them into your first test set.
Discuss your test questions