How do you validate the agents? Do you have to validate them as they keep learning or as the model is updated?

Danielle Rejwan
Danielle Rejwan
  • Updated

We validate Ketryx Agents by designing test scenarios that simulate real-world use cases
and cover both typical and edge-case interactions. These tests ensure the agents provide
consistent, accurate, and reliable outputs. Our approach follows principles of good
software validation as outlined in ISO 13485 and IEC 62304, considering completeness,
traceability, and reproducibility.

Specifically, we start by defining clear requirements and acceptance criteria, selecting
representative test data, and specifying expected outcomes. We execute these tests,
document the results, and analyze any deviations. This validation process ensures that
the AI agents behave as intended, conform to applicable internal quality and compliance
standards, and maintain a high level of reliability and safety for regulated environments.

We revalidate agents when necessary, such as when a change is made to one of the
underlying models, prompts, or relevant parts of the implementation.

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.