An AI agent release checklist for behavior changes.

Use this checklist when a prompt, model, tool, retrieval source, memory policy, or orchestration change is ready for review. It is intentionally advisory: the team decides whether to release, review, or hold.

Start with one concrete change and one clear decision. Fill in the evidence as you review it; do not wait until the end to record missing cases or unresolved questions.

Before comparing versions

• Write the intended outcomes, capabilities, boundaries, and non-goals.
• Name the baseline and candidate versions or configurations.
• List the workflows, tools, cohorts, and failure modes the change could affect.
• Identify cases with no representative evidence.

Review behavior and evidence

• Compare decisions and failure modes, not only aggregate scores.
• Inspect representative traces or bounded evidence for the largest shifts.
• Record affected workflows and users where evidence supports that conclusion.
• Separate observed, missing, unknown, and errored cases.

Decide what to ship

• Confirm important behavior improved, held steady, or regressed.
• State what remains unproven and why it matters.
• Choose one: recommend release, review required, or recommend holding.

What remains unproven

No checklist can establish universal safety or cover every future interaction. Use it to make uncertainty visible and keep the decision accountable.

Read next

Learn the underlying framework in AI agent evaluation or study the refund-agent behavior diff.