Snapshot tests caught a regression in my agent that the unit tests missed
A developer has created AgentSnap, a testing tool designed to catch regressions in AI agents that traditional unit tests might miss. AgentSnap captures the sequence and arguments of tool calls made by an agent, creating a snapshot that can be compared against future runs. This approach proved effective in identifying a bug where a model update caused an agent to incorrectly reorder arguments for a `find_slot` function, leading to booking errors that were not detected by existing tests. The tool supports multiple runtimes and allows for redaction of volatile fields to handle LLM non-determinism. AI
IMPACT Provides a novel testing method for AI agents, helping developers catch subtle regressions missed by traditional tests.