How Do You Test an AI Agent and Give It Feedback?
If we think about how we onboard a really competent new hire, the level of instruction we would provide, we'd want to do something similar for our agents.
Don't say try again, actually say, "Hey, you surfaced this and this isn't beneficial and this is why."
Run a test
The next thing to do with our agent is run a test.
Hear this at 46:08 in the recording.
What to say when the result comes back
Run a test is where we're allowing our agent to then think and provide us with our first result to which we should give feedback. So if your agent comes back and it gives you a result that you love, tell it why you love the results. If your agent comes back and it gives you a result and you have mixed feelings about it, then you'll tell it, "Hey, I like this, but I don't like that," and tell it why. If you absolutely don't like the results, also tell the agent why.
Hear this at 46:15 in the recording.
You can read the reasoning behind a result
As it's reasoning, you'll see exactly what it's doing. You can double-click into those later into the reasoning and review that. That's one of the reasons why I like building in this platform. You can build agents inside of ChatGPT and Claude. They don't always allow you to click into the reasoning. It may show you the reasoning, but I can actually read the source that it pulled from.
Hear this at 38:47 in the recording.
Where this comes from
Every sentence above is Jacqueline V. Twillie's own, spoken live with Women Applying AI on October 7, 2026. The headings, questions, timestamps and links are the only additions. Filler words are cut, and a passage that skips a few seconds starts at the timestamp shown.
Build Your First AI Agent in One Hour: the full build, with the clips, the prompts and the transcript. This article's clip is Run a test, then give feedback with reasons.
Related reading: What Does Run a Test Mean for an AI Agent?, How to Onboard an AI Agent Like a New Hire, How Often Should You Review an AI Agent's Instructions?.
Frequently asked questions
Why give an AI agent guardrails?
One of the things that I like to do with my agents is I give them guardrails. I tell them what they can do and what they can't do. And this is just as important of telling your agent what you want it to achieve.
Build something that actually runs your workflow.
A focused, free 60-minute live session with Jacqueline. You build alongside her, on your own real task, and leave with an AI agent that is already running.