What Should an Executive Sponsor Ask After a Team's First AI Agent Test?

You're the executive sponsor. A team has run its first AI agent test and is asking you to approve the next step. The six questions below come from what I showed and said in the hour: what the AI agent was told, what it can do, and what happens to its access. A team can answer each by showing a screen or the words they used. They restate what I said, and they don't tell you what to decide.

1. What did you tell it, and why?

In the build I gave the AI agent its goals before the first test (12:02). I also said that I give my AI agents guardrails, what they can and can't do:

From the recording, 41:26. One of the things that I like to do with my agents is I give them guardrails.

From the recording, 41:35. And this is just as important of telling your agent what you want it to achieve.

Ask to see the goals and the guardrails in the team's words. For the definition, see What Are AI Agent Guardrails?.

2. Can I see how it reasoned?

Ask the team to open the reasoning. I showed that this is possible while the AI agent was working:

From the recording, 38:47. As it's reasoning, you'll see exactly what it's doing. You can double-click into those later into the reasoning and review that.

I added that I can read the source it pulled from (39:07). See Can I See How My AI Agent Reached Its Answer?.

3. What can it send, and what can it only read?

The permission screen separates the two. Under tool management I selected the read-only tools, and below them the write and delete tools:

From the recording, 44:19. I can then give permission or restrict access to sending emails, updating, creating drafts, forwarding emails, et cetera.

Ask for the list: which tools are read-only, which can write or send, and which the team switched off. For the control itself, see How Do You Control What an AI Agent Can Do in Your Gmail?.

4. How is its access removed?

Ask the team to show you. In the connectors section I showed three dots beside a connected account:

From the recording, 45:04. If I click on the three dots, it allows me to switch an account, but it also gives me access to easily remove access.

I also said that if you change your password in a connected system, the AI agent can no longer reach it until you come back and reauthorize the permission (45:28). Access ends two ways: someone removes it, or a password changes.

5. When are its instructions reviewed?

An AI agent that updates its own instructions needs a person who reads them. I set a recurring reminder:

From the recording, 24:51. I have a 15 minute meeting that reoccurs every three weeks on my calendar where I jump in and I review my agent instructions

Most of my AI agents run a couple of times a week, so a three week cadence works for me (25:15). If you use an AI agent daily, I told the group you may want to check every two weeks (25:10). Ask what the team's cadence is and who owns the calendar entry. See How Often Should You Review an AI Agent's Instructions?.

6. What did you change after the first result?

The first result isn't the finish line. I told the group to give feedback with reasons, whatever they thought of the result, and not to say try again:

From the recording, 46:47. So don't say try again, actually say, "Hey, you surfaced this and this isn't beneficial and this is why."

Ask what the team told the AI agent after the first run, and what changed in the next one. For the test itself, see How Do You Test an AI Agent and Give It Feedback?.

The one-page review sheet

Hand this to the team before the meeting. The answers stay blank until they fill them in.

| Question | Where the answer sits | The team's answer | |---|---|---| | What did you tell it, and why? | The goals and guardrails (41:26) | | | Can I see how it reasoned? | The reasoning, opened after a run (38:47) | | | What can it send, and what can it only read? | Tool management, read-only and write/delete (44:19) | | | How is its access removed? | The three dots beside a connected account (45:04) | | | When are its instructions reviewed? | A recurring reminder and a named owner (24:51) | | | What did you change after the first result? | The feedback given after the test (46:47) | |

What goes wrong

A team that gives feedback without reasons. The advice was not to say try again; say what you surfaced and why it didn't help (46:47).

Access that doesn't update itself. If a password changes in a connected system, the AI agent can no longer reach it, and I said it won't automatically update once the password is changed (45:57).

The recording doesn't show a team's results, so I'm making no claim about what a first test produces.

Where this comes from

This article grows from the session I taught live with Women Applying AI on October 7, 2026. The quoted passages are my own words, taken verbatim from the recording. The steps, tables and checklists around them were drafted by my team's AI agents from those words and my published pages, and checked against the recording before publishing. The six questions and the review sheet were written from what I showed and said.

Build Your First AI Agent in One Hour: the full session with the clips, the prompts and the transcript.

Related reading: How Do You Test an AI Agent and Give It Feedback?, How Often Should You Review an AI Agent's Instructions?, Delegating Execution to an AI Agent Without Delegating Responsibility.