Questions to Ask Before You Buy an AI Agent Tool
Five questions to bring to a purchase or a renewal, each tied to something a leader said at a November 2026 summit.
Ask the vendor five things: what notice you get when AI agents ship in the product, what evidence comes with each answer, how you see cost per workflow, who pays when the model changes, and what the off switch is. BNEDai built the questions from remarks at an online enterprise AI summit on November 12, 2026.
BNEDai built the five questions below from remarks leaders made at an online enterprise AI summit on November 12, 2026. Each remark is reported beside the question it led to.
Ask what notice you get when AI agents ship
Jenn Kosar, PwC's US AI Assurance Leader, said there is shadow AI she worries about more: AI agents showing up in the vendor platforms you already use, in ways that may not be transparent to your organization. The controls, she said, are what procurement and vendor risk management have always done, as long as the questions get asked. She listed notification, transparency, the right to audit, and audit and evidence logs with the access you have to them. If you cannot get that confidence from a vendor, she said, calibrate the autonomy you give the AI agents they deploy.
BNEDai's question: what notice do we get, what can we audit, and where are the logs?
Ask what evidence comes with each answer
Shawn Edwards, Chief Technology Officer at Bloomberg, said one of Bloomberg's principles is that they never want the model to generate a fact or a number. Answers are grounded in more deterministic systems and their sources of truth, and the user sees the paragraph or two from the source, the query and the API calls. He added that this mindset is needed for finance and maybe not for other uses of AI.
Prashanthi Padmanabhan, VP of Engineering for Talent Solutions at LinkedIn, said customers are not going to trust AI agents by default. LinkedIn's hiring assistant says, for every candidate it recommends, why that person fits. LinkedIn calls it "show your work."
BNEDai's question: when this tool gives an answer, what do I see behind it, and where would my team check it?
Ask how you see cost per workflow
Alyssa Cox, VP of Transformation at Cardinal Health, said tracking token use for a particular solution takes observability across your systems, down to which query a workflow is making. Old business cases tracked headcount reductions, she said, and the ongoing cost was whatever the software fees were.
BNEDai's question: can I see usage for each workflow, and who reads that report?
Ask who pays when the model changes
Cox said that when the platform underneath changes its infrastructure or moves from one model to the next, the vendor does not fund Cardinal's side of the transition. Cardinal has to, and her team estimates that ongoing support and puts it in the business case.
BNEDai's question: when you move to a new model, what changes on our side, and what does it cost us?
Ask how it gets turned off
Kosar said to design an off switch in advance, with the automations or interventions that would turn off an AI agent that starts performing outside its boundaries. She also described expiration or re-approval dates that trigger an accountable owner to confirm an AI agent is still useful.
BNEDai's question: how does an AI agent in this product get turned off, who can do it, and how fast?
Bring them to the next renewal
BNEDai suggests bringing these to the next renewal conversation and writing down the answers. The page on what AI agents are already running inside the software you pay for goes further on the first question. The costs behind the third and fourth are in what an AI agent costs after it launches.
Frequently asked questions
Who wrote the five questions?
BNEDai wrote them, as an application of remarks by leaders at Cardinal Health, Bloomberg, LinkedIn and PwC.
What did Bloomberg's Shawn Edwards say about facts and numbers?
He said they never want the model to generate a fact or a number. Answers are grounded in more deterministic systems and sources of truth, and the user sees the source, the query and the API calls.
What did PwC's Jenn Kosar say about the off switch?
She said to design one in advance, with the automations or interventions that would turn off an AI agent performing outside its boundaries.
Source: remarks by Jenn Kosar, PwC's US AI Assurance Leader; Shawn Edwards, Chief Technology Officer, Bloomberg; Prashanthi Padmanabhan, VP of Engineering for Talent Solutions, LinkedIn; and Alyssa Cox, VP of Transformation, Cardinal Health, at an online enterprise AI summit, November 12, 2026.
Reviewed on November 20, 2026.