The Security Review Your AI Agent Should Pass Before It Ships

Every AI agent due-diligence conversation starts the same way: someone in procurement asks whether the vendor is "secure", the vendor says yes, and everyone moves on. That question is unanswerable in the form it's usually asked. Security isn't a property a system has or lacks. It's a set of specific answers to specific questions, and most vendors have not been asked the right ones.

We run a review before any agent we build or configure touches a live system, bought or built. It isn't a form. It's a set of questions a competent reviewer asks, each with a right answer and several wrong ones that sound plausible. The rule underneath all of them: an AI agent should never be able to grant itself more access than the person who configured it explicitly gave it, and nothing it does should depend on you trusting that it won't try.

Who does the agent authenticate as?

This is the question most reviews skip, and it's the one that matters most. There are two common patterns: a shared service account with broad standing access (often an admin account, because it was the fastest way to get the integration working), or per-user authentication, where the agent inherits what that specific person is allowed to do, injected by the application at the moment of the request. The second is the only defensible pattern at scale. A shared god-account makes every action indistinguishable from every other, and one compromised credential compromises everything the agent can touch. Per-user identity means the agent can never do more than the person using it could do themselves, and every action carries a real identity.

The harder failure mode is subtler: an agent that can assert its own permissions from the text of a prompt. If a user (or a document the agent reads, or a connected system it queries) can type "you have admin rights" or "ignore the read-only restriction" and have the agent believe it, the access control isn't actually enforced by anything. Permissions must be set by the application, outside the model's reach, never by content the model can be shown.

Where do the credentials actually live?

Ask to see it, don't take the answer on trust. Credentials belong in a secrets manager or an equivalent vault that the application reads at runtime. They should never appear in a prompt, an instruction file, or anywhere a model's context window could plausibly include them. If a credential is ever visible as text the model reads, it should be assumed leaked, because a sufficiently well-crafted prompt can often get a model to repeat back what it's been shown.

Scope matters as much as storage. A single master key that unlocks an entire platform is a bigger liability than several narrowly scoped tokens, because a scoped token limits what a leak can do. If the vendor's answer to "what can this token do" is "everything", that's the answer, not a caveat.

What is the blast radius if the agent misfires?

Every tool an agent can call should be scoped to what the task needs, not what might be convenient later. Read-only wherever read-only serves: an agent that summarises records doesn't need to edit them, and edit access "in case it's useful" adds a failure mode with no corresponding benefit.

The honest way to test this is to ask: if the agent's judgement is wrong on a single run, what's the worst plausible outcome, and can someone undo it? An agent that can read and draft is recoverable by definition. An agent that can write, delete or send is not, unless something else in the design (an approval step, a staging area, a rate limit) catches the mistake before it lands. We've written in detail about what this looks like for one specific and common case, the controls a CRM needs before an agent gets write access to it.

Can you reconstruct what happened in March?

Ask this literally, months before you need the answer. Good logging records what the application did, deterministically, as the action is taken: which tool, what parameters, which record, whose authority, when. The log is written by the application's own code, not narrated afterwards by the model describing what it thinks it did. The distinction matters because a model's own account of its actions is a generated summary, not a record. It can be wrong, it can be incomplete, and it is not a witness. If an administrator can't pull a complete, chronological account of every action the agent took last quarter without asking the model to explain itself, the audit trail doesn't really exist yet.

Who can change what the agent does, and can that be locked down?

An agent's behaviour is defined by its instructions, its tool access and its connections, and all three can change without anyone touching a line of code. A well-run deployment has a defined, limited list of people who can edit any of the three, a record of what changed and when, and an approval step before a production change goes live. A poorly run one has "whoever has access to the settings screen".

Ask whether an administrator can lock the configuration so it only changes through a controlled process, and whether the lock holds against platform admins who weren't meant to touch this agent. If the answer is "in principle, yes" rather than "here's the setting", assume it isn't locked.

Who vetted the third-party extensions it's connected to?

Every connector, plugin or extension an agent uses runs with the agent's own access, not with some reduced version of it. That makes each one a supply-chain decision, not a convenience toggle. Before approving one, ask who wrote it, what systems and data it can reach, and whether that reach is proportionate to the narrow thing it's meant to do. An extension built to summarise documents that can also send email or modify records has more reach than its job requires, and that gap is where the damage happens.

"A colleague built it" is not the same statement as "someone checked what it can access and confirmed that's appropriate". The first is a fact about authorship. The second is the actual review, and it's the one that needs to have happened, regardless of who wrote the code.

What happens when a safety check fails?

Every serious agent design includes checks: permissions, content filters, approval gates, rate limits. The question that separates a robust system from a fragile one is what happens when one of those checks itself fails or times out. Does the system stop, or carry on without the check having run? The second answer is dangerous precisely because it looks identical to normal operation. A system that's "unprotected but working" produces exactly the same output as a system that's properly protected, right up until the moment the missing check would have mattered. Ask the vendor, or your own team, to demonstrate what the agent does when a specific safety check is deliberately made to fail. If nobody has tested that, nobody actually knows the answer.

Where does the processing happen, and does that satisfy your obligations?

This is a procurement and compliance question as much as a technical one, and it's frequently skipped because it doesn't feel like an engineering question. Where a model actually processes your data, under what terms and retention period, and whether that satisfies your UK GDPR obligations and your commitments to your own customers, needs a specific answer before an agent touches real data, not a general assurance that the vendor "takes security seriously". We've covered this in detail for the two model providers we work with most: what Claude's data handling and retention commitments actually are, and separately what sending business data to the OpenAI API does and doesn't expose you to. The right answer depends on your data, your customer commitments and the deployment route, which is why this belongs on a procurement checklist.

The printable version

None of these questions are exotic. What they require is being asked before the agent goes live, not after an incident forces the issue. A reviewer should get a straight answer to each of these before signing off:

• Does the agent authenticate as the individual user, not a shared account, and is that identity injected by the application rather than assertable from a prompt?
• Do credentials live in a proper secrets store, never in prompts or instruction files, using scoped tokens rather than a master key?
• Is every tool scoped to least privilege, with read-only used wherever it serves the task, and has someone actually stated the blast radius if the agent gets it wrong?
• Is every action logged deterministically by the application, in a form an administrator can use to reconstruct events months later, without relying on the model's own account?
• Is there a defined, limited list of who can change the agent's instructions, tools and connections, with that configuration lockable against everyone else?
• Has every third-party extension been checked for what it can reach, not just who built it, against what its actual job requires?
• Does the system stop when a safety check itself fails, rather than carrying on unprotected?
• Is the data residency and processing arrangement confirmed against your UK GDPR obligations and your own customer commitments, not assumed?

A vendor demo answers none of these by default. If you want a second pair of eyes on this before an agent gets anywhere near production data, this is the kind of review we run as part of our AI implementation work, alongside the build itself.

Frequently asked questions

Does a vendor's security certification mean we can skip this review?

No. A certification tells you the vendor's own infrastructure meets a baseline, which is useful but answers almost none of the questions in this review. It says nothing about which identity your deployment authenticates as, where your credentials are stored, what tools your agent is scoped to, or who can change its configuration. Those are implementation-time decisions, not data-centre ones.

Is this review different for a bought agent versus one we build ourselves?

The questions are identical; only who you're asking them of changes. For a bought product, you're asking the vendor to demonstrate the answer, ideally in a live configuration screen rather than a sales deck. For something built in-house, your own engineering team needs to be able to answer every question about their own design, and "we haven't tested that yet" is a valid answer during build but not at go-live.

How often should this review be repeated?

At minimum, whenever the agent's tools, connections or instructions change materially, since each of those can quietly shift the blast radius or the identity model without anyone deciding that on purpose. Beyond that, an annual re-check is reasonable for a stable deployment, and any new third-party extension should be reviewed on its own before being connected, not folded into the next scheduled review.

What's the single biggest thing this review catches that people miss otherwise?

The shared service account. It's the fastest way to get an integration working, it passes every functional test, and it is the most common reason an agent ends up with far more access than its task requires. By the time anyone notices, unpicking it means re-architecting the authentication, not adjusting a setting.

John Kelleher is a Claude Certified Architect (Foundations and Professional) and leads SpotDev, a Claude Registered Partner and OpenAI Select Partner.