Most discussion of AI risk concerns what a model might say. For a business deploying automation, that is the less interesting problem. The risks that matter concern what the system can reach and what it can do.
Here are the ones that materialise in practice, and what addresses each.
1. Permission inheritance
An AI system operates with the permissions it is given. If it runs under a user’s identity, it can reach everything that user can reach — including material they could technically access but were never expected to find.
This is why over-shared SharePoint environments become a problem the moment a knowledge assistant is introduced. Permissions that were broad for convenience were tolerable when finding a file required knowing it existed. An assistant that searches everything the user is entitled to see removes that practical obscurity in a single step.
Organisations deploy an assistant, and within a week someone has surfaced the salary review spreadsheet or the redundancy planning document. The assistant did nothing wrong. It exposed a permission structure that had been wrong for years.
What addresses it: fix permissions before deployment, not after. Review site and library access, repair broken inheritance, remove standing access nobody needs. This is ordinary Microsoft 365 hygiene, and it is a prerequisite rather than an optional extra.
2. Standing access with no review
An automation is built for a project, given the permissions it needs, and left running. Two years later it still holds those permissions, nobody has reviewed them, the credentials have not been rotated, and the person who built it has moved on.
This is the AI-era equivalent of the service account with domain administrator rights that everyone is afraid to touch.
What addresses it: treat automation identities as first-class accounts. They go in your access review cycle, their permissions are documented alongside the business justification, credentials rotate, and there is a named owner. An automation nobody owns should be turned off.
3. Prompt injection
Instructions concealed in content the AI reads, intended to make it act against your interests. A line of white text in a supplied PDF. An instruction embedded in an email signature. Text on a web page the agent has been asked to summarise.
It matters most for systems that both read untrusted input and can take actions — which describes most useful automation. An agent processing supplier invoices reads documents from outside your organisation by definition.
The uncomfortable part is that this is not fully solvable at the model layer. Filtering and instruction hierarchies help and are worth having, but treating them as sufficient is a mistake.
What addresses it: capability limits. If the agent can only read a specific mailbox and write to a specific staging table, an injection instructing it to email your customer list cannot succeed, because it has no route to do so. Design on the assumption that an injection will eventually work, and make sure the blast radius is bounded. Human approval on consequential actions is the second layer.
4. Data leaving your control
Where does the content actually go? Which provider, in which region, under what retention, and can it be used to train a model?
Consumer and business tiers of the same product frequently differ substantially. This is not a reason to avoid these services — it is a reason to read what you are agreeing to and to make sure staff are using the tier you signed up for rather than a free account with the same logo.
What addresses it: vendor assessment before adoption, covering data location, retention, training use, sub-processors and exit. Access through your identity provider so you know which tier is in use. Data-category rules so staff know what must never go anywhere near it.
5. Silent failure
An agent stops working. Nothing alerts. Three weeks later someone notices that invoices have not been processed since the eleventh.
This is not a security failure in the conventional sense, but it is an operational one with security consequences: the manual process has atrophied, nobody remembers how to do it, and the backlog is now substantial.
A subtler variant is quality drift. The agent still runs, but an upstream format change means it is extracting the wrong field. It fails silently and confidently, and the errors accumulate.
What addresses it: monitor volume, exception rate and latency, and alert on anomalies in all three. Sample outputs periodically against a known-good baseline. Keep the manual process documented and occasionally exercised.
6. Confused deputy
The agent has more authority than the person asking it to do something. A user who cannot approve a purchase asks the agent to process one, and the agent — running with elevated permissions — does it.
This arises whenever an automation’s own permissions exceed those of everyone who can invoke it, which is a common design shortcut.
What addresses it: the agent must check the requesting user’s authority, not just its own. Where that is impractical, restrict who can invoke it to people who hold the underlying authority anyway.
7. Logging that will not answer the question
Six months after deployment, a customer disputes something the automation did. Can you reconstruct what happened — what input it received, what it extracted, what it decided, who approved it?
If the answer is no, you have an unfalsifiable system, and that is a poor position in front of a customer, an auditor or a regulator.
What addresses it: log inputs, tool calls, outputs and approvals, and retain them in line with the underlying business record — which for financial processes is usually years rather than weeks. Test the reconstruction before you need it.
8. Over-trust
The most insidious risk, and the least technical. A system that is right nine times out of ten trains people to stop checking. The tenth case then passes through unexamined, and it is disproportionately likely to be the unusual one that mattered.
What addresses it: design the human step so it is meaningful rather than ceremonial. Surface the agent’s confidence and the source of its extraction, so the reviewer can see what to scrutinise. Sample and audit. Be wary of approval steps that consist of clicking “confirm” on a screen showing nothing but a total.
The pattern
Almost every control above reduces to the same three principles.
Least privilege. The narrowest access that lets the job be done, granted through identity, reviewable and revocable.
Bounded consequence. Assume it will be wrong or manipulated at some point, and make sure the worst outcome is recoverable. Human approval on anything that commits the business.
Evidence. A record complete enough to reconstruct any individual case, retained for as long as the business record it relates to.
None of this is novel security thinking. It is the same discipline applied to a new kind of actor — one that works quickly, at volume, and does not get tired enough to notice something is wrong.