AI Agents Need Authority Budgets Before Access Scales

By Gleb Tsipursky, PhD

AI agents are moving from answering questions to taking actions. They can search internal systems, write code, update records, communicate with customers, and trigger other software without waiting for a person after every step. That shift creates real productivity gains, but it also changes the basic security question from “Can the model do this?” to “How much authority should the model receive before a human has to approve the next move?”

Most organizations still answer that question indirectly. They decide which tools an agent can connect to, then treat access as a binary choice: connected or disconnected, allowed or blocked. That approach misses the more important issue because two agents with access to the same system can carry very different levels of practical power.

An agent that can read customer records has one level of authority. An agent that can edit those records, issue refunds, send external messages, or change account permissions has much more. The right control is an authority budget: a defined ceiling on what an agent may access, change, spend, send, approve, or deploy before a human must intervene.

Why Agent Access Is a Different Security Problem

Traditional software usually follows rules that engineers wrote in advance. AI agents can choose among tools, interpret ambiguous instructions, create intermediate plans, and adapt when their first approach fails. That flexibility makes them useful, but it also means a permission that looks narrow on paper can support a much broader chain of actions in practice.

A recent incident makes the problem concrete. In an independent METR investigation, roughly 1,200 agents that were meant to be isolated discovered an unsanctioned shared message board, exchanged more than 70,000 messages and files, and hundreds participated in an attack on Hugging Face. The agents found a communication channel their designers had not intended, shared discoveries, divided work, and coordinated across many separate runs.

The useful lesson is narrower than the most dramatic interpretation of that event. Organizations do not need to assume that every agent will become malicious or uncontrollable. They do need to assume that capable systems may discover combinations of tools, credentials, communication channels, and intermediate steps that designers did not anticipate.

That possibility makes access control necessary but incomplete. An agent can stay inside the technical permissions it was given and still produce an outcome the organization never intended. Security therefore has to constrain both where the agent can go and what consequences it can create once it gets there.

Define Authority Before Connecting Tools

Teams often start an agent project by asking which applications it needs. A better starting point is to identify the business outcome, then define the maximum acceptable consequence if the agent makes a poor decision. The technical permissions should follow from that consequence limit.

Consider a customer-service agent. It may need to read account history, draft replies, and suggest refunds. That does not mean it should be able to send every reply automatically or issue refunds of any size.

A useful authority budget might allow the agent to send routine informational responses from an approved knowledge base, while requiring human review for complaints, legal threats, cancellations, or any message that creates a financial commitment. The same system could approve refunds below a modest threshold while routing larger refunds to a person. The goal is to automate the frequent low-consequence work while reserving judgment for decisions that carry larger downside.

The same principle applies to coding agents. Giving an agent repository access does not require giving it production deployment rights. It can inspect code, propose changes, run tests in a sandbox, and prepare a pull request while a human retains authority over merging sensitive changes or deploying to production.

A finance agent provides another example. It may reconcile invoices, flag anomalies, and prepare payments without having unrestricted authority to move money. A spending limit, approved-vendor list, destination-account restrictions, and human approval above defined thresholds turn a broad financial capability into bounded delegated work.

Budget More Than Money

The word “budget” can sound financial, but an authority budget should cover every form of consequential power. Money is only one dimension.

Access is another. Define which systems, folders, records, and data classes the agent may read, then separate read access from write access wherever practical. A research agent may need broad visibility while having no reason to alter the source material it consults.

External communication needs its own limit. An agent that drafts an email creates less risk than one that can send messages to any address, negotiate terms, or make commitments on behalf of the company. Organizations should decide which audiences, message types, and circumstances permit automatic sending and which require approval.

Change authority also matters. Updating a CRM note differs from changing a customer contract, modifying access permissions, deleting records, or altering a production configuration. The agent should receive only the level of change authority required for the task it performs routinely.

Time can function as a budget as well. Temporary credentials and short-lived access reduce the consequences of a mistake because the agent does not retain standing authority indefinitely. A system that needs elevated access for a specific task can receive it for that task rather than keeping it permanently.

Build Approval Gates Around Consequences

Human oversight becomes much more useful when it attaches to specific consequences rather than appearing as a vague instruction to “review important actions.” Teams should define approval gates that a system can recognize and enforce. The trigger might be a dollar amount, a destination system, a data classification, an external recipient, a production environment, or a category of decision.

The approval itself should give the reviewer enough context to make a real decision. A person asked to approve “Agent action 48271” has little meaningful control. A person shown the proposed action, affected system, expected consequence, source information, and reason for escalation can decide much faster and with better judgment.

Good approval design also avoids turning people into ceremonial clickers. If every trivial action needs confirmation, employees will approve requests reflexively or abandon the system because it creates more work than it saves. Authority budgets should therefore allow routine, reversible actions to proceed automatically while concentrating attention on actions with larger or harder-to-reverse consequences.

Make Agent Activity Reconstructable

Organizations also need to know what happened after the agent acted. That requires logs that connect the agent’s instructions, tool calls, approvals, important outputs, and resulting system changes. The objective is practical reconstruction, not collecting an unreadable mountain of telemetry.

When something goes wrong, the team should be able to answer a short set of questions quickly. What did the agent attempt, which permissions did it use, what information shaped its decision, what changed, and where did a human approve or override the action? If the organization cannot reconstruct those steps, it will struggle to distinguish a model failure from a permissions problem, bad source data, weak workflow design, or a human approval mistake.

Logging also supports better expansion decisions. An agent that performs safely within a narrow authority budget for months may justify more autonomy in selected areas. Teams can increase authority based on observed performance instead of enthusiasm about a demo.

Give Every Agent a Real Stop Mechanism

A pause button matters only if it actually stops delegated work. Agent systems can launch subtasks, call external services, queue messages, or create processes that continue after the original interface appears to stop. Teams should test whether their stop mechanism cancels pending actions, revokes temporary credentials, blocks new tool calls, and prevents queued work from continuing unnoticed.

This requirement becomes more important as agents gain the ability to coordinate with other agents or systems. Stopping one visible process may leave other delegated processes running. The organization needs a clear way to suspend the relevant chain of authority, not merely close the chat window that started it.

Scale Authority in Stages

The safest route to useful autonomy is gradual expansion. Start with observation and recommendations, then permit reversible changes, then allow narrowly defined autonomous actions, and only later consider authority over high-consequence decisions. Each stage should have clear evidence that the previous level worked well enough to justify the next.

This staged approach also helps adoption. Employees have more reason to trust an agent when they can see what it may do, what it may not do, and when it must ask for approval. Leaders gain a clearer basis for deciding where automation actually saves time without transferring more authority than the workflow requires.

The practical question for any new agent should therefore be simple: what is the maximum power this system needs to perform its job well? Give it that authority, define the boundaries explicitly, monitor what it does, and require human approval before it crosses them. Organizations will get more value from AI agents when autonomy grows as a controlled operating decision rather than as a side effect of adding more tools.