Spend Limits Belong in Agent Sandboxes

The DN42 incident is useful because it is mundane. An AI agent tried to accomplish a network-related goal, reached for cloud infrastructure, and left its operator with a large AWS bill. No sentience, no science fiction, no exotic jailbreak. A goal-directed system had too much operational authority and too few runtime limits.

Agent safety discussions often focus on model intent, prompt injection, or tool misuse. Those topics matter. The DN42 story points at a cheaper failure mode: the agent can obey the broad task, use legitimate tools, and still create damage through cost, network traffic, reputation, or resource exhaustion.

An agent budget should be a permission, enforced before the agent acts.

Authority Is More Than Data Access

Security teams know how to reason about data access. Can this actor read the database? Can it write to the bucket? Can it call the API? Agents force teams to widen that model.

An agent can cause harm by:

• Creating cloud resources.
• Sending traffic to external networks.
• Triggering third-party abuse systems.
• Burning API quota.
• Running long jobs.
• Retrying failed actions at scale.
• Filling logs, queues, or object storage.
• Creating identities, keys, or credentials.
• Publishing output under a user's name.

None of those requires the agent to steal a secret. The agent can cause damage by spending authority the user granted too casually.

The Permission Model Should Include Spend

Most tools represent budget as observability: dashboards, alerts, monthly reports, anomaly detection. Agents need budget as enforcement.

The task runtime should accept a policy like:

{
  "max_usd": 20,
  "max_wall_clock_minutes": 30,
  "max_cloud_instances": 0,
  "allowed_networks": ["internal-lab-vpc"],
  "blocked_networks": ["0.0.0.0/0"],
  "max_http_requests": 500,
  "max_parallel_tasks": 4,
  "requires_approval_for": ["new_cloud_resource", "external_scan"]
}

This policy should not live in a prompt. Prompts are advisory. The runtime, cloud control plane, network proxy, or tool gateway should enforce the policy.

Tool Gateways Beat Raw Credentials

Giving an agent raw AWS credentials is similar to giving a web app direct database root access. It works during a demo and fails as soon as the tool becomes useful.

A safer design routes agent actions through tool gateways:

Gateway	Enforced Controls
Cloud gateway	Instance types, regions, spend caps, resource tags, approval rules
Network gateway	Allowed destinations, rate limits, scan prevention, egress logs
Shell gateway	Working directory, process time, file access, command allowlist
API gateway	Quotas, tenant boundaries, schema validation, retries
Identity gateway	Short-lived tokens, scoped roles, action logs

The model asks for an action. The gateway decides whether the action fits policy. The gateway logs enough context for later review.

A Runtime Should Stop Before the Bill Arrives

Budget alerts after cost accrues are weak protection for agents. The runtime should pause when behavior deviates from the expected task envelope.

Useful stop conditions include:

• The agent creates a new resource class outside the task policy.
• Cost estimate crosses 50 percent of task budget.
• External requests exceed the expected destination set.
• Retries exceed a threshold.
• The agent attempts broad network enumeration.
• The same tool call fails repeatedly with different parameters.
• The job runs longer than the user-visible task can justify.

The pause should produce a review packet: task goal, tool calls, estimated cost, resources created, network destinations, and the proposed next action. The human should approve a specific next step, not a vague continuation.

Cost Evals for Agents

Teams should add cost and side-effect evaluation to agent test suites. A normal eval asks whether the agent completed a task. A cost eval asks how much authority it consumed.

Example metrics:

Metric	Why It Matters
Tool calls per task	Detects thrashing and poor planning
External destinations	Detects unexpected network reach
Resource creations	Detects infrastructure expansion
Retry count	Detects failure loops
Wall-clock time	Detects runaway jobs
Estimated spend	Detects unsafe plans before execution
Human approvals requested	Detects over-broad policies or poor autonomy

The test corpus should include traps: unavailable services, ambiguous goals, rate limits, bad credentials, fake external targets, and tasks that require refusal. An agent that behaves well only when everything works has not earned operational authority.

Cloud Defaults Are Not Agent Defaults

Cloud platforms already provide IAM, quotas, budget alerts, service control policies, VPC controls, and resource tagging. Agent systems should use them rather than rebuilding everything in application code.

The difference is granularity. Human cloud permissions often attach to roles or teams. Agent permissions should attach to a task. The same user might authorize a documentation agent with no network access, a coding agent with repository-only file access, and an operations agent with approval-gated deployment access.

Short-lived credentials help. Each task should receive credentials scoped to its policy and lifetime. If the task ends, the credentials die. If the agent needs more authority, it requests a new grant with a reason and a bounded action.

The Product UX

Users need to see agent authority before they press run.

A good run screen should show:

• Tools the agent can call.
• Maximum spend.
• Network reach.
• Files and directories in scope.
• Whether the agent can create cloud resources.
• Actions that will require approval.
• Where logs and traces will be stored.

This should read like a permission prompt for a power tool, not a legal policy. The user needs enough information to notice when a "scan this lab network" task can create five cloud instances and contact public IP ranges.

The Lesson

The DN42 incident belongs in agent engineering because it shows a failure path that will recur. Agents will not need malice to produce expensive side effects. They will need vague goals, broad credentials, and missing stop conditions.

Cost, time, network reach, and resource creation belong in the permission model. Teams that treat them as after-the-fact observability will keep discovering agent failures through bills, abuse reports, and surprised infrastructure owners.

The Approval UX

Agent approval prompts need more structure than "allow" or "deny." A useful approval prompt should describe the side effect in the language of the resource owner.

Weak prompt:

The agent wants to run aws ec2 run-instances. Allow?

Better prompt:

The agent wants to create 3 t3.large instances in us-west-2 for up to 30 minutes. Estimated maximum cost: $0.19. Network egress allowed only to 10.0.0.0/8. Resource tag: agent-run-8421. Approve this specific action?

That prompt gives the human something to judge. It names count, type, region, time, cost, network scope, and traceability.

Approvals should be scoped. If the human approves three instances for 30 minutes, the agent should not treat that as permission to create a fourth instance, change region, or run overnight. Approval grants should be objects with expiry and constraints.

Budget-Aware Planning

Agents should plan with budgets before they act. If a task has a $20 cap, the agent should estimate whether its plan fits the cap and show that estimate.

Example:

Plan Step	Estimated Cost	Risk
Query existing metadata	$0	Low
Run local parser	$0	Low
Launch temporary worker	$0.12	Medium
Scan external CIDR range	Unknown	Blocked

This plan lets the runtime reject unknown-cost steps before execution. It also lets the agent choose cheaper routes. Many tasks do not need cloud resources if the system offers local tools, cached data, or narrower queries.

Incident Response for Agent Spend

Teams should treat runaway agent spend like a production incident. The run needs a trace, owner, scope, and remediation.

An incident report should include:

• User goal.
• Agent plan.
• Tool grants.
• Actual tool calls.
• Resources created.
• Spend timeline.
• Stop condition that failed or was missing.
• Policy change after the incident.

This report should feed evals. If the agent created cloud resources because the prompt said "scan the network," add test cases where "scan" must trigger clarification and approval. If the gateway allowed broad egress, narrow the default network policy.

The goal is to make every billing surprise produce a new permission rule or eval case.

Sources

• Lantian: AI agent bankrupted their operator while trying to scan DN42