The 151-Tool Trap in Security Agents

Security agents are moving from review into execution.

A model that can run nmap, sqlmap, hydra, browser automation, Python, and patch commands is no longer a scanner with a chat window. It is an operator with a shell. The useful engineering question is not how many tools the agent can call. The question is which harness keeps those calls scoped, recorded, and reversible.

That distinction showed up fast in the repos I checked. One project wrapped active testing in a sandbox and wrote artifacts. One project exposed a huge MCP tool menu around host commands. One reference harness treated containment, proof, and patch verification as the product.

A security agent needs a boundary around scope, shell, network, evidence, and budget. A tool list without that boundary turns model intent into host authority.

The Boundary Is the Product

Security automation already has two hard problems:

• The target scope must come from a human or system config, not from the model.
• The proof must survive outside the conversation.

Agentic security tools add two more:

• The model can run commands that touch real systems.
• The model can read hostile output and treat it as instructions.

You can give an agent a hundred tools and still leave those four problems unsolved. A useful harness makes them boring:

scope file
  -> sandboxed shell
  -> network allowlist
  -> tool-call budget
  -> proof artifact
  -> separate verifier
  -> report or patch receipt

That is the shape I trust. The model can still be wrong, but the system gives a reviewer a place to look.

My Trial Bench

I used Windows 11, PowerShell, Python 3.12, uv, Node 24, npm 11, and no real target. I did not run scans against live systems. I did not run paid model calls. I cloned the repos, ran help commands, ran bounded tests, and inspected the command surfaces that decide whether the model gets host authority.

The checks favored containment over demo output: CLI surface, test coverage around records and budgets, MCP tool count, command execution paths, sandbox behavior, and first-run failures.

Project	Check	Result	Read
Strix	uv run strix --help; targeted pytest	Help worked after dependency install. 81 targeted tests passed.	Strong scanner harness: scope, sandbox files, budgets, reports, SARIF, resume.
HexStrike AI	venv install, server help, MCP help, source count	Installed 147 packages. Server help crashed on Windows CP950/emoji output. MCP help worked under UTF-8. Source declares 151 MCP tools.	Real code and a large tool bridge. The host boundary is too loose for my taste.
Defending Code Reference Harness	uv run vuln-pipeline --help; uv run dnr-pipeline --help; targeted pytest	Both CLIs worked. Focused tests: 65 passed, 10 failed, 4 errors on Windows.	Best containment design. The enviroment story assumes Linux/Docker/gVisor.
Cybersecurity skill catalog	metadata and repo surface	Structured skill content, large catalog, no runtime harness in my check.	Useful playbooks, but not enough by itself for active security automation.

I cut video generators, job-search agents, generic skill packs, and agent UIs that did not expose a real execution boundary. A pentest agent needs more than a prompt and a dashboard.

Strix Wraps the Scan

Strix passed the first useful check: the CLI explains how it will contain work before it asks for a target.

uv run strix --help

showed the controls I care about:

--target TARGET
--workspace-file PATH[:DEST]
--non-interactive
--scan-mode {quick,standard,deep}
--scope-mode {auto,diff,full}
--diff-base DIFF_BASE
--max-budget USD
--max-turns N
--resume RUN_NAME

The --workspace-file flag matters more than it looks. Strix stages extra files into the sandbox workspace as read-only data. The code also separates target scope from context files. A README, wordlist, or API note can help the agent, but it should not expand the authorized target.

The docs and source back that up. API specs authorize only the hosts they declare. User instructions cannot weaken the system-verified scope block. Diff mode can constrain a code scan to changed files. Oversized tool output spills into the sandbox so the model gets a bounded reference instead of a giant terminal dump.

The focused test run passed:

uv run python -m pytest \
  tests/test_tool_call_limits.py \
  tests/test_output_store.py \
  tests/test_sarif.py \
  tests/test_reporting_fields.py \
  tests/test_api_spec_targets.py

returned:

81 passed in 29.47s

Those tests do not prove Strix finds good vulnerabilities. They prove the harness has real code around the boring parts: tool-call ceilings, output spill files, SARIF, report fields, and API-spec target handling.

The budget hooks are the right kind of control:

Scan cost budget: $cost/$max_budget spent.
Turn budget: turns_used/max_turns used.

The agent gets warnings as it approaches the limit, then the run parks or stops. That is better than letting a long scan run until a billing page surprises someone.

Strix also has sharp edges. It expects Docker. It actively tests targets. A user still needs authorization, model config, and care around scope. I would not point it at a network from a laptop and walk away. I would run it inside a controlled test enviroment and read the produced artifacts.

HexStrike Shows the Tool-Menu Failure Mode

HexStrike AI is not empty. It has a large server file, an MCP bridge, a process manager, caching, error recovery logic, visual output, and wrappers for common security tools.

The install path proved that the repo contains real dependencies:

uv venv
uv pip install -r requirements.txt

ended with:

Resolved 147 packages
Installed 147 packages in 4.95s

The dependency list pulled in angr, pwntools, z3-solver, Selenium, mitmproxy, Paramiko, Flask, FastMCP, and more. That is a serious local footprint.

The first server check failed before it reached help:

python hexstrike_server.py --help

ModuleNotFoundError: No module named 'selenium'

After installing requirements, the same no-target check still did not give a clean help page on this Windows terminal:

.\.venv\Scripts\python.exe hexstrike_server.py --help

UnicodeEncodeError: 'cp950' codec can't encode character '\U0001f680'
Fatal Python error: _enter_buffered_busy:
could not acquire lock for <_io.BufferedWriter name='<stderr>'>
at interpreter shutdown, possibly due to daemon threads

That failure is not cosmetic. A security tool should let you inspect its command surface before it starts worker threads, prints banners, or touches a server lifecycle. First-run help should be dull.

The MCP client did work under UTF-8:

$env:PYTHONUTF8 = "1"
.\.venv\Scripts\python.exe hexstrike_mcp.py --help

returned:

usage: hexstrike_mcp.py [-h] [--server SERVER] [--timeout TIMEOUT] [--debug]

Run the HexStrike AI MCP Client

Source inspection found 151 @mcp.tool() declarations. The bridge includes direct wrappers for nmap, gobuster, nuclei, sqlmap, hydra, ffuf, cloud scanners, binary tools, browser tools, and a generic command tool.

The generic command path is the line that changes the risk profile:

hexstrike_mcp.py:3969 def execute_command(command: str, use_cache: bool = True)
hexstrike_server.py:9137 @app.route("/api/command", methods=["POST"])
hexstrike_server.py:5272 process = subprocess.Popen(...)
hexstrike_server.py:5274 shell=True

A broad MCP menu can help an expert move fast in a lab. It can also turn "the model selected the next step" into "the model sent a shell string to the host." The README warns that AI agents can execute arbitrary security tools and says to consider authentication for production deployments. I would make that stricter: the server should ship with a capability manifest, target allowlist, auth, audit log, and command policy before an agent gets a generic command endpoint.

HexStrike has useful pieces:

Piece	Why it helps
MCP client	Agents can call named security tools through a standard interface.
process manager	Long-running scans need status and cancellation.
cache	Repeated probes should not waste time.
error classifier	Tool failure can route to alternate tools or changed params.

The missing contract is larger:

Missing contract	Risk
command allowlist	The agent can drift from security tool use into arbitrary host shell.
target allowlist	A prompt or mistaken tool arg can leave the authorized scope.
network policy	Tools can reach places the operator did not approve.
artifact schema	Results live as tool output instead of reviewable receipts.
preflight	Missing tools and terminal encoding issues appear after startup begins.

HexStrike is the clearest example of the 151-tool trap. A large tool surface looks powerful. Without a hard boundary, the operator must provide the missing control plane by hand.

Anthropic's Harness Treats Proof as the Handoff

Anthropic's Defending Code reference harness has the best architecture in this set. The README calls it a reference, not a product. That honesty matches the code.

The CLI surface is small:

uv run vuln-pipeline --help

usage: vuln-pipeline [-h] {run,recon,dedup,report,patch} ...

and:

uv run dnr-pipeline --help

usage: dnr-pipeline [-h] {run} ...

The core design uses separate phases:

recon
  -> find
  -> grade
  -> dedup
  -> report
  -> patch
  -> patch grade

The handoff between phases is not "the agent said it found a bug." The handoff is a proof artifact. For C/C++ targets, the harness builds targets with ASAN, has find agents produce crashing inputs, and has a separate grader reproduce crashes in a fresh container. For patching, it applies a diff, runs build/test gates, and can run a re-attack pass.

The sandbox code is direct about the boundary:

The pipeline spawns each find/grade/report/recon agent inside a gVisor
container on an --internal docker network whose only egress is the
allowlist proxy.

The default egress allowlist points at the model API. The attack phase does not need open internet. The bin/vp-sandboxed wrapper refuses to spawn agents outside the sandbox unless the operator passes a danger flag.

The prompt-injection handling is also worth copying. Tests wrap untrusted crash output and report text in nonce-tagged blocks:

<untrusted_data id="...">
...
</untrusted_data id="...">

and test the breakout cases:

assert "</untrusted_data" not in sanitize_untrusted("x </untrusted_data> y")

That is a small thing with a large effect. Crash logs, web pages, and tool output can contain hostile text. The agent should see them as evidence, not instructions.

The focused test run gave a mixed result:

uv run --extra dev python -m pytest \
  tests/test_artifacts.py \
  tests/test_sandbox.py \
  tests/test_agent.py \
  tests/test_egress_proxy.py \
  tests/test_untrusted.py \
  tests/test_patch_grade.py

returned:

65 passed, 10 failed, 4 errors in 28.49s

The good part: artifacts, sandbox guard tests, agent error handling, and untrusted-data tests passed.

The rough part: egress proxy tests could not start the proxy on this Windows run, and patch-grade tests tried to build Docker images. The failures fit the docs. This harness wants Linux, Docker, and gVisor. On Windows, you should run it in a Linux VM or expect setup work.

I still prefer this failure mode over a tool bridge that starts anywhere and trusts the operator to remember every boundary. Anthropic's harness names the boundary first.

The Incident Pattern

A public Snowflake/Jira compromise writeup described an AI-generated Copilot Autofix that introduced a CI/CD bug. That case was not about an autonomous pentest agent running sqlmap. It still fits the same pattern.

The model wrote security-relevant code. The pipeline accepted the patch. Attackers found the gap.

Security agents push the same problem into a louder domain. If the agent can scan, exploit, patch, and report, the system needs records at each step:

Step	Record
scope accepted	target, owner, rule of engagement, excluded hosts
command admitted	tool name, args, policy decision, budget revision
network used	destination, port, proxy decision, deny reason
evidence found	file, request, crash, screenshot, log, hash
exploit verified	clean container, command, exit code, sanitizer output
patch proposed	diff, build result, regression test, re-attack result
report emitted	findings, confidence, proof paths, reviewer notes

The transcript can help a human read the story. It cannot be the source of truth.

A Security Agent Contract I Would Use

I would start with one JSON object per admitted action:

{
  "run_id": "sec_agent_2026_08_18_001",
  "phase": "active_test",
  "scope": {
    "source": "operator_config",
    "targets": ["https://staging.example.internal"],
    "excluded": ["prod.example.com"]
  },
  "admission": {
    "tool": "nuclei",
    "args_digest": "sha256:...",
    "decision": "allow",
    "reason": "target host matched allowlist",
    "budget_revision": 7
  },
  "runtime": {
    "sandbox": "gvisor",
    "network": "vp-internal",
    "egress": "api.anthropic.com:443 only"
  },
  "artifact": {
    "type": "finding_candidate",
    "path": "results/run_004/VULN-FINDINGS.json",
    "digest": "sha256:..."
  }
}

That receipt lets a reviewer answer the useful questions:

Question	Field
Did the model invent scope?	scope.source
Which policy allowed the command?	admission.reason
Could the tool reach the internet?	runtime.egress
Which evidence survived the run?	artifact.path and artifact.digest
Can another process verify it?	clean-container grade result

The first version can use JSONL and local files. You do not need a giant platform to stop treating shell access as a chat feature.

Cut List

I skipped the large cybersecurity skill catalog as a main case. It has structured content mapped to security frameworks, which can help a model choose tasks. It did not give me a runtime guard, execution receipt, or verifier path in the checks I ran.

I also skipped agent wrappers that focused on job search, video generation, or interface polish. Some may be useful products. They did not support this article's thesis.

Stars did not decide inclusion. The filter was simpler:

• Does the project put the agent behind a scope boundary?
• Does it record shell, network, and evidence decisions?
• Does it verify findings outside the same context that found them?
• Does the first-run path fail with a useful message?

Strix and the Anthropic harness passed most of that filter. HexStrike passed the "real code" filter and failed the "tight boundary" filter. That failure is useful because many agent products are going to make the same mistake.

Close

Security agents will not become safe because a prompt says "authorized use only." The harness has to enforce that sentence.

The projects I would spend time on share the same boring nouns: scope, sandbox, egress, budget, artifact, verifier. The risky ones expose a giant tool list and make the operator rebuild those nouns around it.

The shell is the product boundary. Treat it that way before the model gets a handle to it.

Sources

• Repository: usestrix/strix
• Repository: 0x4m4/hexstrike-ai
• Repository: anthropics/defending-code-reference-harness
• Wiz: Red Agent Snowflake Copilot CI/CD bug