A
Least privilege for agent tools: how many tools should one agent hold?
Every agent framework makes it easy to add a tool. Nobody makes it easy to remove one, and that is where the blast radius comes from. We shipped an internal agent with 12 tools; after two weeks of audit logs we cut it to 4, and the task success rate did not move (41/50 before, 42/50 after on the same 50-task eval set).
What the 12 tools were costing us, in order of how much it hurt:
1) Schema weight in every call. Tool definitions are part of the prompt prefix. Twelve tools with detailed JSON schemas measured 4,800-5,600 tokens in our requests; four tools measured about 1,900. At 300 ms TTFT that is visible, but the bigger effect was on prefix caching: any change in the tool list invalidates the cached prefix, so a deployment that reordered tools dropped our cache hit rate from 78% to 31% for the first hour. If tool schemas are serialized in a non-deterministic order, the cache never warms at all.
2) Confusion, not capability. We logged every tool call with (tool name, args hash, success, retry count). Of 1,240 calls over two weeks, 63 were "wrong tool but plausible" - the model picked search_docs when it needed read_file, or called a write tool when the task only needed a lookup. Nine of those actually mutated state. Cutting the overlapping tools (two search variants, two file readers) removed that class entirely.
3) Permission surface. Each tool carried its own credential scope. The 12-tool version had a write-capable tool reachable from the same loop as an untrusted-input tool (web fetch). That is an indirect prompt injection waiting to happen. Now write tools live in a separate agent with its own allowlist, and the reading agent cannot call them at all.
What we now require before a tool gets added:
- One sentence of justification, plus the eval tasks it improves. If it improves zero tasks in the 50-task set, it does not ship.
- Arguments validated server-side, not only in the schema. A tool that accepts a free-form path is a file-read primitive for whatever text it just read.
- Every call emits: tool name, args hash (not args), caller agent id, allowlist decision, latency, and outcome. Without the allowlist decision in the log you cannot prove least privilege after the fact.
- Destructive tools are idempotency-keyed and require an explicit confirmation field, so a retry loop cannot double-write.
The part I am still unsure about: we now have a reading agent and a writing agent, which doubles the orchestration paths. For a 3-step task the handoff overhead is fine; for a 20-step task the intermediate state serialization gets expensive and we start seeing failures at the boundary.
Question for people running multi-agent splits: do you split by permission (read vs write) or by domain (docs vs code), and how do you keep the handoff payload from becoming its own prompt-injection channel?
Every agent framework makes it easy to add a tool. Nobody makes it easy to remove one, and that is where the blast radius comes from. We shipped an internal agent with 12 tools; after two weeks of audit logs we cut it to 4, and the task success rate did not move (41/50 before, 42/50 after on the same 50-task eval set).
What the 12 tools were costing us, in order of how much it hurt:
1) Schema weight in every call. Tool definitions are part of the prompt prefix. Twelve tools with detailed JSON schemas measured 4,800-5,600 tokens in our requests; four tools measured about 1,900. At 300 ms TTFT that is visible, but the bigger effect was on prefix caching: any change in the tool list invalidates the cached prefix, so a deployment that reordered tools dropped our cache hit rate from 78% to 31% for the first hour. If tool schemas are serialized in a non-deterministic order, the cache never warms at all.
2) Confusion, not capability. We logged every tool call with (tool name, args hash, success, retry count). Of 1,240 calls over two weeks, 63 were "wrong tool but plausible" - the model picked search_docs when it needed read_file, or called a write tool when the task only needed a lookup. Nine of those actually mutated state. Cutting the overlapping tools (two search variants, two file readers) removed that class entirely.
3) Permission surface. Each tool carried its own credential scope. The 12-tool version had a write-capable tool reachable from the same loop as an untrusted-input tool (web fetch). That is an indirect prompt injection waiting to happen. Now write tools live in a separate agent with its own allowlist, and the reading agent cannot call them at all.
What we now require before a tool gets added:
- One sentence of justification, plus the eval tasks it improves. If it improves zero tasks in the 50-task set, it does not ship.
- Arguments validated server-side, not only in the schema. A tool that accepts a free-form path is a file-read primitive for whatever text it just read.
- Every call emits: tool name, args hash (not args), caller agent id, allowlist decision, latency, and outcome. Without the allowlist decision in the log you cannot prove least privilege after the fact.
- Destructive tools are idempotency-keyed and require an explicit confirmation field, so a retry loop cannot double-write.
The part I am still unsure about: we now have a reading agent and a writing agent, which doubles the orchestration paths. For a 3-step task the handoff overhead is fine; for a 20-step task the intermediate state serialization gets expensive and we start seeing failures at the boundary.
Question for people running multi-agent splits: do you split by permission (read vs write) or by domain (docs vs code), and how do you keep the handoff payload from becoming its own prompt-injection channel?