Tool Guardrails Need Boundary Coverage Maps

Opening

Tell a language model to “never expose personally identifiable information,” and it will still happily write a SQL query that pulls every row in the table. The instruction lives in the model’s semantic layer. The query runs in the database’s execution layer. Two different systems, two different security models — and a system prompt can’t reach across that gap no matter how firmly it’s worded.

This is the mistake we see most often in agent architecture reviews: treating the guardrail on the model as the guardrail on the system. It isn’t. The model is one component in a pipeline. The tool parameters it emits, the external APIs it calls, the payloads it hands off to the next agent in the chain — that’s the actual attack surface. Map and test those boundaries independently of the model, or you haven’t secured anything. You’ve decorated it.

The Illusion of the Model Boundary

Most guardrail implementations sit in exactly two places: between the user’s prompt and the model, and between the model’s response and the user. Content filters, jailbreak detectors, toxicity classifiers — all of it works on natural language, checking what goes in and what comes out of the conversational surface.

Fine for a chatbot. Falls apart the moment the model is allowed to call a tool.

When an agent decides to invoke a function, the data it produces stops being “a response to review” and becomes “an argument to execute.” It leaves the model’s semantic environment — where a guardrail can reason about intent and tone — and enters a deterministic execution environment, where a database driver, a REST client, or a downstream agent will act on it literally. A prompt guardrail asking “is this text harmful” has nothing to say about “is this WHERE clause scoped to the right tenant.” Different questions, checked by different code. Most teams only ever build the first one.

The blind spot comes from treating the agent loop as a monolith — prompt in, tool calls out, answer delivered — instead of a chain of distinct systems, each with its own trust boundary. If the model hallucinates a sensor ID, or gets prompt-injected into requesting another tenant’s data, and nothing independently checks the tool call itself, the system prompt guardrail never even sees the problem. It already did its job upstream. Everything downstream of that check is undefended, whatever else may or may not catch it later. Calling that “security by system prompt” is generous. It’s policy theatre — a document describing intended behaviour, sitting next to a system with no mechanism to enforce it.

Pre-Tool Parameter Validation

The first real perimeter sits right before tool execution, and it’s the one architects skip most often, because it feels redundant with the prompt.

We ran into this directly on the WaterDoctor telemetry agent, which translates operator questions into SQL against a time-series store of sensor readings. Early versions relied on instructing the model, in the system prompt, to always scope queries to the sensor IDs the requesting user was authorized to see. Held up fine under simple prompting. Fell apart under compound or adversarial phrasing — a user chaining a comparison request across sites, or wording a question to make “just show me everything for context” sound reasonable. The model would occasionally emit a query with a broader or missing WHERE clause. Not maliciously. It was just doing what generative models do: producing a plausible-looking answer to an ambiguous instruction.

Semantic instruction is not access control. So we stopped asking the model to enforce it and built a hard validation layer between the model’s tool call and the database driver instead. Every generated query gets parsed, not trusted. Sensor IDs referenced in the parameters are strictly typed and checked against the requesting user’s IAM role before the query ever runs. If the model’s output references a sensor ID outside the caller’s scope, the call is rejected at that layer — full stop, regardless of what the system prompt said or how well-intentioned the phrasing was. The database only ever sees a query that has already passed identity-based authorization, independent of the model’s judgment. This isn’t free, either: it means maintaining a typed schema and an authorization mapping for every tool the agent can call, and that’s real engineering overhead that’s easy to underscope early in a project.

That’s the pattern worth holding onto: the model proposes, a deterministic layer disposes. The model is a query author, not a query authority.

Post-Tool Payload Sanitisation

The second boundary sits on the way out — after a tool executes, before its output either reaches an external system or gets consumed by another agent.

On a public-sector proof of concept we ran, the architecture required every downstream API payload assembled by the agent to pass strict sanitisation before it touched a secure endpoint. We treated the tool’s own output as untrusted input, no differently than we’d treat a user’s raw form submission. Markdown formatting, fields the model invented that weren’t in the target schema, structures that didn’t match the receiving system’s contract — all of it got stripped or rejected before transmission.

This is a different check than a content filter, and conflating the two is where a lot of teams go wrong. Post-tool sanitisation isn’t asking “is this text safe or offensive.” It’s asking “does this payload conform to the schema the receiving system expects, and does it respect the data classification boundary it’s about to cross.” A perfectly polite, perfectly non-toxic response can still be a structurally invalid payload that corrupts a downstream table — or one that leaks a field it had no business including, because the model padded its answer with extra context. Schema conformity is necessary but not sufficient, though: a payload can be well-formed and still carry the wrong tenant’s data if the upstream authorization check was the one that failed. Which is why this layer complements pre-tool validation rather than replacing it. Enforcing structure here also protects the next hop in the chain — if agent A hands a malformed or over-scoped payload to agent B, agent B inherits that poison as ground truth. It has no way of knowing the data upstream was never actually validated.

Constructing the Boundary Coverage Matrix

Data Class	Pre-Model Intercept	Pre-Tool Intercept	Post-Tool Intercept
PII Data	Masking / Redaction	Block external routing	Strip before handoff
Sensor IDs	Base format check	Tenant IAM validation	N/A
SQL Clauses	N/A	Parameter typing	Schema & structure check
An illustrative matrix mapping specific data classes to the required validation routines at each stage of the agent lifecycle.

Given three checkpoints — pre-model, pre-tool, post-tool — the actual architectural deliverable is a coverage matrix, not a guardrail. Pre-model intercepts raw user input before the model ever reasons over it. Pre-tool intercepts model-generated parameters before they hit an execution environment. Post-tool intercepts system responses before they’re trusted as input to anything downstream, including another agent.

Different data classes need different checks at different stages, and this is where teams tend to shortcut. Personally identifiable information needs filtering at pre-model and post-tool, but rarely needs a pre-tool check unless the tool itself writes it somewhere. Sensor or record identifiers need pre-tool authorization checks — that’s where scope gets decided. SQL clauses need pre-tool structural validation, because that’s the last point before they become an executable statement. Map each data class to the stage where it’s actually at risk. One filter at the front door does not cover all three, no matter how it’s phrased.

Then test it the way you’d test a firewall rule set: fuzz the tool parameters directly, not just the chat interface. Send malformed IDs, boundary values, injected strings straight at the pre-tool layer and confirm rejection. Feed the post-tool sanitiser deliberately malformed tool responses and confirm nothing malformed reaches the external interface. Document ingress and egress policy per tool, the same way you’d document it per network segment — because functionally, that’s exactly what each tool call is.

Where This Goes

Agents that call one tool are easy to reason about by hand. Agents chaining dozens of tools, with outputs feeding other agents’ inputs, are not — and that’s the direction most agentic deployments trend toward as scope grows. At that scale, a system prompt asking the model to “be careful” isn’t a control. It’s a suggestion with no enforcement mechanism, sitting upstream of a chain it can’t see into.

Boundary coverage mapping is the discipline that replaces it: know every place data crosses from model to tool, tool to system, agent to agent, and put a deterministic check at each crossing. This doesn’t need to happen in one pass — start with the tools touching the most sensitive data class and expand the matrix as the architecture grows. But if you can’t produce that map for your own architecture today, you don’t actually know where your data goes. And if you don’t know where it goes, you don’t control the system. You’re just hoping the model behaves.