Agent Sandboxes Aren’t Safe Without UAT Seed Data Contracts

The Modern Sandbox Illusion

Here’s a scene we’ve lived through more than once: provision an isolated Linux environment for an agent, wire up a headless browser, mount a persistent volume, hand it an SQLite database that’s three months stale. The agent proposes a migration that drops a column something else depends on. Everyone calls it a hallucination and blames the model.

It isn’t a model problem. It’s a data problem wearing an AI costume.

Managed agents today ship with real infrastructure — sandboxed containers, browser automation, file systems that survive across sessions, memory that persists between runs. Infra teams have gotten genuinely good at locking down egress traffic and container boundaries. What almost nobody has a policy for is the seed data that populates the sandbox on day one: who wrote it, when it was pulled, whether it still matches the schema the agent is about to touch.

Bad seed data breaks agent output faster than a badly worded system prompt, and it’s cheaper to fix. We just haven’t been treating it as a first-class infrastructure concern.

Legacy UAT Scars Apply to AI Agents

Strip away the branding and a managed agent sandbox is a User Acceptance Testing environment built for a non-human tester. We’ve seen this failure mode before — long before “agent” meant anything more than a customer service rep with a headset.

In 2018 we delivered a government system that stalled for three weeks in UAT, and the code wasn’t the culprit. The UAT database was a masked snapshot of production, refreshed weeks earlier, and in the interim the business rules changed upstream: new eligibility constraints the masked dataset didn’t reflect. Testers flagged records that should have been rejected under the new rules but sailed through anyway, because the data itself predated the rules. Three weeks of sign-off cycles burned on a discrepancy that had nothing to do with application logic.

Human testers eventually noticed and complained loudly. That’s the part that doesn’t carry over to agents. An AI agent working against stale seed data doesn’t file a ticket saying “this dataset looks wrong.” It quietly reasons its way around the gap and produces an answer that’s internally consistent and factually detached from the current state of the world — no complaint, no escalation, just a confidently wrong artifact that looks finished.

We already know what stale UAT data does to a delivery timeline. We should stop being surprised when it does the same thing to an agent crew.

Hallucinating History in an Outdated SQLite Fixture

We ran into a version of this internally at wGrow. We’d spun up an agent crew to handle routine database migration scripts, working off schema analysis against a local fixture. The agent kept generating a migration path that dropped columns the live schema actually needed.

Our engineers spent two days chasing the wrong problem — rewriting the prompt three times, fiddling with temperature settings, bolting on explicit constraints about “never drop a foreign key without confirmation.” None of it touched the actual defect.

The fixture was the problem. The local SQLite database the agent was reasoning against didn’t have the foreign key constraints that had been pushed to staging two days earlier. From the agent’s point of view, dropping those columns was correct — nothing in its visible state said otherwise. It wasn’t inventing a bad migration. It was writing a perfectly logical migration for a database that no longer existed.

That’s the mechanism worth internalizing: an agent learns the shape of “history” largely from the data we hand it. Give it an alternate history and it will write consistent, well-reasoned fiction to match. The failure isn’t in the reasoning chain. It’s upstream, in provisioning.

State Contamination as a Fatal Error

Vendors sell persistent sandbox state as a convenience feature: run after run, the agent’s workspace carries forward, so it doesn’t have to rebuild context from scratch. In practice, persistence without a reset discipline is a contamination vector.

Run an agent through several iterations against a state that’s slowly drifting from source of truth, and each iteration compounds assumptions baked in from earlier, less accurate runs. The agent isn’t just working from stale data anymore — it’s working from stale data that its own previous outputs have further mutated. That’s a worse failure mode than a single bad snapshot, because it’s self-reinforcing and invisible from outside the context window.

Masked production extracts are tempting because they feel “real,” but they carry too much uncontrolled variance for repeatable agent runs. Every field is a potential surprise the agent has to reason about, most of it irrelevant to the task at hand. Synthetic fixtures are more disciplined — but only if they’re versioned and validated the same way code is. Either way, the target is narrow: the minimum dataset required for the agent to make one safe, correct change, injected fresh at the start of the run.

State contamination and undocumented sandbox drift should be treated as fatal errors, not warnings. Every agent run should start from a baseline you can verify cryptographically, not one you assume is still valid because it worked last week.

Enforcing Strict Seed Data Contracts

The fix is boring, which is exactly why it works. Seed data going into an agent sandbox needs a data contract — machine-readable and enforced by infrastructure, not by convention.

Concretely, that means:

• A seed data bundle carries a cryptographic hash; the sandbox verifies it before the agent is allowed to boot.
• The bundle carries a creation timestamp tied to the source environment it was pulled from — staging, production-masked, or synthetic generator run.
• A hard expiry is enforced. At wGrow we run a 72-hour maximum on any seed bundle used in an active agent workflow.
• Past 72 hours, the sandbox refuses to execute. Not a warning banner — a hard stop.

That last point matters more than the others. A soft warning gets ignored the same way stale UAT data got waved through in 2018 — someone decides “it’s probably fine” under deadline pressure, and it wasn’t. A hard limit takes that decision away from a human who’s incentivized to ship anyway. It also forces the real fix: automating the data preparation pipeline instead of relying on someone remembering to refresh a fixture.

The trade-off is real. A hard stop can block a legitimate fix when the refresh pipeline lags behind, and that’s genuinely frustrating under deadline pressure. But the answer is a faster pipeline, not a softer check — the same discipline that made the 72-hour limit necessary in the first place.

Automating the Baseline

Exotic AI failure modes get the conference talks: prompt injection, jailbreaks, agents going rogue on tool calls. Bad seed data doesn’t get a talk. It just quietly eats two engineer-days on a phantom migration bug, or three weeks of UAT sign-off on a government project, and nobody writes it up because “the data was stale” isn’t an interesting story.

It should be. It’s the cheaper failure to prevent, and the one most teams still have no process for.

Practically: build a CI/CD pipeline whose only job is refreshing, hashing, and signing agent seed bundles on a schedule that matches your data volatility, not your deployment cadence. Don’t let a developer manually drag a test file into an agent’s workspace — that’s the same failure mode as a tester hand-editing a UAT record to make a demo pass, and it produces the same downstream confusion. Write the data contract before you write the agent prompt, not after the agent has already started producing output you don’t trust.

The infrastructure for running agents is mature enough now that the interesting engineering problems have moved. They’re not in the sandbox anymore. They’re in what you put inside it before you press run.