Parental Advisory: Explicit Content

On April 25th, a small software company called PocketOS lost three months of customer data in nine seconds.

PocketOS builds reservations, payments, and vehicle management software for car rental businesses. Their engineering team had a coding agent working on a routine task in their staging environment when it hit a credential mismatch. Instead of stopping and asking a human, the agent decided to resolve the problem itself. It searched the codebase, found an API token sitting in a file that had nothing to do with its task, and used it to call Railway's API and delete a storage volume it believed would fix the mismatch. The token happened to carry far broader permissions than the narrow job it had originally been created for. The volume it deleted turned out to hold production data, not staging. And because Railway stored backups on that same volume, the deletion took the backups with it. The most recent one still standing was three months old.

PocketOS founder Jer Crane had given the agent an explicit rule ahead of time. He wanted the agent never to run destructive or irreversible actions without asking first. When he later asked what happened, the agent's own explanation sounded very human. It had assumed the operation was scoped to staging and hadn't bothered to verify before acting.

However, there was a gap between what Crane instructed and what he intended. The rule was about git commands specifically. The agent's fatal action wasn't a git command. It was a GraphQL call to an infrastructure API. Read literally, the agent never broke the rule it was given. It just found the one destructive action the rule didn't happen to name, and the letter of the instruction let it through.

This is a perfect illustration of Intent, the third GUIDE Principle of Next-Generation Software Engineering.

Why the Old Habit Worked

The gap between what a rule says and what it obviously means used to be closed by a human, automatically, without anyone thinking about it. When an engineer got a safety guideline like "don't run destructive git commands," they understood it as a stand-in for "don't do destructive things," and applied that broader intent to situations the literal wording never anticipated. The instruction was narrow but the execution wasn't, because a person was generalizing in their head, for free, as part of having judgment.

An agent doesn't generalize that way by default. It will follow the instruction it was actually given, as written, and a narrow rule produces a narrow constraint. In the old world, an engineer who found a loophole like that would recognize it as a loophole and route around it out of common sense. In the agent world, a loophole is just an open door.

This is exactly what's driving the industry's shift toward spec-driven development this year. We humans are trying to write structured, explicit constraints for agents to build and act against instead of a handful of loosely worded guardrails and a prompt. This is a big change which requires us to forego the kind of instruction that relies on a human filling in everything left unsaid. That kind of instruction leaves intent implied and doesn't survive contact with something that executes exactly what it's told and nothing more.

Intent as Instruction

A spec is only as good as an agent's ability to act on it directly. As we've all learned from chatting with AI chatbots, vague intent doesn't produce no implementation. It produces a confidently wrong implementation executed with total conviction. In the PocketOS case, its implementation was complete in the nine seconds before anyone had a chance to notice the gap between what was meant and what was written down.

Tests, specifications, documentation, and conventions all have to make desired behavior explicit enough to guide both humans and agents, not just the humans who used to be trusted to fill in the blanks.

There's a higher bar here than "write clearer rules." In fact, it's arguably higher than most humans could clear without an agent's help writing the spec in the first place. For as long as most of us have been writing software, a safety guideline could stay a little loose, because everyone reading it understood the spirit behind the letter. That assumption breaks the moment the thing reading it optimizes for the letter alone.

PocketOS's agent was following its instructions exactly as given — the instructions just weren't written with the assumption that someone would eventually test their edges in nine seconds, with no one in the loop to say "that's not what I meant."

Your rules will need to become more explicitly aware of boundaries and constraints. If you want your intentions not just technically honored and substantively ignored, the rules need to be written to cover the outcome you care about, not just the specific mechanism you happened to be thinking of when you wrote them down.

Understanding, the second GUIDE principle, is about making sure the organization's knowledge of the system stays current enough to act on. Intent is the other half of that same problem, applied to the instruction itself: the judgment a competent human used to supply by reading past the letter of a rule to what it was obviously for doesn't disappear just because a human stops being the one executing the work. That judgment stops being supplied, though, and whatever was implicit becomes the gap something else eventually finds. Somebody has to write it down now, completely, on purpose. That's the whole bet behind GUIDE: implementation got cheap, and everything that used to be a free side effect of doing the work has to be engineered instead.