Policy gates for consequential agent actions

Abstract

A control layer that routes every agent action to a human does not survive contact with a working team; one that routes none is not a control. This note sets out why consequential-action classification is the load-bearing decision, describes the tiered model in the current implementation, and reports a failure we consider more instructive than the design: the denial gate was verified working for one project while every project created through the onboarding wizard reached active status with an empty denial policy, silently, with no warning anywhere in the interface.

Summary

Policy gating is usually presented as the easy part of agent governance — the part you draw as a diamond on a diagram — and treated as solved once an approval step exists. It is not the easy part. The difficulty is not stopping an action; it is deciding which actions are worth stopping, and then making sure the rules that decide are actually populated.

The problem with gating everything

A gate that routes every action to a human has a predictable failure mode. The reviewer approves the first twenty, learns that approving is always the right answer, and stops reading. The control still exists in the architecture diagram and in the audit log, and it has stopped functioning as a control. From the outside — including from the ledger — a rubber-stamped approval is indistinguishable from a considered one.

This is why we treat classification, not enforcement, as the core design problem. Enforcement is mechanical. Deciding what deserves enforcement is where the system either earns a reviewer’s attention or spends it.

A note on vocabulary

We use consequential rather than risky. “Risky” implies detection — a probability estimate, a threshold, a false-positive budget. Classification here is a deterministic judgement about blast radius made before execution, and the word should not smuggle in a different mechanism.

The tiered model

Every project carries its own rules rather than inheriting a global list, because what counts as consequential is a property of the codebase, not of the tool.

Classification tiers and their outcomes
Tier	Outcome	Rationale
Infrastructure	Fast-track approval when confidence is high and tests pass	Narrow blast radius, mechanically verifiable, and the class where approval fatigue does the most damage.
Functional	Standard approval card	The ordinary case: a human reads what changed and why.
Data	No card at all — escalate to an engineer with full context	A one-tap approval is the wrong instrument for a change touching data paths, regardless of confidence.
Denied path	Refused outright before any commit exists	Some paths should not be proposed against at all. A refusal here is a successful outcome, not a failure.

Classification combines path and diff heuristics against the project’s own rules with a model classification as a second opinion. Where the two disagree the stricter class wins — an asymmetry chosen deliberately, because the cost of over-classifying is a human’s minute and the cost of under-classifying is an unreviewed change.

Where the gate runs

Placement matters as much as the rule. The denial check runs immediately after diagnosis produces its file list, and before the change is applied, before typecheck and tests run, and before anything is committed. A denied proposal never reaches a branch, a push, or a pull request.

Running the check later would still block the merge, and would still look correct in the ledger. It would also mean the agent had already written the change to a branch that exists in the customer’s repository. “Refused” and “refused after doing it somewhere else” are different claims.

Verified

Confirmed against a real repository: a correct, genuine fix touching a denied module was refused, the incident reached blocked status with its own deny decision, and the provider’s API confirmed no branch and no pull request were ever created.

An inert gate

That verification was real, and it proved less than we initially read it as proving. It demonstrated the gate for one project — a project whose denial rules existed only because they had been inserted by hand.

Neither creation path wrote the field. The onboarding wizard omitted it from its insert, the command-line registration path did the same, and neither drafter emitted it. The database column defaults to an empty list, so every project created through the wizard reached active status with a denial check that could never fire — silently, with no warning anywhere in the interface.

Checked against the production database rather than inferred from the code: of three projects, the two onboarded through the wizard carried 28 and 26 sensitive paths, 17 and 16 infrastructure paths, and zero denied paths each. Every blocked incident in the system belonged to the one hand-seeded project. The two wizard-onboarded projects had nine resolved incidents between them and had never blocked anything.

Why this is the useful finding

The gate was correct. The tests were correct. The live verification was correct. The feature was still absent for every real user, and nothing in the system said so. A control that is present in the code, and empty in the data, reports as working from every angle except the one that matters. We now treat “is this rule populated?” as a separate question from “does this rule work?”

What we changed

Denial rules became a requiredfield in both drafters’ schemas, not an optional one — so a model that omits the key fails validation loudly instead of producing a project with no denial policy.
The drafting prompt invertsthe usual “when uncertain, be strict” instruction for this tier specifically. Over-broad sensitive paths cost a reviewer a few minutes; over-broad denied paths block legitimate fixes outright and make the agent useless for that repository.
An empty list is a legitimate reviewed answer, and the interface says so inline. The confirmation event records how many rules were approved, so “reviewed and empty” is distinguishable from “never considered”.
The project detail page’s indicator was backwards — it flagged projects for having a denial policy. It now flags projects that have none.

Open question

The fix is forward-looking only. Projects onboarded before it still carry empty denial policies and require a human to populate them. A migration cannot invent a policy that was never authored, and inventing one is exactly the failure this article is about.

One further change came out of the same pass. The classification functions had been duplicated between the agent and the dashboard. That duplication is defensible for code that legitimately differs per service; it was not defensible here, because the dashboard’s policy simulator exists to tell a human what the engine will do. Any drift would have made the simulator lie about the enforcement path. Both now re-export the same function, asserted by identity rather than by parallel behavioural tests.

Open questions

Path rules are a weak language. The current model is a small set of path prefixes and exact matches per project. Blast-radius-size rules and dependency-version-bump rules are designed and deliberately deferred. A richer language buys precision and costs auditability — a rule a reviewer cannot read is not obviously better than a coarse one they can.
We have no measurement of approval fatigue. The entire argument for tiering rests on it, and we are reasoning from first principles rather than from data. A system that classifies well and is still rubber-stamped has not been shown to be better than one that classifies badly.
Who may loosen a rule?Policy edits should themselves be governed — versioned, attributed, and potentially requiring approval, with the policy version in force recorded against every change so the question “what governed this when it merged?” has an answer. None of that is built.