SWARMAgentic AI and cybersecurity research
Swarm

AI agent guardrails: from advice to a binding decision

Fabrice Pizzi · Swarm project

A purge is scored as low risk and executed; the controlled path checks the target, authority and required approval and refuses the purge without that approval.
A low score does not replace required approval: the execution boundary must enforce non-compensable rules.
Still imageOpen GIF

An advisory guardrail can warn that an action looks dangerous. A binding guardrail must also prevent the effect when required conditions are absent. The difference matters when an agent has valid credentials but acts on a false premise.

Access control addresses part of the problem: who can act on which resource? It does not prove the proposed action is justified, necessary or consistent with business conditions. A reasoning error can remain within the technical permissions of a legitimate account.

A “low-risk” purge that was never approved

Consider an agent analysing document storage usage. It proposes purging an archive it considers obsolete. A risk model gives a low score because the volume is small and the name resembles a test environment. The policy nevertheless requires explicit approval before deleting that data category.

If the score is sufficient to proceed, an estimate has replaced a requirement. Deletion could interrupt processing, remove evidence needed for investigation or hinder restoration.

A mandatory condition does not become optional because a model estimates the risk is low.

The control must examine the exact resource, operation, current authority and required approval. Without approval, the purge remains denied. Storage analysis may continue if authorised; that permission does not include deletion.

Combine models without giving them the same role

A LLM can propose an action and provide analysis. A classifier can flag anomalies. A quantitative model can support prioritisation when its data, assumptions and calibration are explicit. Deterministic rules enforce conditions that a score must not compensate for.

This composition preserves model usefulness without turning probability into permission. A score is not proof of truth; a conveniently chosen threshold is not a scientific measurement. False negatives, false positives and changing contexts still need evaluation.

CaMeL studies another useful separation: controlling flows and capability use when an agent handles untrusted data. Its findings belong to its own research setting, not to a protection fully implemented or qualified in Swarm.

Where must the decision take effect?

A rule is binding only if access to the resource actually passes through its enforcement point. If the agent can use another tool or child process to reach the resource directly, refusal in the orchestrator is insufficient.

Swarm’s engine controls defined transitions and operations. Its Trust pilot supports examination of per-action decisions with simulated effects. Global operating system and network confinement must not be presented as active. Protection depends on the scope actually covered and the alternative paths still accessible.

Verify the decision and its reach

For this purge, evaluation should distinguish the classifier’s warning, the policy decision, effective refusal and absence of the effect on the target resource. It should also try a target changed after approval and an alternative access path. None of these outcomes follows merely from an architecture diagram.

Traceability retains material for this verification. Ordinary logs are not immutable; a digest alone does not establish an independent witness or prove that history was never deleted.

The Swarm repository makes this allocation of responsibilities open to examination. A useful guardrail states which conditions it can check, which effects it can block and which boundaries remain outside its control.