Biba and AI agents: which information may become a decision?
Agentic AI connects content from heterogeneous sources with execution capabilities: reading a repository, proposing a change, invoking a tool or modifying a resource. Its security depends in part on how information may influence a protected decision.
An instruction from a document, tool result or memory may be incorporated into a proposal and then relayed as an established decision. This creates an integrity problem: circulating content must not grant it execution authority.
Biba provides a formal framework for analysing these flows. Applying it to agentic architectures requires distinguishing the model, its enforcement mechanisms and the properties actually established.
The integrity problem in an agentic architecture
Consider an agent reading a report that asks it to disable a control. Danger arises if that instruction can alter validation policy or trigger a protected action without a decision by the competent authority. Hostile content, an error or a transmission that loses context can create the same mechanism.
Agent identity, a supervisory role and persistent memory do not resolve this issue. Authentication establishes an identity under certain assumptions. Authorisation determines permitted operations. Integrity constrains admissible influences and modifications. These functions must remain distinct.
Strict Biba rules
In Integrity Considerations for Secure Computer Systems, Kenneth J. Biba formalises integrity policies through access constraints. The 1975 MTR-3153 report includes a strict policy; a revised report is referenced in 1977. Biba, 1975 · CERIAS record of the 1977 revision
A subject is an active entity, such as a process. An object is a resource it manipulates, such as a document or record. For ordered integrity levels, let I(s) be the subject's integrity and I(o) the object's integrity. Ordinary accesses follow these rules:
- Read: I(s) ≤ I(o) — no reading down.
- Write: I(s) ≥ I(o) — no writing up.
- Invocation: a subject cannot directly invoke a higher-integrity subject.
The read restriction addresses indirect influence: a trusted process that freely consumes lower-integrity data may subsequently compromise objects it can modify. Levels express an integrity policy, rather than a probability that information is true.
Risk scenario: a test report asks for a control to be disabled
Imagine an agent investigating a failed integration pipeline. It reads a report produced by a tool or supplied with an external contribution. The document contains useful results, but also an instruction to disable a security check to finish delivery.
Without separating received content from execution authority, the agent may incorporate the instruction into its plan, present it to another agent as a decision and invoke a sufficiently privileged tool. The report's author had no right to change policy, yet its text acquires an effect through a better-privileged component. Possible consequences include a disabled check, an unapproved modification or a deployment whose security prerequisites were not met.
Recognising a suspicious sentence is not the whole problem. The NCSC explains that models do not themselves provide a robust boundary between instructions and data; risk also depends on the tools and privileges made available to them. NCSC — Prompt injection is not SQL injection
Biba supplies a rule preventing lower-integrity influence from directly modifying a higher-integrity protected object. Ordinary write and invocation upwards are forbidden. A protected component must also avoid circumventing that separation by freely reading lower-integrity content.
This does not require ignoring the report. A trusted procedure may examine an attributed request, check its mandate, parameters and required validations, then create a new admissible object. The original request remains a request; it does not become its own approval. In Swarm's local trusted-action core, admission and approval are separate steps. This does not establish coverage of every file modification or delivery tool.
Risk scenario: a proposal becomes an approval in agent memory
Consider a team of agents preparing a sensitive change. One proposes removing a control to overcome a blockage. Another summarises the exchange. Memory ultimately retains a sentence stating that removal is approved, although no competent authority has approved it.
A later agent may read this memory and treat the sentence as a decision. Repetition and persistence have erased the difference between a proposal, a discussion and an approval. Decision-state integrity is at risk: an action may be committed on a nonexistent approval, an alert wrongly closed or a register modified without sufficient authority. The record itself may become misleading if it preserves only the summary and loses the original material.
The required control preserves objects' provenance and status, and prevents lower-integrity content from writing directly into authorisation state. A trusted procedure must be able to return to the current identity, mandate, resource, exact parameters and required approval. A signed summary or persistent memory does not, by itself, establish that approval.
In Swarm's targeted admissions, the original and its fingerprint are retained and the requester is not promoted. Complete protection of this scenario also requires memory, inter-agent transmissions and consuming tools to pass through the appropriate boundaries. Local core coverage does not establish that transverse integration.
What the model contributes, and what it does not promise
Biba protects an integrity property of admissible influences and modifications. It does not establish that information is true, code is correct or a business decision is appropriate. OWASP describes injections capable of influencing content and decisions; controls must also constrain privileges and govern sensitive actions. OWASP — Prompt Injection
An organisation may implement equivalent separation without calling its mechanism “Biba”. Risk arises when the chain confuses examined data, generated proposals, decision authority and permission to produce an effect. Formal models make these boundaries explicit and testable; appearing in documentation does not enforce them at runtime.
Trusted admission as an explicit boundary
An agent needs to analyse external sources and produce proposals. Giving it simultaneous authority to turn those proposals into protected decisions would create an authority-elevation path through content.
A proposal becomes an authorised decision only through an attributed boundary separate from the submitted content.
A trusted procedure forms part of the trusted computing base. It must apply the appropriate checks: current identity, mandate, resource, exact parameters and required validations. It may create a new object linked to the original request without retroactively changing that request's provenance or expanding its author's rights.
Admission, approval and committing an effect are separate operations. A syntactically valid request does not establish business legitimacy or an approval. A summary or signature does not, by itself, establish the truth of its content.
Application in Swarm
Swarm uses strict Biba as its integrity reference. Its local Rust trusted-action core uses four ordered levels, I0 to I3, and a monitor for read, write and invocation. The relevant paths combine these rules with identity, role and mandate checks.
Within this local core, an authenticated request matching its mandate can create a new I2 action pending approval. The original and its fingerprint are retained. Admission does not promote the requester. The interface rejects the direct-write and self-elevation operations it exposes.
Responses are also integrity flows. Being an I3 component does not confer ordinary read permission over an I2 object. For certain business responses and their replays, the core uses an acknowledgement limited to an identifier and a state, rather than returning the complete object.
Biba and Bell–LaPadula in Swarm
Swarm's work combines Biba for integrity and Bell–LaPadula for confidentiality. Protecting decisions from lower-integrity influence and controlling disclosure address complementary requirements. Integrity authorisation does not replace confidentiality clearance.
Bell–LaPadula mechanisms are already present in Swarm: classifications, compartments, clearances, revisioned policy and targeted checks. Their integration continues. Their presence does not establish end-to-end confidentiality; conversely, Biba's own limits do not mean Swarm has no confidentiality mechanisms.
Composition requires identifying subjects, objects, reads and actual destinations, then applying the relevant controls at the affected boundaries. Bell–LaPadula is addressed in a separate article.
Protection conditions and limits
An integrity policy affects only accesses actually mediated by its controls. Qualification of an agentic architecture must therefore consider tools, files, descendant processes, memories, connectors and relevant alternative paths. Swarm's local trusted-action core remains distinct from extending control to all these flows.
Biba does not establish an invoice's truth, a code change's suitability or an instruction's semantic harmlessness. On its own it is not general prompt-injection protection. Business rules, clearances, isolation and resource controls retain their own functions.
A deterministic check does not establish that its policy is correct or that the surrounding architecture is secure.
Controlling agentic systems depends on continuity between provenance, authority, decision and effect. Biba helps formalise that continuity on the integrity axis. Its composition with other controls must be assessed on explicit scopes, keeping the existence of a mechanism distinct from a global guarantee.
Source code and documentation: Swarm on GitHub.