Agent Reputation: How Multi-Agent Systems Decide Whom to Trust
Agent reputation can help route work and calibrate trust, but reputation should be contextual, evidence-based, decay over time, and never become a permanent identity score.
Trust between agents is a routing decision under uncertainty
Research on LLM agents in trust games shows that model-based agents can exhibit trust-like behavior and that their behavior varies with conditions and model properties. [1] That does not mean an agent “feels” trust. For system design, trust is better understood operationally: how much authority or workload should one component grant another given limited evidence about competence, reliability, and incentives. For a production team, that means the architecture should expose a concrete contract rather than relying on the model to infer organizational etiquette. The more agents involved, the more valuable explicit boundaries become because small ambiguities multiply across messages, retries, and handoffs.
Trust is contextual
A useful role definition names the job, required evidence, allowed tools, authority boundary, and return contract. That turns a social metaphor into an operational interface and gives evaluators something concrete to test.
Reputation must be multidimensional rather than a single score
Classical multi-agent trust research treats trustworthiness as multifaceted rather than reducible to honesty alone. [2] An agent can be excellent at retrieval but weak at numerical reasoning, dependable with read-only tools but risky with writes, or accurate in one domain and stale in another. Reputation should therefore attach to task-relevant dimensions rather than collapse a whole agent into one universal number. This also affects the motivational experience. Users gain confidence when progress has visible ownership and when the system can explain why a specialist was involved. Hidden delegation may be technically elegant but still feel erratic if the interface cannot connect each action back to the user’s chosen goal.
Capability discovery is not the same as demonstrated reliability
The A2A specification uses Agent Cards to publish capabilities, skills, endpoints, and authentication requirements so agents can discover one another. [3] Those declarations are useful metadata, but self-described capability is not evidence of successful performance. A robust router should distinguish what an agent claims it can do from what evaluation and production traces show it can do reliably. A useful implementation pattern is to record the decision as structured state: what was delegated, why this role was selected, what evidence it returned, and whether the result was accepted. That record supports debugging, evaluation, and later correction without requiring access to private chain-of-thought.
Claims need evidence
Independence is valuable only when the branches can actually contribute different information. If every worker inherits identical context and instructions, the system may multiply tokens without multiplying perspective.
Trust records need recency, provenance, and uncertainty
NIST’s AI RMF emphasizes ongoing measurement, documentation, accountability, and risk management across an AI system’s lifecycle. [4] Reputation systems need the same lifecycle discipline. Scores should record which task produced the evidence, when it occurred, what model and tools were involved, and whether the outcome was independently verified. Old success should not permanently outweigh recent degradation. The important boundary is between helpful specialization and organizational theater. Adding another agent should reduce uncertainty, isolate risk, or increase coverage. If it only adds another voice without a distinct responsibility, the system is paying coordination cost without buying capability.
Governance must define who can create and consume reputation signals
OpenAI’s multi-agent guidance notes that separate agents can be useful for independent bounded tasks and comparison of findings. [5] That diversity also matters for reputation. If routing always chooses the highest-rated worker, lower-rated agents never receive enough trials to demonstrate improvement. Some controlled exploration is necessary or the reputation system freezes early accidents into permanent hierarchy. Evaluation should therefore include failure cases where agents disagree, return late, repeat work, or provide incomplete evidence. A team architecture is robust when the coordinator handles those cases predictably rather than assuming every worker will always return a perfect answer.
Unknown is not untrustworthy
The product should treat uncertainty as state. A worker can return complete, blocked, partial, disputed, or needs-review rather than forcing every interaction into a success/failure binary that hides coordination problems.
Reputation can become self-fulfilling if routing never explores alternatives
Bad reputation mechanisms create feedback loops. A favored agent gets more tasks, gains more data, and appears increasingly proven; neglected agents get fewer opportunities and remain uncertain. Designers should separate “unknown” from “bad,” keep confidence intervals, and use evaluation tasks to refresh evidence independent of production traffic. Reputation without exploration can become a popularity contest among components. For gamified AI, progress indicators should reflect durable task state rather than internal activity. Spawning three workers is not three units of progress. A milestone is earned when a verified dependency is resolved, an artifact is accepted, or the user’s objective measurably advances.
Failures should update the relevant dimension rather than poison identity
A failure should update the dimension the failure actually reveals. One timeout may lower availability confidence but say little about reasoning quality. A policy violation is different from a formatting error. A domain-specific factual miss should not automatically revoke access to unrelated capabilities. Fine-grained records support safer recovery because the coordinator can reroute the affected class of work without discarding a useful agent entirely. Designers should also protect reversibility. When a coordination decision can be changed cheaply, the system can explore. When it changes external state or user commitments, ownership, approval, and rollback need to be explicit before the action occurs.
Reputation should decay
The final system should preserve one accountable path back to the user’s objective. Internal agents can be replaced, rerouted, or upgraded, but the product must still explain what is happening in terms the user can understand and control.
Use reputation as evidence for a decision, not as a moral label
The goal of reputation is not to create machine status. It is to make delegation more informed. The best design treats reputation as an evidence layer that can be inspected, challenged, and overridden under policy. Trust should remain contextual, revisable, and tied to consequences. When a system starts using one opaque score to decide every opportunity, it has recreated the weaknesses of simplistic human reputation systems inside an automated hierarchy. This keeps the architecture aligned with human agency: the team may be complex internally, but the user should retain a clear view of the objective, meaningful choices, and the consequences of agent actions. Multi-agent sophistication is valuable only when it makes that relationship stronger rather than more opaque.
Works Cited
- 01
- 02
- 03A2A Protocol Working Group — Agent2Agent Protocol Specification a2a-protocol.org
- 04NIST — AI Risk Management Framework Core airc.nist.gov
- 05OpenAI Developers — Multi-agent developers.openai.com
CodeHistory is a living archive. Citations document the evidence used for this edition; later evidence may refine the account.