“Disclaimer: This article documents my personal exploration of AI governance frameworks and regulatory concepts. Nothing in this post constitutes legal advice, compliance certification, or a guarantee of regulatory conformance. AI governance requirements vary by jurisdiction, sector, and use case. Consult qualified legal and compliance professionals before making governance decisions. NosisTech LLC accepts no liability for outcomes arising from use of this content.”
I rebuilt a collective intelligence pattern into a small consensus engine. The point was not to make another chatbot. The point was to test what happens when one model response is replaced by a structured debate between specialized roles.
This build creates a tiny review board. One role looks at infrastructure risk. One role looks at governance and oversight. One role looks at cost, adoption, and operations. The system makes them propose, review, score, and synthesize instead of letting one answer dominate the entire decision.
What This Agent Does
The agent runs three fictional business scenarios through a small consensus loop. Each scenario is reviewed by a Cloud Security Architect, an AI Governance Reviewer, and a Business Operations Lead.
Each role proposes a plan. Then the roles review each other. One role temporarily becomes an adversarial critic, which means its job is to look harder for weak points instead of politely agreeing. The agent then scores the proposals, selects the top proposal, and asks the model to synthesize a final consensus plan.
The Review Board
The easiest way to understand this build is as a boardroom, not a chatbot.
A normal prompt is one person in a room giving an answer. This build puts three people in the room with different incentives. The security role sees access control and infrastructure risk. The governance role sees oversight, auditability, and policy fit. The operations role sees cost, training, and rollout friction.
That matters because real AI decisions do not fail in only one dimension. A plan can be secure but too expensive. It can be operationally attractive but weak on oversight. It can look compliant in language while still missing the basic implementation details needed for a controlled pilot.
Vocabulary You Need
Consensus engine: A control loop that collects multiple proposals, compares them, and produces one final answer.
Shared context: The common workspace where proposals, reviews, scores, and audit events are stored during the run.
Adversarial critic: A temporary role that reviews a proposal more aggressively to surface weak assumptions.
Structured output gate: A validation step that checks whether the model returned the shape the program expected.
Fallback: A deterministic response the program uses when the model output is incomplete or unusable.
What This Is Not
This is not a production governance system. It does not retrieve live regulations, verify legal obligations, or produce compliance certification.
It is not a security assurance tool. It does not scan infrastructure, inspect code repositories, or validate access controls in a real environment.
It is not a full multi-agent framework. I intentionally kept only the architecture needed to demonstrate proposal, critique, adversarial review, score parsing, fallback behavior, and synthesis.
Why I Built This
If an AI system is reviewing a deployment plan, I want it to expose competing lenses before producing the final answer. I want security, governance, and operations visible in the same run.
A small system can still show debate, review, score contracts, and fallback behavior without dragging in a full orchestration framework.
What Actually Happened
What surprised me: Gemini kept falling back to neutral review scores even after the prompt was tightened. DeepSeek gave stronger score variation but returned one empty synthesis, which forced the fallback consensus path to prove its value.
What I would change next: I would add an evidence packet registry so agents have to cite approved source material before debating. I would also add debate memory compression so long review loops can preserve the important objections without carrying the whole transcript forward.
Why this matters: The useful lesson is not that one provider scored better than another. The useful lesson is that a consensus architecture needs gates, fallbacks, and audit trails because model behavior varies even when the workflow stays the same.
The Architecture: How It Works
The architecture has five parts.
First, the agent loads provider configuration from .env. This keeps the model, base URL, and key outside the code.
Second, the system creates three role-specialized agents. These are not three different Python programs. They are three different role prompts using the same LiteLLM-compatible model connection.
Third, each agent proposes a plan for the same fictional scenario.
Fourth, each agent reviews the other proposals. The first reviewer temporarily becomes an adversarial critic, which changes the review posture.
Fifth, the engine averages the review scores, selects the top proposal, and asks the model to synthesize a final consensus. If synthesis comes back empty, the deterministic fallback produces a safe educational result.
What to Watch When You Run It
The first thing to watch is the review scores. Gemini completed the workflow, but its review outputs repeatedly failed the strict JSON gate and used the neutral fallback score of 5.0. DeepSeek returned more varied scores across the same review loop.
The second thing to watch is the final consensus. One DeepSeek run returned empty synthesis text, so the deterministic fallback filled the gap. That is the point of the fallback layer: the run still produces an inspectable result when the model output is incomplete.
The Build: Step by Step
REQUIRED_ENV = ("LITELLM_BASE_URL", "MODEL_NAME", "LITELLM_API_KEY")
def require_environment() -> tuple[str, str, str]:
"""Load required environment variables before any model call."""
load_dotenv()
missing = [name for name in REQUIRED_ENV if not os.getenv(name)]
if missing:
print("[ERROR] Missing environment variables: " + ", ".join(missing))
sys.exit(1)
return tuple(os.getenv(name, "") for name in REQUIRED_ENV)
BASE_URL, MODEL_NAME, API_KEY = require_environment()
MODEL_NAMES = [name.strip() for name in MODEL_NAME.split(",") if name.strip()]What This Code Is Actually Doing
This is the configuration gate. Before the agent talks to any model, it checks that the environment has the values it needs. The important detail is that MODEL_NAME can hold more than one provider, separated by commas, so I can test the same architecture against multiple models without changing the Python file.
def call_model(prompt: str, model_name: str, max_retries: int = 3) -> str:
"""Call the LiteLLM proxy with short exponential backoff."""
for attempt in range(max_retries):
try:
response = client.chat.completions.create(
model=model_name,
messages=[{"role": "user", "content": prompt}],
max_tokens=700,
)
return (response.choices[0].message.content or "").strip()
except Exception as error:
message = str(error).lower()
if "rate" in message or "429" in message:
print("[WARN] Rate limit reached. Retrying soon.")
time.sleep(2 ** attempt)
continue
print("[ERROR] The model call failed. Check your LiteLLM configuration.")
return ""What This Code Is Actually Doing
This is the model call wrapper. A wrapper is a small function that puts one controlled doorway around a repeated action. Instead of scattering model calls throughout the file, every prompt passes through this one function, which makes retries and friendly error messages easier to control.
@dataclass
class SharedContext:
proposals: dict[str, str] = field(default_factory=dict)
reviews: dict[str, dict] = field(default_factory=dict)
audit: list[str] = field(default_factory=list)
def log(self, entry: str) -> None:
"""Append one readable event to the audit trail."""
self.audit.append(entry)What This Code Is Actually Doing
This is the shared workspace for the debate. It stores what each role proposed, how each proposal was reviewed, and what happened during the run. In plain English, it is the meeting notes for the agent boardroom.
class CollaborativeAgent:
def __init__(self, role: str, focus: str) -> None:
"""Create an agent with a role and review focus."""
self.role = role
self.original_role = role
self.focus = focus
def set_adversarial(self) -> None:
"""Temporarily switch this agent into critic mode."""
self.role = "Adversarial Critic"
def reset_role(self) -> None:
"""Restore the agent role used for proposals."""
self.role = self.original_roleWhat This Code Is Actually Doing
This class gives one model multiple expert identities. A class is a reusable template for an object. Here, the object is an agent role with a name, a focus area, and the ability to temporarily become a critic before returning to its original role.
def parse_review(raw_text: str) -> dict:
"""Validate model review JSON or return a neutral fallback."""
fallback = {
"score": 5.0,
"risk": "medium",
"reason": "Review did not match the expected contract, so a neutral fallback was used.",
}
try:
review = json.loads(raw_text[raw_text.find("{"): raw_text.rfind("}") + 1])
score = float(review["score"])
risk = str(review["risk"]).lower()
reason = str(review["reason"]).strip()
except (ValueError, KeyError, TypeError, json.JSONDecodeError):
return fallback
if not 1 <= score <= 10 or risk not in {"low", "medium", "high"} or not reason:
return fallback
return {"score": score, "risk": risk, "reason": reason}What This Code Is Actually Doing
This is the output contract gate. The model is asked for JSON, which is a structured data format with keys and values. If the model returns prose, missing fields, a bad score, or an unsupported risk label, the program uses a neutral fallback instead of trusting malformed output.
critic = AGENTS[0]
critic.set_adversarial()
scorebook: dict[str, list[float]] = {role: [] for role in context.proposals}
for reviewer in AGENTS:
for role, proposal in context.proposals.items():
if role == reviewer.original_role:
continue
review = reviewer.review(proposal, model_name)
label = reviewer.role + " on " + role
context.reviews[label] = review
scorebook[role].append(review["score"])
context.log(label + " scored " + str(review["score"]) + " with " + review["risk"] + " risk.")
critic.reset_role()What This Code Is Actually Doing
This is the debate loop. One role becomes the critic, every role reviews the other roles, and the scores are stored by proposal owner. The self-review check prevents an agent from grading its own proposal, which keeps the demonstration cleaner.
averages = {role: sum(scores) / len(scores) for role, scores in scorebook.items() if scores}
selected_role = max(averages, key=averages.get)
consensus = call_model(prompt, model_name)
if not consensus.strip():
consensus = fallback_consensus(scenario, selected_role, context.proposals[selected_role])What This Code Is Actually Doing
This is the decision and fallback layer. The engine averages the scores, selects the strongest reviewed proposal, and asks the model for a final consensus. If the model returns empty text, the program uses a deterministic fallback, which means the fallback is produced by regular Python logic instead of another model guess.
Errors I Hit During This Build
Error: One DeepSeek run produced an empty final consensus.
Cause: The model returned no usable synthesis text for that step.
Fix: I added deterministic fallback synthesis so the run still produces an inspectable educational result.
Why This Matters
Everyone talks about multi-agent systems as if adding more agents automatically adds more intelligence. This build showed: the value comes from structure, not headcount.
A useful consensus system needs roles, context, scoring, adversarial pressure, validation, and fallback behavior. Without those pieces, a debate can become three prompts wearing different costumes.
This build gives me a compact pattern I can reuse in governance, security, operations, and product review workflows. It shows how to turn disagreement into architecture.
This build is a learning pattern, not an authority layer. I would not rely on this kind of consensus output for real governance, legal, compliance, security, or operational decisions without independent validation, qualified human review, and environment-specific controls.
Tools Used
LiteLLM (MIT): https://github.com/BerriAI/litellm
python-dotenv (BSD-3-Clause): https://github.com/theskumar/python-dotenv
OpenAI Python SDK (Apache-2.0): https://github.com/openai/openai-python
NosisTech Agent Engineering (MIT): https://github.com/nosistech/agent-engineering
(c) 2026 NosisTech LLC. Licensed under CC BY 4.0.
Use freely, just credit us.