I built a General Problem Solver agent that does not treat every business claim as equal. It checks the claim first, accepts what matches the trusted record, and escalates only the messy cases into deeper reasoning.
What This Agent Does
This agent reads a fictional business claim, extracts the key metric, and compares that metric against a trusted reference value. If the claim is close enough, the system accepts it.
If the claim conflicts with the trusted value, the system escalates to a General Problem Solver. That second stage breaks the discrepancy into smaller questions, looks for an analogy, proposes a hypothesis, scores confidence, and records one lesson from the reasoning cycle.
The Escalation Desk
The cleanest way to understand this build is an escalation desk.
A junior analyst checks the number first. If the claim says product returns were 8.3 percent and the trusted record says 8.1 percent, the difference is small enough for this demo to accept.
But if a company claims late shipments were 4 percent and the trusted record says 17.2 percent, that is not a small formatting issue. The case moves to the senior strategist. That strategist does not just answer. It asks what definition, scope, timing, or data pipeline issue could explain the mismatch.
Vocabulary You Need
Verification means checking a claim against a trusted reference before using it.
Tolerance means the allowed difference between two numbers before the system treats the mismatch as important.
Escalation means routing a harder case to a stronger reasoning process instead of forcing the first check to handle everything.
General Problem Solver means a structured reasoning loop that decomposes a problem, finds an analogy, forms a hypothesis, tests confidence, and learns from the result.
What This Is Not
This is not a production fact-checking system. It does not connect to real databases, scrape websites, verify legal claims, or certify business accuracy.
It is also not a full analytics platform. I removed notebooks, charts, statistical modeling, machine learning classifiers, and heavy data libraries. The point is the architecture: verify first, escalate only when the mismatch deserves reasoning.
Why I Built This
I wanted a pattern that shows the difference between answering and reasoning. A normal chatbot can explain a discrepancy after the fact. That is not enough.
The stronger architecture is a pipeline that decides when reasoning is needed. This build turns that idea into a small, inspectable Python agent that can run across providers through LiteLLM.
What Actually Happened
What happened: Gemini completed the pipeline, but it relied on deterministic fallbacks for the claim and GPS stages. DeepSeek produced richer GPS reasoning for the escalated cases.
What broke: Gemini returned nested JSON objects where the code expected plain strings. That caused a Python type error during output printing.
What surprised me: The failure was not bad reasoning. It was output shape drift. The model gave useful content in a structure the first parser did not accept.
What I would change next: I would add a small output normalizer for every structured model response before display.
Why this matters: Multi-provider agents need to handle meaning and shape separately. A model can be conceptually correct and still break the program if the schema is not normalized.
The Architecture: How It Works
The agent has three layers.
First, it extracts one business claim from fictional text. Second, it compares the extracted metric against a trusted in-memory value. Third, it escalates failed cases into the GPS reasoning cycle.
The key decision is simple: a claim that passes verification is accepted, while a claim that fails verification is routed into deeper reasoning.
What to Watch When You Run It
Watch the route line. ACCEPTED means the verification gate found the claim close enough to the trusted record.
ESCALATED_TO_GPS means the claim drifted too far and needed deeper reasoning. Provider differences matter here. In my run, Gemini exercised the fallback path, while DeepSeek generated full GPS reasoning for the failed cases.
The Build: Step by Step
SCENARIOS = [
{"name": "Lumina Home Goods", "article_text": "Lumina Home Goods reported an 8.3 percent product return rate for Q1.", "fallback_claim": {"claim_text": "product return rate", "metric_key": "return_rate_pct", "claimed_value": 8.3, "unit": "percent"}, "trusted_facts": {"return_rate_pct": 8.1}},
{"name": "Vector Freight", "article_text": "Vector Freight Solutions announced that only 4 percent of shipments arrived late last quarter.", "fallback_claim": {"claim_text": "late shipment rate", "metric_key": "late_shipment_pct", "claimed_value": 4.0, "unit": "percent"}, "trusted_facts": {"late_shipment_pct": 17.2}},
{"name": "Cedar Desk Software", "article_text": "Cedar Desk Software stated that its assistant resolves 91 percent of support tickets without human intervention.", "fallback_claim": {"claim_text": "automation containment rate", "metric_key": "automation_containment_pct", "claimed_value": 91.0, "unit": "percent"}, "trusted_facts": {"automation_containment_pct": 54.0}},
]What This Code Is Actually Doing
This is the fictional test bench. Each scenario gives the agent a short business claim, a backup structured claim, and a trusted reference number. The fallback claim matters because model output can vary. If a provider returns unusable JSON, the demo still proves the verification and escalation architecture.
def verify_claim(claim, trusted_facts):
"""Compare a claim to the trusted fictional reference value."""
metric_key = claim["metric_key"]
if metric_key not in trusted_facts:
return {"status": "FAIL", "reason": "No trusted record exists for this metric.", "trusted_value": None, "drift": None}
trusted_value = float(trusted_facts[metric_key])
drift = abs(float(claim["claimed_value"]) - trusted_value)
status = "PASS" if drift <= TOLERANCE else "FAIL"
reason = "Drift is within tolerance." if status == "PASS" else "Drift exceeds tolerance."
return {"status": status, "reason": reason, "trusted_value": trusted_value, "drift": drift}What This Code Is Actually Doing
This is the verification gate. It compares the claimed number with the trusted number. If the difference is small, it returns PASS. If the difference is too large, it returns FAIL. That failure is what activates the General Problem Solver.
def solve_problem(client, model_name, scenario, claim, verification):
"""Run the five-stage General Problem Solver cycle for failed claims."""
packet = {"scenario": scenario["name"], "claim": claim, "verification": verification}
prompt = (
"A fictional business claim failed verification. Return one JSON object with sub_problems, "
"analogy, hypothesis, confidence, test_plan, and meta_lesson. Follow this cycle: decompose "
"the discrepancy, use one cross-domain analogy, generate one testable hypothesis, score "
"confidence from 0.0 to 1.0, and state one meta-learning lesson. Context: "
+ json.dumps(packet)
)What This Code Is Actually Doing
This is the GPS handoff. The agent packages the failed claim and asks the model to reason in a controlled structure. The model is not asked for a loose essay. It is asked for sub-problems, an analogy, a hypothesis, a confidence score, a test plan, and one lesson.
def to_text(value):
"""Convert provider-shaped JSON values into safe display text."""
if isinstance(value, str):
return value
if isinstance(value, dict):
name, description = value.get("name"), value.get("description")
if name and description:
return str(name) + ": " + str(description)
if description or name:
return str(description or name)
return json.dumps(value, ensure_ascii=True)What This Code Is Actually Doing
This is the provider normalization layer. One model may return a plain sentence. Another may return an object with name and description. This function converts both into readable text before printing. That small layer fixed the crash I hit during testing.
def route_scenario(client, model_name, scenario):
"""Route one fictional scenario through verification and optional GPS escalation."""
print("\nScenario: " + scenario["name"])
claim = extract_claim(client, model_name, scenario)
verification = verify_claim(claim, scenario["trusted_facts"])
print("Claim: " + claim["claim_text"] + " = " + str(claim["claimed_value"]) + " " + claim["unit"])
print("Verification: " + verification["status"] + " - " + verification["reason"])
if verification["status"] == "PASS":
print("Route: ACCEPTED")
return
print("Route: ESCALATED_TO_GPS")What This Code Is Actually Doing
This is the trust-then-escalate router. The agent does not use the expensive reasoning path for every case. It only escalates when verification fails. That is the architectural lesson: reasoning is more useful when the system knows when to invoke it.
Errors I Hit During This Build
Error message: TypeError: can only concatenate str (not “dict”) to str
What caused it: Gemini returned nested JSON objects for some GPS fields. The code expected strings and tried to concatenate a dictionary into printed text.
How I fixed it: I added a normalization function that turns provider-shaped JSON into display-safe text. If the model returns a name and description, the agent combines them into a readable sentence.
Second issue: Gemini kept triggering deterministic fallbacks.
What caused it: The model output did not always match the strict JSON shape expected by the first parser.
How I fixed it: I made the claim prompt more specific by telling the model the exact metric key allowed for each scenario. I also kept the deterministic fallback because provider drift is part of the lesson.
Third issue: The first version accepted unknown metrics by default.
What caused it: The verifier originally treated a missing trusted record as acceptable.
How I fixed it: Unknown metrics now return FAIL, which routes the case to review instead of accepting an unverifiable claim.
Why This Matters
Business AI needs more than fluent answers. It needs control points.
This build shows one of those control points in a small form. A claim is checked before it is trusted. A mismatch gets escalated. The reasoning output is structured enough to inspect. That is the difference between an agent that talks and an agent that can participate in a governed workflow.
Tools Used
LiteLLM (MIT): https://github.com/BerriAI/litellm
NosisTech Agent Engineering (MIT): https://github.com/nosistech/agent-engineering
(c) 2026 NosisTech LLC. Licensed under CC BY 4.0.
Use freely, just credit us.