What This Agent Does
The Knowledge Gap Discovery Agent reviews a small fictional product knowledge corpus and finds places where customer signals point to an unresolved need. It is designed around a simple question: what are customers and teams repeatedly mentioning that nobody has directly studied yet?
The agent does not let the model choose the top gap. Python scans the notes, applies three gap-detection strategies, ranks the evidence, and selects the top gap first. Only after that does the model draft a hypothesis, and the code checks that the model stayed inside the known facts.
The Missing Shelf
The easiest way to understand this build is to imagine a warehouse.
Every shelf has a label. One shelf has customer support notes. Another has product notes. Another has design notes. Another has old interview findings.
A normal search agent looks for items already on the shelves. This agent looks for the empty shelf that keeps being referenced.
That is the knowledge gap.
For example, if support keeps hearing about smart lighting routines, and product keeps mentioning the same topic, but there is no direct research note studying that issue, the empty shelf becomes visible. The agent is not claiming the company has found the answer. It is saying, “This topic keeps appearing across the map, but the investigation is missing.”
That is useful because product teams often drown in notes. The hard part is not having more feedback. The hard part is knowing which missing question deserves attention next.
Vocabulary You Need
Knowledge gap: A missing piece in the company’s understanding. In this build, a gap appears when customer or team signals point to a topic that has not been directly studied.
Corpus: The collection of notes the agent reads. Here, the corpus is a small fictional set of product notes, support notes, and team observations.
Deterministic ranking: Ranking done by normal Python rules, not by a model’s opinion. The same evidence produces the same top gap.
Hypothesis: A testable idea about what the gap might mean. In this build, the hypothesis is only a starting point for investigation.
Output contract: A strict format and fact check for model output. If the model misses the required fields or adds unsupported terms, the agent uses fallback text.
What This Is Not
This is not a scientific discovery system, market research platform, customer analytics product, or production roadmap tool.
It does not pull from live customer data. It does not browse reviews. It does not rank real products. It does not decide what a company needs to build.
This is a small educational rebuild that isolates one pattern: scan local evidence, detect gaps, rank them deterministically, ask the model for a hypothesis, and reject the hypothesis if it drifts outside the record.
Why I Built This
A model can write a convincing product idea even when the evidence is thin, scattered, or unsupported. The output sounds strategic, but the path from evidence to idea is often invisible.
I wanted the opposite. First, the code finds the gap. Then the model gets a narrow writing job. The model is not the strategist in this build. It is the drafting layer after the evidence gate.
What Happened
What surprised me: Both providers fell back on the deterministic hypothesis. That was not a failure of the build. It proved the output contract was strict enough to reject model text that did not pass the gate.
What I would change next: I would add a small display of all ranked gaps, not only the top one. That would make the evidence map easier to inspect while keeping the model away from the ranking step.
Why this matters: Product teams do not only need more AI-generated ideas. They need a visible chain from evidence to gap to hypothesis.
The Architecture: How It Works
The agent has four stations.
The first station checks the notes. Each note needs an organization, team, topic, mention count, direct-study flag, age, and recent mention count.
The second station groups notes by organization and topic. This lets the agent see when two teams are pointing at the same issue.
The third station detects gaps using three strategies: referenced but unstudied, cross-team intersection, and stale but active.
The fourth station lets the model draft a hypothesis for the top gap. Then the output contract checks whether the model used the right gap id, returned the required fields, and avoided unsupported high-risk language.
What to Watch When You Run It
The first thing to watch is that both providers select the same top gap. That is the point of deterministic ranking. The model changes, but the selected gap stays stable.
The second thing to watch is the hypothesis status. In the final run, both providers returned fallback. That means the model drafting layer did not pass the contract gate, so the agent used deterministic text.
The third thing to watch is the audit id. Each provider run records the gap id, organization, topic, strategy, score, and hypothesis status in an in-memory audit entry.
The Build: Step by Step
REQUIRED_FIELDS = {"note_id": str, "organization": str, "team": str, "topic": str, "mentions": int, "studied_directly": bool, "age_days": int, "recent_mentions": int}
def validate_note(note):
"""Return an error string if a note is invalid."""
if not isinstance(note, dict):
return "Note is not a dictionary."
for field, kind in REQUIRED_FIELDS.items():
if field not in note or not isinstance(note[field], kind):
return "Field " + field + " has the wrong type."
for field in ["mentions", "age_days", "recent_mentions"]:
if note[field] < 0:
return "Field " + field + " must be zero or greater."
return NoneWhat This Code Is Actually Doing
This is the intake desk. The agent checks that every note has the fields needed for the rest of the workflow. A knowledge gap system is only useful if the evidence entering it has a consistent shape.
def build_corpus():
"""Return a fictional product knowledge corpus."""
return [
{"note_id": "LH-01", "organization": "Lumina Home Goods", "team": "Product", "topic": "smart_lighting_routines", "mentions": 6, "studied_directly": False, "age_days": 120, "recent_mentions": 5},
{"note_id": "LH-02", "organization": "Lumina Home Goods", "team": "Support", "topic": "smart_lighting_routines", "mentions": 4, "studied_directly": False, "age_days": 115, "recent_mentions": 4},
{"note_id": "VF-01", "organization": "Vector Freight", "team": "Ops", "topic": "route_delay_notifications", "mentions": 7, "studied_directly": False, "age_days": 60, "recent_mentions": 6},
{"note_id": "CD-01", "organization": "Cedar Desk Software", "team": "Engineering", "topic": "offline_sync_conflicts", "mentions": 8, "studied_directly": False, "age_days": 180, "recent_mentions": 7},
]What This Code Is Actually Doing
This is the evidence shelf. The corpus is fictional on purpose, but it acts like a small collection of product notes. Each note says who saw the signal, what topic it was about, how often it appeared, and whether the team studied it directly.
def make_gap(number, org, topic, strategy, note_ids, scores):
"""Create one scored gap dictionary."""
novelty, evidence, testability = scores
return {"gap_id": "GAP-" + str(number).zfill(3), "organization": org, "topic": topic, "strategy": strategy, "evidence_note_ids": note_ids, "novelty": novelty, "evidence_strength": evidence, "testability": testability, "score": round((novelty + evidence + testability) / 3, 3)}What This Code Is Actually Doing
This is the label maker. When the agent finds a gap, it gives that gap an id, records the organization and topic, names the strategy that found it, and calculates a score. The score is simple so the ranking is easy to inspect.
def detect_gaps(corpus):
"""Validate notes and return gap candidates from three strategies."""
notes, gaps = [], []
for note in corpus:
error = validate_note(note)
if error:
print("Invalid note skipped: " + error)
else:
notes.append(note)
grouped = {}
for note in notes:
grouped.setdefault((note["organization"], note["topic"]), []).append(note)What This Code Is Actually Doing
This is the sorting table. The agent validates each note, keeps the usable ones, and groups them by organization and topic. Grouping matters because a single note can be noise, but repeated signals across a topic can reveal a pattern.
for (org, topic), items in grouped.items():
ids = [item["note_id"] for item in items]
if sum(item["mentions"] for item in items) >= 4 and not any(item["studied_directly"] for item in items):
gaps.append(make_gap(len(gaps) + 1, org, topic, "referenced_but_unstudied", ids, (8, 7, 7)))
if len({item["team"] for item in items}) >= 2:
gaps.append(make_gap(len(gaps) + 1, org, topic, "cross_team_intersection", ids, (9, 8, 8)))
if any(item["age_days"] >= 90 and item["recent_mentions"] >= 3 for item in items):
old_ids = [item["note_id"] for item in items if item["age_days"] >= 90 and item["recent_mentions"] >= 3]
gaps.append(make_gap(len(gaps) + 1, org, topic, "stale_but_active", old_ids, (7, 8, 6)))
return gapsWhat This Code Is Actually Doing
This is the gap detector. It looks for three kinds of missing shelves. A topic can be referenced but unstudied, spread across multiple teams, or old but still active. Those three signals help the agent find questions worth investigating.
def rank_gaps(gaps):
"""Sort gaps by score descending, then gap id ascending."""
return sorted(gaps, key=lambda gap: (-gap["score"], gap["gap_id"]))What This Code Is Actually Doing
This is the ranking step. Python sorts the gaps before the model gets involved. That means the model is not choosing the priority. The model only writes after the top gap is already selected.
def draft_hypothesis(client, model_name, gap):
"""Request and validate a compact hypothesis JSON object."""
prompt = ("Return compact JSON only with gap_id, hypothesis, why_this_gap_matters, simple_test_plan. "
"Use only the provided gap facts. Do not invent metrics, dates, teams, customers, revenue, compliance, legal, medical, financial, or security claims. Gap facts: " + json.dumps(gap))
raw = call_model(client, model_name, prompt)
start, end = raw.find("{"), raw.rfind("}") + 1
if start == -1 or end <= start:
return fallback_hypothesis(gap), "fallback"What This Code Is Actually Doing
This is the hypothesis drafting station. The model receives the selected gap and a narrow instruction: return structured JSON and stay inside the facts. If the model does not return usable JSON, the agent uses fallback text.
try:
parsed = json.loads(raw[start:end])
except json.JSONDecodeError:
return fallback_hypothesis(gap), "fallback"
required = ["gap_id", "hypothesis", "why_this_gap_matters", "simple_test_plan"]
combined = " ".join(str(parsed.get(key, "")) for key in required).lower()
if parsed.get("gap_id") != gap["gap_id"] or any(not parsed.get(key) for key in required) or any(term in combined for term in FORBIDDEN_TERMS):
return fallback_hypothesis(gap), "fallback"
return parsed, "accepted"What This Code Is Actually Doing
This is the contract gate. The agent checks that the model used the correct gap id, filled in every required field, and avoided unsupported high-risk terms. If the output drifts, the agent keeps the workflow moving with deterministic fallback text.
def record_audit(audit_log, model_name, gap, status):
"""Append a hashed audit entry to the in-memory audit log."""
entry = {"audit_id": "AUD-" + str(len(audit_log) + 1).zfill(4), "model_name": model_name, "gap_id": gap["gap_id"], "organization": gap["organization"], "topic": gap["topic"], "strategy": gap["strategy"], "score": gap["score"], "hypothesis_status": status}
entry["hash"] = hashlib.sha256(json.dumps(entry, sort_keys=True).encode()).hexdigest()
audit_log.append(entry)
return entryWhat This Code Is Actually Doing
This is the audit log. The entry records which model ran, which gap was selected, which strategy found it, what score it received, and whether the hypothesis was accepted or replaced with fallback text.
Errors I Hit During This Build
Model hypothesis fallback:
Both gemini-flash and deepseek-v4-pro returned fallback hypothesis text in the final runtime. That was acceptable for this build because the contract gate protected the output instead of letting unsupported model text through.
Why This Matters
The larger lesson is that product discovery needs evidence discipline.
AI can write product ideas quickly. That is useful, but speed alone does not create trust. The real leverage comes from showing which evidence created the gap, which rule surfaced it, and whether the model stayed inside the record.
This build turns product discovery into an inspectable chain: notes, gap, rank, hypothesis, gate, audit. That is the part worth carrying forward.
Tools Used
LiteLLM (MIT): https://github.com/BerriAI/litellm
OpenAI Python SDK (Apache-2.0): https://github.com/openai/openai-python
python-dotenv (BSD-3-Clause): https://github.com/theskumar/python-dotenv
NosisTech Agent Engineering (MIT): https://github.com/nosistech/agent-engineering
(c) 2026 NosisTech LLC. Licensed under CC BY 4.0.
Use freely, just credit us.