Every powerful AI agent needs a way to decide where its answer should come from.
Some questions should be answered from private company knowledge. Some questions require current outside information. The mistake is letting the model guess when the source is unclear.
This build demonstrates a simplified version of the Phidata-style pattern: internal knowledge first, web fallback second, and source transparency every time.
The important part is not the framework. The important part is the routing decision.
What This Agent Does
This agent acts as a small research assistant.
When a question comes in, it checks a private internal knowledge file first. Think of that file like a company binder sitting on the agent’s desk. If the answer is in the binder, the agent uses it.
If the internal knowledge does not contain a strong enough match, the agent goes outside the binder and performs a web search. That is the fallback path.
Every response also prints the source used:
[SOURCE] internal knowledge
or:
[SOURCE] web search
That source label matters. It tells the analyst whether the answer came from the company’s own material or from outside information.
The agent is built in Python on top of a LiteLLM-compatible HTTP endpoint. No Phidata package is installed. No LangChain. No SQLite memory. No embeddings. No vector database. No third-party Python dependencies.
The whole lesson is source routing.
Why I Built This
Phidata’s architecture is useful because it gives builders a clean way to think about agents. The framework talks about memory, knowledge, and tools.
Memory means the agent remembers prior conversations.
Knowledge means the agent can use private information instead of relying only on model training.
Tools mean the agent can reach outside itself and take an action, like searching the web.
Those are strong ideas. But for this post, I did not want the framework to hide the pattern. If the goal is to understand the architecture, the first version should be small enough to read.
So I rebuilt the core routing idea in plain Python.
The question I wanted to answer was simple: can an agent check internal knowledge first, decide when that knowledge is not enough, then fall back to a tool while telling the user exactly which source it used?
That sounds basic, but it is one of the most important business patterns in agent design.
A founder does not want an AI system that treats company policy, outdated training data, and live web snippets as if they are all the same. Source matters. Internal knowledge and external search carry different levels of trust.
What Happened
What I expected: A complicated framework-style build with memory, embeddings, and multiple moving parts.
What actually happened: The simplified repo became much clearer when I removed the pieces that belonged in other posts. Memory is already covered elsewhere. Embeddings are already covered elsewhere. This post needed to teach routing.
What broke: The older version depended on embedding calls, model routing, and database memory. Those parts created complexity that distracted from the source-routing lesson.
What surprised me: The plain Python version explained the architecture better than the larger version. Once the question became “internal source or external source,” the whole agent was easier to reason about.
What I would change next: I would add stronger source scoring later, but only after the basic routing pattern is completely clear.
Why this matters: Business users need to know where an answer came from. A useful agent should not blur internal company knowledge and outside search results into one vague response.
The Architecture: How It Works
The agent has three layers that activate in sequence.
First, it loads the internal knowledge file. This is a plain text file called knowledge.txt. The file is split into chunks using blank lines. Each chunk becomes one piece of internal knowledge the agent can search.
Second, it compares the user’s question against those internal chunks using word overlap. Word overlap means the agent counts how many meaningful words appear in both the question and the internal knowledge. This is not as advanced as embeddings, but it is much easier to understand.
Third, if internal knowledge does not produce a good enough match, the agent calls a web search fallback. In this repo, the fallback uses DuckDuckGo’s public instant answer endpoint through Python’s standard library.
Then the selected context goes to the model.
The model does not receive every possible source. It receives the source the routing logic selected. That keeps the prompt clean and makes the answer easier to audit.
Pillar One: Internal Knowledge
The internal knowledge file is the agent’s first stop.
Imagine a front desk employee with a company handbook. When someone asks a question, the employee should check the handbook before calling outside for help.
That is what this agent does.
The repo version does not use embeddings. It does not convert paragraphs into vectors. It simply reads text chunks and compares words.
This is less powerful than meaning-based search, but it is a better teaching version because the logic is visible.
Pillar Two: Source Routing
Source routing is the decision layer.
The agent asks: “Do I have enough internal information to answer this?”
If yes, it uses internal knowledge.
If no, it uses web search.
This is like a receptionist deciding whether a question can be answered from the office binder or whether it needs an outside lookup. The value is not only the answer. The value is knowing which path was used.
Pillar Three: Web Search Fallback
The web search fallback is the agent’s outside tool.
When internal knowledge is not enough, the agent makes a search request and uses the result as context for the model.
This prevents the agent from pretending internal knowledge contains something it does not contain. Instead of stretching weak context, it changes sources and labels that change clearly.
That is the business lesson. The agent should not force every question through the same channel.
The Model-Agnostic Layer
The model call routes through a LiteLLM-compatible HTTP endpoint.
The code reads these values from the environment:
LITELLM_BASE_URL MODEL_NAME LITELLM_API_KEY KNOWLEDGE_FILE
That means the model provider is configuration, not business logic.
The code does not import the LiteLLM Python package. It does not import the OpenAI SDK. It uses Python’s built-in HTTP tools to send a request to the configured endpoint.
This keeps the dependency list extremely small. In fact, the current requirements.txt has no Python package dependencies.
What If You Want to Run This Locally?
The current repo is designed around a LiteLLM-compatible endpoint. That endpoint can route to different supported providers depending on how your environment is configured.
The important point is that agent.py does not need to know the provider details. It only needs a base URL, a model name, and an API key.
For a local setup, the same idea applies. The local model route would be configured behind the LiteLLM-compatible endpoint, and the Python file would still send the same type of request.
That is the value of the model-agnostic layer. The agent’s job is source routing. The endpoint’s job is model routing.
Those responsibilities stay separate.
The Build: Step by Step
Step 1: Load environment values.
def load_env():
env_path = PROJECT_DIR / ".env"
if not env_path.exists():
return
for line in env_path.read_text(encoding="utf-8").splitlines():
if line.strip() and "=" in line and not line.lstrip().startswith("#"):
key, value = line.split("=", 1)
os.environ.setdefault(key.strip(), value.strip())What This Code Is Actually Doing
This function reads the .env file.
Think of .env as a settings card for the agent. Instead of hardcoding the model name or API key inside the Python file, the agent reads those values from a separate file.
The function goes line by line. If a line is blank, starts with #, or does not contain an equals sign, it skips it. If the line contains a setting, it splits the line into a key and a value.
That is how a line like this becomes usable by the agent:
MODEL_NAME=YOUR_MODEL_NAME_HERE
The business value is simple: configuration changes without code changes.
Step 2: Check required configuration.
def load_config():
required = ["LITELLM_BASE_URL", "MODEL_NAME", "LITELLM_API_KEY", "KNOWLEDGE_FILE"]
missing = [name for name in required if not os.getenv(name)]
if missing:
print("[ERROR] Missing environment variables: " + ", ".join(missing))
sys.exit(1)
return {
"base_url": os.getenv("LITELLM_BASE_URL").rstrip("/"),
"model": os.getenv("MODEL_NAME"),
"api_key": os.getenv("LITELLM_API_KEY"),
"knowledge_file": os.getenv("KNOWLEDGE_FILE"),
}What This Code Is Actually Doing
This is the pre-flight checklist.
Before the agent searches knowledge or calls a model, it checks whether the required settings exist. If something is missing, it stops and tells you exactly what is missing.
That matters because beginners often run into confusing errors that appear much later in the process. This function catches the simple setup problems early.
It is like checking whether the car has fuel before diagnosing the engine.
Step 3: Load the internal knowledge file.
def load_knowledge(config):
path = project_path(config["knowledge_file"])
if not path.exists():
raise FileNotFoundError(f"Knowledge file not found: {path}")
return [chunk.strip() for chunk in path.read_text(encoding="utf-8").split("\n\n") if chunk.strip()]What This Code Is Actually Doing
This function opens the internal knowledge file and breaks it into chunks.
A chunk is just a block of text. The code uses blank lines as dividers. If the file has three paragraphs separated by blank lines, the agent treats those as three searchable pieces of knowledge.
Think of it like cutting a long company document into index cards. Each card contains one idea. When the user asks a question, the agent can compare the question against each card.
This is much easier to understand than a vector database, and for a teaching build, that clarity is the point.
Step 4: Turn text into searchable words.
def words(text):
stop_words = {
"a", "an", "and", "are", "does", "for",
"in", "is", "of", "on", "the", "to", "what"
}
return set(re.findall(r"[a-z0-9]+", text.lower())) - stop_wordsWhat This Code Is Actually Doing
This function turns a sentence into a clean set of useful words.
A set is a Python container that keeps only one copy of each item. If the word “services” appears three times, the set keeps it once.
The function also removes common words like “the,” “and,” and “what.” Those words appear everywhere, so they do not help the agent decide whether a chunk is relevant.
This is like removing filler words from a search query so the important terms stand out.
Step 5: Search internal knowledge first.
def search_internal_knowledge(query, chunks):
query_words = words(query)
external_words = {"competitor", "competitors", "latest", "market", "news", "pricing", "stock", "today"}
best_chunk = None
best_score = 0
for chunk in chunks:
chunk_words = words(chunk)
if query_words & external_words and not query_words & external_words & chunk_words:
continue
score = len(query_words & chunk_words)
if score > best_score:
best_chunk = chunk
best_score = score
return best_chunk if best_score >= 2 else NoneWhat This Code Is Actually Doing
This function is the internal librarian.
It receives the user’s question and the internal knowledge chunks. Then it compares the question against each chunk by counting shared words.
The & symbol means intersection. In plain English, it means “show me the words that appear in both places.”
There is also a small guardrail for external topics. If the user asks about things like competitors, market news, pricing, stock, or today’s information, the agent becomes more careful. It does not force an internal answer unless the internal chunk also talks about that external topic.
At the end, the agent only trusts the internal match if the score is at least 2. That means at least two useful words overlapped.
It is a simple rule, but it teaches an important habit: the agent needs a reason before it claims internal knowledge is relevant.
Step 6: Use web search when internal knowledge is not enough.
def web_search(query):
params = urlencode({
"q": query,
"format": "json",
"no_redirect": "1",
"no_html": "1",
})
request = Request(
f"https://api.duckduckgo.com/?{params}",
headers={"User-Agent": "NosisTech-Agent-Engineering-Demo"},
)
try:
with urlopen(request, timeout=10) as response:
data = json.loads(response.read().decode("utf-8"))
except Exception:
return "No web result was available."
if data.get("AbstractText"):
return data["AbstractText"]
for topic in data.get("RelatedTopics") or []:
if isinstance(topic, dict) and topic.get("Text"):
return topic["Text"]
return "No web result was available."What This Code Is Actually Doing
This function is the outside lookup desk.
If the company binder does not have the answer, the agent asks DuckDuckGo for a result. It first looks for an abstract, which is a short summary. If that is not available, it checks related topics.
If the search fails, the function returns a plain fallback message instead of crashing.
That matters because tools fail in the real world. Internet requests time out. Search APIs return empty results. A useful agent needs a graceful fallback when the outside tool does not produce anything.
Step 7: Choose the source and ask the model.
def answer_question(config, question):
context = search_internal_knowledge(question, load_knowledge(config))
source_label = "internal knowledge" if context else "web search"
context = context or web_search(question)
return call_llm(config, context, question, source_label), source_labelWhat This Code Is Actually Doing
This is the main routing desk.
First, the agent loads the internal knowledge. Then it searches that knowledge. If it finds a usable match, it keeps the source label as internal knowledge.
If it does not find a usable match, it calls web search and changes the label to web search.
Then it builds the message for the model. The message includes the source label, the selected context, and the user’s question.
The model is told to answer only from the provided context. That does not make hallucination impossible, but it does narrow the model’s job. Instead of asking the model to “know everything,” the agent gives it a specific source and asks it to answer from that source.
Step 8: Print the answer and the source.
def run_agent(config, query):
print(f"[USER] {query}")
answer, source_label = answer_question(config, query)
print(f"[AGENT] {answer}")
print(f"[SOURCE] {source_label}\n")What This Code Is Actually Doing
This function prints the user’s question, the agent’s answer, and the source that was used.
That last line is the point of the whole build.
The user does not have to guess whether the answer came from internal knowledge or web search. The agent tells them.
For a business user, that is like seeing a receipt attached to the answer.
Errors I Hit During This Build
In testing, the built-in demo answered “What services does NosisTech offer?” from internal knowledge. The competitor question fell back to web search, but DuckDuckGo did not return useful context, so the model said the provided context was insufficient. A separate test question, “What is OpenAI?”, verified the web fallback with a real web result.
The older version of this post included errors from the embedding-based build. Those errors no longer apply to the simplified repo because embeddings, cosine similarity, SQLite memory, the OpenAI SDK, and LiteLLM’s Python package are no longer part of this implementation.
The current simplified version has a smaller set of likely failure points.
Missing environment variables. If LITELLM_BASE_URL, MODEL_NAME, LITELLM_API_KEY, or KNOWLEDGE_FILE is missing, the agent stops early and prints the missing values. That is the setup checklist working as designed.
Knowledge file not found. If KNOWLEDGE_FILE points to a file that does not exist, the agent raises a clear file-not-found error. The fix is to create the knowledge file or update the path in .env.
Web result unavailable. If the web request fails or DuckDuckGo does not return a usable abstract or related topic, the fallback returns No web result was available. The agent still completes the flow instead of crashing.
Model endpoint unavailable. If the LiteLLM-compatible endpoint cannot be reached, the model call raises a connection error. The fix is environment-specific: confirm the endpoint is running and that the .env values match that environment.
Why This Matters
Every agent that handles company information needs a source strategy.
Without one, the model may blend private knowledge, training data, and outside snippets into one answer. That is dangerous because the user cannot tell what the answer is based on.
This build shows a cleaner pattern.
Internal knowledge gets the first chance. If it is enough, the agent answers from it. If it is not enough, the agent uses a web fallback. Either way, the source is labeled.
That is the minimum standard for a business research assistant. The answer should come with provenance. Provenance means knowing where something came from.
What This Means
If your team is evaluating AI agent frameworks, this pattern answers one of the most important business questions: where did the answer come from?
For a consulting firm, logistics operation, financial services team, or founder-led company, that matters immediately.
A question about your own services should come from your internal knowledge. A question about current competitors or market news may need an outside source. Those are different categories of information, and the agent should treat them differently.
The value here is not complexity. The value is judgment.
This simplified Phidata-pattern agent does one thing clearly: it routes the question to the right source, gives that source to the model, and tells the user which source was used.
That is how agent architecture starts becoming trustworthy to a non-technical operator. Not because the model is magic. Because the workflow is understandable.
Tools Used
LiteLLM (MIT): github.com/BerriAI/litellm
duckduckgo-search (MIT): github.com/deedy5/duckduckgo_search
python-dotenv (BSD-3-Clause): github.com/theskumar/python-dotenv
Phidata (MIT): github.com/phidatahq/phidata
NosisTech Agent Engineering (MIT): github.com/nosistech/agent-engineering
(c) 2026 NosisTech LLC. Licensed under CC BY 4.0. Use freely, just credit us