I built a private AI voice assistant on a gaming laptop using Ollama, Open WebUI, Kokoro TTS, and Cloudflare — no cloud inference required.
Disclaimer: This article documents a personal technical experiment on my own infrastructure. The configurations, rules, and steps described reflect what worked in my specific environment and should not be treated as security advice, professional recommendations, or guaranteed protection against any threat. Every environment is different. Always validate independently, test in a non-production setting first, and consult qualified security professionals before applying any security-related configuration to production systems. NosisTech LLC accepts no liability for outcomes resulting from use of this content. This documents a specific configuration that worked in my environment. Hardware specs, Windows versions, network conditions, and software versions all affect results. Nothing here is a prescription. Treat it as a reference, not a recipe.
I wanted a local AI assistant that could talk back to me. Not a chatbot in a browser tab. Something I could speak to, that would respond out loud, that knew my business context, and that kept every word on my machine.
This is what I built. Here is how it works, what broke along the way, and what the final setup looks like.
The Hardware
The machine is an Acer Predator with an RTX 5070 Ti with 12GB GDDR7 VRAM, and 32GB DDR5 RAM at 6400MHz, Windows 11 Spanish. This matters because the Spanish locale moves the Desktop to a different path than you expect, which caused at least one mistake I will get to.
The Stack
Four components make this work together:
Ollama handles local model serving. It runs as a background process and exposes a local API. I installed it via the Windows installer. No Docker, no WSL, no complications.
Open WebUI is the interface. I installed version 0.9.2 via pip into a Python 3.12.10 environment, not Docker. It runs as a Windows Task Scheduler task so it starts automatically on boot. I access it through a browser on any device on my network.
Kokoro TTS runs on port 8880 and provides the voice. I connected it to Open WebUI via the Audio settings using the OpenAI-compatible endpoint. Voice is bm_lewis. The model field is kokoro.
Whisper handles speech-to-text locally. Open WebUI uses faster-whisper internally with the base model. It converts my voice to text before sending it to the language model.
The Models
My primary model for text work is qwen36-tuned, a custom Modelfile build on top of qwen3.6:35b-a3b-q4_K_M. That model is 36B parameters and runs across VRAM and shared memory on my setup.
For my AI assistant specifically, I settled on gemma3:12b as the voice brain. More on why in the next post.
The Knowledge Base
Open WebUI has a built-in RAG system. I created a collection called JARVIS Vault and uploaded documents covering my business context, service offerings, cert roadmap, and content strategy. The embedding model is nomic-embed-text. Hybrid search is on. Relevance threshold is 0.3. Top K is 5.
When JARVIS answers a question, it searches the Vault first, then uses the language model to generate a response grounded in that context.
Remote Access
I access Open WebUI from my phone and from outside my home network through a Cloudflare Tunnel. The tunnel points to Open WebUI’s local port. Cloudflare Access adds OTP authentication so only I can reach it. The cloudflared service runs as a Windows service with a config file at C:\Users\[username].cloudflared\config.yml.
This replaced an earlier attempt at using a subdomain directly, which I removed after deciding the added complexity was not worth it for my setup.
What AI Assistant Can Do
I speak a question out loud. Whisper converts it to text. The text goes to the language model. The model generates a response. Kokoro speaks the response back. The whole loop, when working correctly, takes between 5 and 30 seconds depending on the question length and which model is loaded.
AI assistant knows who I am, what my business does, what certifications I am working toward, and how I like to be spoken to. That context lives in the JARVIS Vault and in the system prompt baked into the model itself.
What I Learned
Twelve gigabytes of VRAM is enough to run a capable voice assistant if you choose the right model size. Bigger is not better for real-time voice. A model that responds in 5 seconds is worth ten times more than one that responds in 3 minutes.
The system prompt matters more than the model size for behavioral compliance. A smaller model with tight behavioral rules beats a larger model with a loose prompt every time.
Local AI for voice is real, usable, and private. The setup took several sessions to stabilize. The next post covers all the places it broke and how I fixed them.