I expected this to be simple. Download a model, open a chat, done. That’s not exactly how it went.
A few weeks ago I decided to stop relying entirely on cloud AI tools and start running models locally on my Acer Predator laptop. I had read about Ollama and LM Studio, both are tools that lets you run open-source large language models directly on your machine. What followed was a series of small frustrations, unexpected discoveries, and eventually something that actually works.
The Python Version Problem
The first thing that tripped me up was Python. I downloaded the latest version, Python 3.14, assuming newer always means better. It did not allow me to run Ollama. Ollama’s dependencies require Python 3.12. I had to uninstall, go back, download the right version, and start over. Small thing, but nobody warns you about it.
Ollama Is Not Enough on Its Own
I thought downloading Ollama and pulling a model would give me a full chat interface. It doesn’t. Ollama is essentially a backend engine. It runs the model, manages memory, and exposes an API. To have a clean interface where you can chat, upload documents, and switch between models, you need Open WebUI installed separately.
Once I figured that out and got Open WebUI running at localhost:8080, everything clicked. The interface is clean, it supports multiple models, and it feels close to what you’d expect from a commercial AI product.
The Models I Installed
I ended up running two models: Qwen3 30B A3B and Gemma 4 26B. Both are large models, the kind that would have required serious server hardware just two years ago. On my laptop they run slower than Claude or Gemini, obviously. But the quality of responses genuinely surprised me. For drafting, summarizing, reasoning through problems both models hold up well.
The PDF Problem and Nomic Embed Text
Getting document uploads to work in Open WebUI was the other headache. PDFs and Word files weren’t processing correctly until I found out I needed to install a separate embedding model called nomic-embed-text. I still don’t fully understand what it does under the hood, something about converting documents into a format the model can search through, but once I ran the install command, document uploads started working.
Why This Matters to me
Running AI locally means my data doesn’t leave my machine. For compliance consulting and AI governance work, that’s not a small thing, it’s a meaningful difference. I’m not there yet in terms of replacing cloud AI for everything, but having a local setup that actually functions is a real step forward.
If you’re thinking about doing this yourself, know that the path isn’t perfectly smooth. But it works. And working, even imperfectly, is a good place to start.