I set all of this up and do not fully understand everything I did. But it works, it is fast enough to be useful.

That is the point of documenting it.

The Hardware

My machine is an Acer Predator PHN16S-71. CPU: Intel Core Ultra 9 275HX. GPU: NVIDIA RTX 5070 Ti with 12GB VRAM. RAM: 32GB DDR5 at 6400 MT/s. The GPU is the engine. The RAM is the overflow tank. The gap between them, 12GB VRAM versus 32GB total RAM, is the central tension in everything that follows.

The Models

I am running two models. Gemma4 26B is Google’s multimodal model, capable of processing images alongside text. At Q4_K_M quantization it weighs 19GB. It benchmarks at 26.39 tokens per second on my hardware and is my choice for long-form writing and complex reasoning. Qwen3 30B A3B is a Mixture of Experts model from Alibaba. Despite the 30B label, only about 3 billion parameters activate per token, which is why it runs at 41.64 tokens per second which is 58% faster than Gemma4. It is my daily driver for fast iterative work. Both run through Ollama, with Open WebUI as the interface.

The Problem I Was Solving

When I ran ollama ps I saw that Gemma4 26B was loading at 44% CPU / 56% GPU. That 44% CPU means nearly half the model was sitting in regular RAM instead of the GPU. Every token generated required pulling data from both places. The default context window was also 4096 tokens —which is not enough for writing a full article or reviewing a long document without the model losing track of what it started with.

Context Window: 4096 → 8192

In Open WebUI’s model settings, under Advanced Parameters, there is a field called num_ctx. This controls how many tokens the model can hold in working memory at once. A token is roughly three-quarters of a word. So 4096 tokens is approximately 3,000 words. Doubling it to 8192 means the model can now hold around 6,000 words in context simultaneously which is enough to maintain coherence across a full article or compliance document. I still have to play around with higher number but don’t think my laptop can handle them.

GPU Layers: Default → 99

The num_gpu parameter tells Ollama how many layers of the model to load onto the GPU. Setting it to 99 is effectively saying: load as many as physically fit. My VRAM is 12GB. Gemma4 26B is 19GB. Some of the model will always spill into RAM. But maximizing GPU layers means the most computationally intensive parts of inference run on hardware designed for exactly this kind of parallel matrix math. The practical result is faster token generation and more consistent response speed. I inmediately noticed the change in my gpu on task manager.

Temperature: Default → 0.7

Temperature controls the randomness of the model’s word choices. At zero, the model always picks the statistically most likely next word — accurate but repetitive. At one, it takes creative risks that can drift into incoherence. At 0.7, the output reads like a confident human writer: direct, natural, and always controlled. For professional writing this is the right balance. I tried different temperatures and from .5 to .7 seemed to work for me. Try not to go above that because you might get incoherence.

PredatorSense Configuration

Running AI models is a sustained compute workload. I configured PredatorSense with Scenario Profile set to Gaming, Mode on AC set to Turbo, Fan Control set to Max, and the linked app set to ollama.exe. By linking ollama.exe to the Gaming profile, the laptop automatically switches to Turbo and Max Fan the moment Ollama launches, and returns to the Daily Use profile when I close it. No manual switching required.

What This Actually Means

I now have two private AI models running on hardware I own, with no data leaving my machine, no monthly API costs for local tasks, and configurations tuned for the kind of work I actually do: writing, document generation, governance research, and long-context analysis. Neither model replaces Claude Sonnet, Opus, or Gemini Pro for my most complex work. But for drafting, research, and experimentation especially when I want to keep mine or client context completely private, this setup is genuinely capable.