Most people interact with AI through someone else’s platform. You go to Claude.ai, or Gemini, or ChatGPT, and you use what they give you.
I wanted something different; my own server, running my own setup, where I controlled what models were available and how they connected. This is the story of how I built that, what broke along the way, and what I ended up with.
Starting with a Hostinger VPS
The foundation was a Virtual Private Server from Hostinger. A VPS is essentially a computer you rent in a data center, it runs 24/7, has a public IP address, and you connect to it remotely via SSH. If you’ve never used SSH before, the first time you type a command into a remote server is a strange feeling. You’re controlling a machine that isn’t in the room with you. I installed Ollama on the VPS first. Ollama is the engine that actually runs AI models, it handles loading them into memory, serving requests, and managing which model is active. Getting it installed was the straightforward part.
The Problem with Mixing APIs
Here’s something I didn’t know before this project: Ollama and Claude use different API formats. Ollama speaks in OpenAI-compatible format. Claude speaks in its own format. They don’t understand each other natively. To fix this I installed LiteLLM, a proxy tool that sits between your interface and the AI models, translating requests so everything speaks the same language. You configure it with a simple file, tell it which models to expose and what API keys to use, and it runs on port 4000 as a permanent background service. Once LiteLLM was running, my Open WebUI interface could see Claude models alongside the local Ollama models. That was the first time I had multiple AI providers in one place, all accessible from a single chat interface.
Adding Gemini – A Different Kind of Problem
Adding Gemini came later and involved its own set of friction. Google requires every API key to be attached to a Project in Google Cloud Console — something I didn’t know going in. The first time I tried to generate a key, the interface told me there were no projects to attach it to and I couldn’t proceed. I had to go to Google Cloud Console separately, create a project container first, then come back to generate the key. Once I had the key, adding Gemini to Open WebUI was actually elegant. Google exposes Gemini through an OpenAI-compatible endpoint, which means you don’t need any special plugin; you just add it as a custom connection with the right base URL and your API key. A few minutes later, Gemini 1.5 Flash and Gemini 1.5 Pro appeared in my model dropdown alongside everything else.
What I Have Now
From one interface, I can switch between Claude, Gemini, and local Ollama models depending on what a task needs. Cloud models for complex reasoning or client work. Local models for private documents or experimentation. I built this without a software engineering background. I got stuck multiple times. I had to look up what SSH was, what a proxy tool was, what most things presented here are. But the pieces exist, they’re designed to work together, and if you’re willing to sit with the confusion long enough, it comes together. That’s been the pattern with most of this infrastructure work. Not easy, but doable. And once it’s done, it’s yours.