8/11/2025

The Definitive Guide to Installing Ollama and OpenWebUI on Ubuntu with an NVIDIA GPU

Hey everyone! So, you've been hearing all the buzz about running large language models (LLMs) on your own machine. It's a game-changer, honestly. No more API fees, no more rate limits, just pure, unadulterated AI power right at your fingertips. If you've got an Ubuntu machine with a decent NVIDIA graphics card, you're sitting on a goldmine. The problem is, getting it all to play nice can sometimes feel like a bit of a dark art.

I've been there, banging my head against the wall, trying to figure out why my fancy GPU was sitting idle while my CPU was screaming for help. But don't worry, I've gone through the pain so you don't have to. This guide is the culmination of a LOT of trial and error, forum-diving, and documentation-reading. We're going to walk through every single step to get Ollama and the gorgeous OpenWebUI running flawlessly on your Ubuntu system, fully accelerated by your NVIDIA GPU.

Let's get this rig built.

First Off, Why Bother Running This Locally?

Before we dive into the technical stuff, let's talk about why this is so COOL.

Total Privacy: The models run on your hardware. Your conversations, your data, your prompts—it all stays with you. This is HUGE.
No More Bills: Experimenting with different models via APIs can get expensive. Running them locally costs nothing but the electricity you're already using.
Customization & Control: You can download any compatible model you want, from massive 70-billion parameter beasts to small, nimble models perfect for coding assistance. You're in complete control.
It's Just Fun: Seriously, there's nothing quite like having your own private AI assistant that you can tinker with. The learning potential is massive.

This setup is perfect for developers, hobbyists, and anyone who's curious about the future of AI. You can build incredible things, from a chatbot that knows your personal documents inside & out to a home automation brain that runs entirely offline.

The Pre-Flight Checklist: What You'll Need

Let's get our ducks in a row before we start slinging commands. Here's what you should have ready.

An Ubuntu Machine: This guide is specifically for Ubuntu. I'm using Ubuntu 24.04 Desktop, as some folks have run into issues with the Server version out of the box.
An NVIDIA GPU: This is the star of the show. You'll want a card with at least 6GB of VRAM, but honestly, more is better. Something from the RTX 20-series, 30-series, or 40-series is ideal. The key is that it has a CUDA Compute Capability of 5.0 or higher. If you're not sure, you can check NVIDIA's official list.
Decent RAM: Aim for at least 16GB of system RAM for running 7B (7 billion parameter) models, and 32GB or more for the larger ones.
Disk Space: These models are not small. Make sure you have at least 30-50GB of free space on a fast drive (like an SSD) to be comfortable. Some models alone can take up over 20GB.
A Fresh Mindset: If you've tried installing NVIDIA drivers before & failed, we're going to wipe the slate clean. It's the most reliable way to ensure success.

Ready? Let's do this.

Part 1: The Foundation - Getting Your NVIDIA GPU Ready

This is the most critical part. If your NVIDIA drivers aren't set up correctly, Ollama won't be able to use your GPU, and you'll be stuck in the slow lane.

Step 1: Update Your System & Check for the GPU

First things first, let's open a terminal and make sure your system is up-to-date.