So, you've got an AMD GPU & you're ready to dive into the world of local Large Language Models (LLMs) with Ollama. That's awesome! Running models like Llama 3 or Phi-3 on your own hardware is a game-changer for privacy, customization, & just plain fun. But, as you might be discovering, getting Ollama to play nice with AMD graphics cards isn't always as straightforward as with their Nvidia counterparts.

Here's the thing: it's TOTALLY doable. You just need to know a few tricks of the trade. I've spent a good amount of time tinkering with this stuff, and I'm here to walk you through it all, step-by-step. We'll cover everything from the official way to the "let's get this working even if my card isn't on the fancy list" way.

Honestly, once you get it set up, you'll be amazed at the power you have sitting right there in your computer. Let's get to it.

First off, let's talk about why this can be a bit of a process. For Nvidia GPUs, there's CUDA, which is the super-established platform for GPU computing. For AMD, the equivalent is ROCm (Radeon Open Compute platform). ROCm is what lets software like Ollama tap into the massive parallel processing power of your AMD graphics card.

The catch? ROCm officially supports a specific list of GPUs. If your card isn't on that list, Ollama might just throw its hands up & either not work or fall back to using your CPU, which is MUCH slower. But here's a little secret that the Linux community figured out a while ago: "not supported" doesn't always mean "won't work."

And that's where the magic happens.

This little environment variable is your golden ticket if you have an "unsupported" AMD GPU. Seriously, this is the key to unlocking GPU acceleration for a whole bunch of cards that aren't on the official list.

So, what does it do? In simple terms, it tells ROCm to essentially pretend your GPU is a different, supported model. It bypasses the hardware compatibility check & lets your GPU get to work. For a lot of cards, especially in the RX 6000 & 7000 series, this works like a charm.

The most common value you'll see people using is . This seems to be a pretty versatile setting that works for a lot of different GPUs. But how do you know what your GPU's "GFX version" is? You can usually find this information by searching online for your specific GPU model + "GFX version". A great resource for this is TechPowerUp's GPU database.

Now that you know the secret handshake, let's get into the nitty-gritty of setting this all up.

Linux, & particularly Ubuntu, is where AMD & ROCm shine the brightest. Here's how to get everything up and running.

First things first, you need to get the proper drivers installed. This is the foundation for everything else.