Hey everyone, so you've dived into the super exciting world of running Large Language Models (LLMs) locally. It's a game-changer, right? No more relying on cloud APIs, you have total control. You've probably got a beast of a machine with a powerful GPU, all set to crunch those complex calculations. But here's a thought that might keep you up at night: is your local LLM actually using your expensive graphics card?

Honestly, it's a super common question & a valid concern. You've invested in the hardware, so you want to make sure you're getting every ounce of performance out of it. Turns out, just having a powerful GPU isn't enough; you need to make sure everything is set up correctly. Let's walk through how to check if your LLM is putting your GPU to work, & what to do if it's not.

Before we dive into the nitty-gritty of software & code, let's make sure you know what you're working with. It's like checking if you have all the ingredients before you start baking a cake.

The easiest way to check your GPU is to use the Task Manager. Just press , click on the "Performance" tab, & you should see your GPU listed there. It'll tell you the model name & how much dedicated GPU memory (VRAM) you have. This is a crucial number to keep in mind, as it'll determine the size of the models you can run.

Another way is to use the "System Information" tool. Press , type , & hit Enter. Go to "Components" > "Display" to see your GPU details.

On a Mac, you can go to the Apple menu > "About This Mac." The "Overview" tab will show you your graphics card.

If you're on Linux, you're probably comfortable with the terminal. You can use the command to get information about your hardware. To specifically look for your graphics card, you can use: