8/26/2024

Step-by-Step Guide to Installing Ollama on Linux

Welcome to your complete guide for installing Ollama on Linux! If you’ve been dreaming of running large language models locally, you're in the right place. Let's jump right into it and get you set up in no time.

What is Ollama?

Ollama is a powerful tool that allows you to run large language models like Llama 3.1, Phi 3, Mistral, and Gemma 2 right on your local machine without the hassle of complex configuration or heavy server costs. It’s designed to make running AI models efficient & straightforward, whether you’re a developer, an AI enthusiast, or just someone curious about language models.

Why Install Ollama Locally?

When you install Ollama locally, it offers several benefits:

Privacy: Your data remains on your machine without being sent to external servers.
Speed: No latency from remote API calls.
Control: You have complete control over the models & how they run.
Customization: Tailor settings & models according to your needs.

Pre-Installation Requirements

Before diving into the installation, you'll want to ensure a few things are in order:

You have a Linux distribution (like Ubuntu) installed.
Your system has at least 8 GB of RAM for running smaller models.
If you are planning to run larger models like Llama 3.1 or Phi 3, ensure you have 16 GB or more of RAM.
It’s also highly recommended to have either an NVIDIA GPU for CUDA support or AMD GPU for ROCm support to utilize the full potential of Ollama.

Step 1: Update Your System

Keeping your system updated is always a good first step. Open your terminal & run:

1
2

bash
sudo apt-get update && sudo apt-get upgrade

Step 2: Verify Your GPU Support

If you're using an NVIDIA GPU, you can verify it's working properly by typing:

1
2

bash
nvidia-smi

You should see information about your GPU. If you’re using an AMD GPU, ensure ROCm drivers are installed. Check for support via:

1
2

bash
rocm-smi

Step 3: Install Required Tools

To install Ollama, you need to have

curl

installed. You can install it by executing:

1
2

bash
sudo apt-get install curl

After that, let's ensure that any necessary libraries for your GPU are also set up:

For AMD GPUs, make sure to check the AMD ROCm documentation for the best installation practices.
For NVIDIA GPUs, you need the CUDA toolkit. You can find installation instructions here.

Step 4: Install Ollama

The installation of Ollama can be performed with a simple command. In your terminal, run:

1
2

bash
curl -fsSL https://ollama.com/install.sh | sh

This will download & install Ollama on your system.

If you prefer a manual installation, you can download the Linux package with:

1
2

bash
curl -fsSL https://ollama.com/download/ollama-linux-amd64.tgz | sudo tar zx -C /usr

Step 5: Add Ollama as a Startup Service (Recommended)

Setting Ollama as a service allows it to run in the background. Here’s how to do that:

Create a user for Ollama:
1 2bash sudo useradd -r -s /bin/false -m -d /usr/share/ollama ollama
Create a service file: Create a new file called
1ollama.service
in `/etc/systemd/system/
1 2bash sudo nano /etc/systemd/system/ollama.service
Insert the following content: ```ini [Unit] Description=Ollama Service After=network-online.target
[Service] ExecStart=/usr/bin/ollama serve User=ollama Group=ollama Restart=always RestartSec=3
[Install] WantedBy=default.target ```
Start the service:
1 2 3 4bash sudo systemctl daemon-reload sudo systemctl enable ollama sudo systemctl start ollama

Step 6: Installing GPU Drivers (Optional)

For NVIDIA GPUs:

If you want to optimize performance for an NVIDIA GPU, install the CUDA drivers:

Download CUDA from NVIDIAs Developer site.
Verify installation with:
1 2bash nvidia-smi

For AMD GPUs:

Ensure ROCm is installed by following the official ROCm installation guide.

Step 7: Running Ollama

Now, let’s check if everything works correctly. Open a terminal and run:

1
2

bash
ollama serve

This command runs the Ollama server & you should see the server logs in your terminal window.

Step 8: Pulling a Model

To experiment with Ollama, you can pull one of its supported models easily. For instance, to pull the Llama 3 model, type:

1
2

bash
ollama pull llama3

This downloads the model from their repository. You can also find available models on their model library.

Step 9: Using Your Model

To run the downloaded model, simply execute:

1
2

bash
ollama run llama3

You can also interact with the model in the terminal. If you find it useful, you might consider setting up an easier interface for chatting.

Step 10: Viewing Logs & Troubleshooting

If you encounter any issues, you can review the logs. For systemd, this can be done with:

1
2

bash
journalctl -u ollama --no-pager

Additionally, verify that the Ollama service is running:

1
2

bash
sudo systemctl status ollama

If you see any errors, make sure your setup meets the installation requirements.

Step 11: Uninstalling Ollama (If Needed)

Should you wish to remove Ollama, it's quite simple. First, stop the service:

1
2
3

bash
sudo systemctl stop ollama
sudo systemctl disable ollama

Then remove the service file:

1
2

bash
sudo rm /etc/systemd/system/ollama.service

And finally, remove the Ollama binary:

1
2

bash
sudo rm -r /usr/bin/ollama

You’ll also want to delete the models by running:

1
2

bash
sudo rm -r /usr/share/ollama/.ollama/models

Conclusion

You did it! You’ve successfully installed Ollama on your Linux machine, enabling the ability to run sophisticated language models locally. Remember, this setup empowers you to control your models, ensuring privacy while gaining high-speed access.

Discover Arsturn

As you explore the world of language models like Ollama, don’t forget about Arsturn, which allows you to instantly create custom ChatGPT chatbots for your website. Boost engagement & streamline customer interactions effortlessly. There’s no coding skills required, making it a breeze for any business to enhance user experience.

By taking advantage of powerful AI solutions like Ollama and Arsturn, you can unlock endless possibilities in your applications while saving time & resources.

Happy coding, and may your AI journey be insightful & productive!