8/27/2024

Deploying Ollama on Digital Ocean: A Comprehensive Guide

Are you ready to dive into the world of Large Language Models (LLMs)? If you're looking to host Ollama, a powerful API for creating and running LLMs, right on Digital Ocean, you're in for a treat! In this blog post, I'll walk you through the entire process step-by-step, ensuring you can launch your own Ollama instance with ease.

What is Ollama?

Ollama is a framework that allows you to build and run language models efficiently on your local machine or server. It provides a simple API for managing these models, with various pre-built options available. The beauty of using Ollama lies in its ability to enable developers to have total control over their models while minimizing dependency on external APIs, ultimately leading to reduced costs.

Why Use Digital Ocean?

When it comes to deploying Ollama, Digital Ocean stands out as a fantastic choice. With its user-friendly interface, robust API, and competitive pricing, Digital Ocean is an excellent choice for both startups and established companies. The ability to spin up droplets (virtual private servers) allows you to tailor your resources based on the needs of your application. Plus, their global data center locations can help optimize latency.

Key Benefits of Digital Ocean for Deploying Ollama

Affordable Pricing: Digital Ocean offers a free trial to new users which provides a $200 credit for the first two months, making it easy to experiment without breaking the bank. You can learn more about their pricing here.
Simple Setup: The Digital Ocean dashboard is intuitive, making the creation and management of droplets a breeze.
Scalability: Whether you start small or need to ramp up, Digital Ocean allows you to easily upgrade your droplet with higher resources.

Prerequisites Before You Start

To get all set to deploy Ollama on Digital Ocean, there are a few prerequisites you need:

A Digital Ocean Account. If you don’t have one yet, create a free account here.
Funding Account: You can take advantage of their Free Trial which gives new users a $200 credit, valid for 2 months. Students can also access additional benefits through the GitHub Student Pack.
Basic knowledge of Linux command line operations will be helpful.

Step 1: Create Your Linux Droplet

Let's get started by creating your own Linux droplet:

Go to the Dashboard: Once your account is set up, log into your Digital Ocean account and head over to the control panel.
Create a Droplet: Click the big green
1Create
button and select Droplets.

Step 1.1: Picking Droplet’s Prime Location

Choosing a data center location closer to your target audience is crucial for reducing latency. For example, if you’re in India, selecting the Bangalore location can optimize response times.

Step 1.2: Choose the Operating System

You need an operating system (OS) that aligns with your project requirements. Here, I’d recommend Ubuntu as it's widely used within the developer community and supports most applications. However, you are free to choose other Linux flavors based on your preferences.

Step 1.3: Allocate Power for Your Droplet

Ollama’s heavier models like Llama-2 usually require at least 8GB of RAM. For our initial setup, we can stick to a basic droplet, but don't worry; you can always upgrade later.

Step 1.4: Setting Up Secure Access

Set a strong password for your droplet. Also, consider using SSH keys for enhanced security, if you're comfortable with the process. This setup is recommended for long-term security purposes.

Step 1.5: Finalize Your Choices

Give your droplet a name, for example, “ollama-server.” Ensure you've double-checked all selected options before hitting the Create Droplet button. It might take a few seconds for Digital Ocean to create your droplet.

Step 2: Connecting to Your Droplet

Once your droplet is created, it’s time to connect to it!

The connection process depends on your operating system:
- Windows: Use Windows Terminal or PowerShell.
- macOS/Linux: Use your built-in terminal.

Step 2.1: Finding Your Droplet’s IP Address

In the Digital Ocean dashboard, locate your newly created droplet.
Copy the IP address provided.

Open the terminal on your system.
Initiate the connection using the following command:
1 2shell ssh root@your_droplet_ip_address
Replace
1your_droplet_ip_address
with your actual droplet's IP.
Type
1yes
to trust the host's fingerprint.
Enter the password you set during the droplet creation process. Upon successful completion, you’ll see the command prompt of your droplet, ready for use!

Step 3: Set Up Your Environment

After logging in, first things first, update your package list:

1
2

shell
sudo apt update && sudo apt upgrade

Step 3.1: Installing Ollama

The next step is to install Ollama, which provides an easy installation process. You can do this with one simple command in your terminal:

1
2

shell
curl -fsSL https://ollama.com/install.sh | sh

This command downloads and runs the installation script for Ollama.

Step 3.2: Running Your First Model

Once installed, you can run your first model! I recommend trying the uncensored version if you’re feeling adventurous. To do this:

1
2

shell
ollama run llama2-uncensored

When you run this command, Ollama will await your inputs.

Step 3.3: Interaction

Start chatting with Ollama! For fun, you can ask questions like: > “Why is the sky blue?” Ollama will respond based on its model. This is where the fun begins!

Save Costs with Arsturn

Now that your LLM is up and running, why not take your AI game a step further? Explore how you can create custom AI chatbots using Arsturn. Arsturn allows you to instantly build chatbots for your websites, enhancing engagement and seamlessly connecting with your audience.

User-friendly Design: No coding experience? No problem! Create your chatbot effortlessly.
Customizable AI: Use your own branding to make it uniquely yours.
Insightful Analytics: Gain valuable insights into your audience and improve satisfaction.

Claim a free chatbot for your domain on Arsturn today—no credit card required!

Conclusion

Deploying Ollama on Digital Ocean is a fantastic way to harness the power of LLMs in a cost-effective manner. With the right setup and configuration, you can enjoy the flexibility of running your own AI without the headache of hefty API charges. Plus, with Arsturn, you can take your AI to even greater heights by engaging your audience in innovative ways. Happy coding!

Now you have everything needed to set up your Ollama LLM on Digital Ocean effortlessly! Enjoy experimenting and developing amazing projects. Happy Deploying!