Setting Up Ollama with Google Cloud Functions

Why Use Ollama?

Advantages of Google Cloud Functions

Key Benefits of Cloud Functions:

• Cost Efficiency: Only pay for the resources you use.
• Automatic Scaling: Grows seamlessly from zero to many instances as needed.
• Event-Driven Architecture: Easily triggered by events in Google Cloud services.

Prerequisites

1. A Google Cloud account (use the free trial to get started).
2. Installed the Google Cloud SDK on your local machine.
3. Familiarity with basic command-line operations.
4. A simple understanding of Docker can help but isn't strictly necessary.

Step 1: Set Up Your Google Cloud Environment

1. Create a New Project: First, log into your Google Cloud Console & create a new project through the project selector. Make sure billing is enabled for the project.
2. Enable Required APIs: Navigate to APIs & Services and enable:
   • Google Cloud Functions API
   • Cloud Storage API
   • Cloud Pub/Sub API
   • Artifact Registry API
   • Cloud Build API
   You can enable these APIs using the command line as well:

   1 2 3 4 5 6 bash gcloud services enable cloudfunctions.googleapis.com \ artifactregistry.googleapis.com \ cloudbuild.googleapis.com \ storage.googleapis.com \ pubsub.googleapis.com
3. Install the gcloud CLI: If you haven’t done so, install the gcloud CLI tools.
   • Run
     1 gcloud init
     to authenticate your account & set your project.

   Step 2: Prepare Your Ollama Model

4. Docker Setup: Before deploying with Cloud Functions, create the necessary

   1 Dockerfile

   . It allows you to define how your model runs locally and provides a container for deployment. Here’s a sample

   1 Dockerfile

   :

   1 2 3 4 5 6 7 8 dockerfile FROM ollama/ollama:0.3.6 ENV OLLAMA_HOST 0.0.0.0:8080 ENV OLLAMA_MODELS /models ENV OLLAMA_DEBUG false ENV MODEL gemma2:9b RUN ollama serve & sleep 5 && ollama pull $MODEL ENTRYPOINT ["ollama", "serve"]

   This Dockerfile sets up the Ollama environment for the chosen model.
5. Building Your Docker Image: Once your Dockerfile is ready, you need to build your image. Run:

   1 2 bash docker build -t ollama-gemma .

Step 3: Push Your Docker Image to Artifact Registry

1. Create a Repository: Create a Docker repository for your project:

   1 2 bash gcloud artifacts repositories create ollama-repo --repository-format=docker --location=us-central1
2. Authenticate Docker: Run the following command to authenticate your Docker CLI to interact with Google Cloud services:

   1 2 bash gcloud auth configure-docker
3. Tag & Push Your Docker Image: Tag your image and push it to your new repository:

   1 2 3 bash docker tag ollama-gemma us-central1-docker.pkg.dev/YOUR_PROJECT_ID/ollama-repo/ollama-gemma docker push us-central1-docker.pkg.dev/YOUR_PROJECT_ID/ollama-repo/ollama-gemma

Step 4: Setting Up Cloud Functions for Ollama

1. Deploying the Function: With your Docker image pushed, you can now deploy it on Cloud Functions. Use the following command to deploy:
   1 2 3 4 5 bash gcloud functions deploy ollama-gemma \ --image=us-central1-docker.pkg.dev/YOUR_PROJECT_ID/ollama-repo/ollama-gemma \ --trigger-http \ --allow-unauthenticated
   This command creates a new HTTP-triggered function with your Ollama model.

Step 5: Testing Your Setup

1. Get the Function URL: After the deployment, you’ll see an output that includes your function’s URL link.
2. Testing Using cURL: Open your terminal & run:
   1 2 bash curl -X POST YOUR_FUNCTION_URL -H "Content-Type: application/json" -d '{"model":"gemma2:9b", "prompt":"Why is the sky blue?"}'
   If everything is set up correctly, you should receive a response from the Ollama model with the answer to that prompt.

Step 6: Monitor and Maintain Your App

Conclusion

FAQs

• What LLMs can I use with Ollama? You can explore a variety of models including Llama 3, Mistral, and Phi 3.
• Can I customize my model? Absolutely! Ollama lets you create and modify your models to fit your brand's unique needs.
• Do I need any special permissions to set up? Yes, be sure to set up the necessary IAM roles as outlined in the tutorial for a secure deployment.