8/27/2024

Using Ollama in an ExpressJS App

In the fast-paced world of technology, incorporating Artificial Intelligence (AI) into your applications can enhance user experience significantly. One such fantastic tool is Ollama, a framework allowing developers to run large language models like Llama and Mistral conveniently on their machines. Today, we’ll explore the ins and outs of using Ollama in an ExpressJS application, giving you powerful AI capabilities at your fingertips.

What is Ollama?

Ollama is an innovative framework designed to simplify the process of running complex language models locally. It supports a wide range of models, allowing developers to build applications that leverage natural language understanding capabilities without being reliant on external APIs. Have you ever wanted to enhance your chatbot, create content generators, or develop AI-driven tools? With Ollama, those possibilities are at your fingertips!

Why Use ExpressJS?

ExpressJS is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications. The key reasons for utilizing ExpressJS include:

Simplicity: Maintaining and developing your server application becomes a breeze.
Modular: Its design allows for a clean separation of concerns and easy integration of different middleware.
Community: A huge community means a plethora of shared resources and plugins available to ease development.

Setting Up Your Environment

Before diving into the code, ensure you have all prerequisites installed:

Node.js (version 14.x or higher)
npm (Node Package Manager)
Docker (for managing the Ollama container)

1. Install Ollama

First things first, let’s get Ollama installed! You can set it up easily by running the following command in your terminal:

1
curl -fsSL https://ollama.com/install.sh | sh

This command pulls the latest version of Ollama and installs it on your machine. Be prepared, as it might take a little while!

2. Pull a Model

Once Ollama is installed, you need to pull a model. For this purpose, we'll use

Llama 3.1

, a well-known and highly capable model.

1
ollama pull llama3.1

3. Create Your ExpressJS Application

Now that you have Ollama ready, it’s time to create your ExpressJS application. Start a new project by creating a directory, navigating to it, and initializing it with npm:

1
2
3
mkdir my-ollama-app
cd my-ollama-app
npm init -y

Now, install Express and any other required dependencies:

1
npm install express body-parser dotenv

4. Building Your Express Server

Next, let’s create a basic server setup. In your project root, create a file called app.js:

1
2
3
4
5
6
7
8
9
10
11
const express = require('express');
const bodyParser = require('body-parser');

const app = express();
const PORT = process.env.PORT || 3000;

app.use(bodyParser.json());

app.listen(PORT, () => {
    console.log(`Server is running on port ${PORT}`);
});

Now you're ready to run your server!

1
node app.js

Open up your browser and hit

http://localhost:3000

, you should see a message in the terminal confirming your server is up!

5. Integrating Ollama into Your Express App

Here comes the exciting part—integrating Ollama to handle queries in our ExpressJS application! We’ll set up an API endpoint that allows you to send messages, which will be processed by the Ollama model.

In your app.js, add the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
const fetch = require('node-fetch'); // Install this package
// Add the following route to handle chat
app.post('/api/chat', async (req, res) => {
    try {
        const { prompt } = req.body;

        const response = await fetch('http://localhost:11434/api/generate', {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
            },
            body: JSON.stringify({ model: 'llama3.1', prompt }),
        });

        const data = await response.json();

        return res.status(200).json(data);
    } catch (error) {
        console.error(error);
        return res.status(500).json({ error: 'Something went wrong!' });
    }
});

This code sets up a POST endpoint

/api/chat

that will send a prompt to the Ollama model you've pulled earlier. It accepts a JSON request body with the prompt and responds with the model's output.

6. Testing Your Integration

Now that everything's set up, it's time to test this. You can use a tool like Postman or curl to send a request to your Express API:

1
curl -X POST http://localhost:3000/api/chat -H "Content-Type: application/json" -d '{"prompt": "What is the capital of France?"}'

The output should give you a response from the Llama 3.1 model! If you run into any issues with response formatting, check the console logs for debugging.

7. Best Practices for Security

Handling requests and responses between your client and server carries certain risks. Ensure your ExpressJS application is secure by following these practices:

Input Validation: Always validate and sanitize user inputs to prevent attacks like SQL Injection or XSS.
Rate Limiting: Use packages such as express-rate-limit to reduce the risk of abuse.
CORS Headers: Set proper CORS headers in your Express setup to manage which domains can access your API.
Environment Variables: Use a
1.env
file to store sensitive configuration variables instead of hard-coding them into your source code.

8. Deploying Your Application

Once your application is ready and tested locally, consider hosting it on a cloud service. Here are a couple of options:

Heroku: Deploy your application with ease using Heroku. It provides ample documentation and built-in security features.
Docker: Since Ollama can run inside a Docker container, consider containerizing your Express app and deploying it on a platform like AWS or Google Cloud.

Conclusion

Integrating Ollama with ExpressJS opens up a world of possibilities for your applications, leveraging powerful AI-driven capabilities to engage users and enhance their experience. Remember, with great power comes great responsibility; make sure to follow best practices for security and always test thoroughly.

If you're looking to take things a step further, consider using Arsturn to instantly create custom ChatGPT chatbots for your website. With Arsturn’s user-friendly interface, you can boost engagement and grow conversions with AI-driven conversations tailored to your audience's needs. Unlock the potential of conversational AI today!

With that, get started integrating Ollama in your projects & let AI elevate your development experience!