Setting Up Ollama with ServiceNow for IT Operations

What is Ollama?

Why Integrate Ollama with ServiceNow?

• Automation: Automate repetitive tasks using Ollama's intelligent responses, thus freeing up valuable time for IT staff to focus on strategic initiatives.
• Enhanced Decision-Making: Leverage the AI capabilities of Ollama to derive insights from large datasets, enabling quicker, data-driven decision-making.
• Scalability: With Ollama, you can seamlessly scale your operations as the AI can handle numerous requests simultaneously.
• Customization: Tailor the interaction experience by customizing the models to meet the specific requirements of your business.

Pre-Requisites for Setting Up Ollama with ServiceNow

• A ServiceNow Developer instance (you can sign up for one here).
• Administrative access to your ServiceNow instance.
• Ollama installed on a server or local machine where the ServiceNow midserver can communicate with it.

Step 1: Download & Install Ollama on Your System

1. Download Ollama: You can download the Ollama installer by visiting this link.
2. Install Ollama: Run the installer and follow the prompts to install the application on your local machine.
3. Verify Installation: Open your command prompt and run the command
   1 ollama -v
   to ensure that it has been installed correctly.

Step 2: Set Up the Ollama Midserver

1. Installation Instructions: Check the full list of instructions in the Ollama GitHub repository.
2. Configure Midserver: You’ll want to ensure your midserver settings allow communication with the Ollama setup.
3. Run Midserver: After setting it up, you can start the midserver and ensure it’s running properly before proceeding.

Step 3: Download Required LLM Models

1. List Available Models: To see what’s available, execute:
   1 ollama list
2. Download the Gemma Model: Use the command below to download the Gemma model:
   1 ollama run gemma:2b
3. Check Model Status: Ensure the model is downloaded correctly and is ready for use.

Step 4: Creating a ServiceNow REST API Action

1. Open ServiceNow Flow Designer: Navigate to the Flow Designer in your ServiceNow instance.
2. Create a New Action: Click on “Create new action” and name it (e.g.,
   1 Ollama REST API Connection
   ).
3. Input Configuration: Configure the action to use the midserver. To do this, make sure your midserver is running and selected in the settings.
4. Define Input Parameters: Define the input parameters that your action will take. For instance, this could include
   1 model
   ,
   1 prompt
   , etc.
5. API Call Setup: Here’s a simple JSON payload that you can test:
   1 2 3 4 5 { "model": "gemma:2b", "prompt": "Why is the sky blue?", "stream": false }
6. Test Your Action: Use the test features in Flow Designer to check if your action is working as expected!

Step 5: Managing Responses & Outputs

1. Log Responses: Create a mechanism to log the responses for auditing and analysis.
2. Utilize Outputs: Use the output data from Ollama for various operational needs such as updating incident records or feeding data into other platforms.

Step 6: Automating Future Interactions

• Create Business Rules: Set business rules based on the responses from your Ollama integration. For example, automatically categorize incoming tickets or assist in drafting responses to common inquiries.
• Monitor Performance: Regularly assess how the AI is performing against key metrics to ensure effectiveness.

Troubleshooting Common Issues

• Midserver Connection Problems: Make sure the midserver is operational and correctly configured to communicate with the Ollama API. Configure firewall rules if necessary.
• Model Performance: If the model isn’t responding or producing strange outputs, check the model logs for any errors that could provide insight into the problem.
• Resource Usage: Monitor your system resources. Running LLMs can be resource-intensive; therefore, ensure your system meets the minimum CPU and memory requirements listed on the Ollama GitHub documentation.

Conclusion

Final Thoughts