Build AI-powered applications using OpenLLM and Vultr Cloud GPU

HTML: Markup language
CSS: Styling language
JavaScript: Scripting language
Web APIs: Programming interfaces
Learn web development
Get to know MDN better
OpenLLM is an open-source platform that enables you to create AI-powered production applications such as personalized chat-bots, recommendation systems, and more. It allows you to generate responses by sending an API prompt, along with parameters, to manipulate the response. OpenLLM's library includes all major models such as Mistral, Falcon, and Llama.
In this article, we'll demonstrate the steps to deploy the Falcon 7B model with OpenLLM on a Vultr GPU server to generate API responses, which can be used for creating AI-powered applications. You'll learn how to install the required dependencies and how to create an OpenLLM service with persistence. Additionally, we'll cover setting up Nginx as a reverse proxy for efficient load balancing and securing your application with a Secure Sockets Layer (SSL) certificate to enable HTTPS.
To efficiently deploy Artificial Intelligence (AI) or Machine Learning (ML)-powered applications, using cloud GPUs is one of the most efficient ways. Cloud GPUs provide access to the latest technology, enabling you to continuously build, deploy, and globally serve large-scale applications.
Sign up and log in to the Vultr Customer Portal.
Navigate to the Products page.
From the side menu, select Compute.
Click the Deploy Server button in the center.
Select Cloud GPU as the server type.
Select A40 as the GPU type.
In the "Server Location" section, select the region of your choice.
In the "Operating System" section, select Vultr GPU Stack as the operating system.
Vultr GPU Stack is designed to streamline the process of building Artificial Intelligence (AI) and Machine Learning (ML) projects by providing a comprehensive suite of pre-installed software, including NVIDIA CUDA Toolkit, NVIDIA cuDNN, Tensorflow, PyTorch, and so on.
In the "Server Size" section, select the 48 GB option.
Select any more features as required in the "Additional Features" section.
Click the Deploy Now button on the bottom right corner.
After setting up a Vultr server as described earlier, this section will guide you through installing the dependency Python packages necessary for running OpenLLM and verifying the installation.
Install the required packages.
Here's what each package represents:
Verify the installation.
If the installation is successful, the system will find and execute openllm, displaying its help information. This indicates that openllm is correctly installed and recognized by the system. If openllm is not installed properly, the command will likely return an error.
In this section, you'll learn how to create an OpenLLM service that starts the service automatically when the system boots up and runs the Falcon 7B model for inference.
Get the openllm path.
Copy and paste the path into the clipboard. You'll use this in Step 4.
Create an OpenLLM service file.
Paste the following content into the service file. Make sure to replace User and Group values with your actual values. Also replace WorkingDirectory with the OpenLLM path (without openllm) you copied earlier and Execstart with the actual OpenLLM path including the executable binary.
Verify the status of the service.
The output will look like this:
Enable the service to start automatically whenever the system boots up.
Nginx acts as a reverse proxy between your web server and clients. It directs incoming requests based on your request configuration settings.
In this section, you'll learn how to configure your OpenLLM application for reverse proxy for efficient request handling and load balancing using Nginx. You'll also learn how to configure OpenLLM itself for reverse proxy functionality.
Log in to the Vultr Customer Portal.
Navigate to the Products page.
From the side menu, expand the Network drop down, and select DNS.
Click the Add Domain button in the center.