https://github.com/build5nines/azd-litellm
AZD Template for deploying LiteLLM running in Azure Container Apps
https://github.com/build5nines/azd-litellm
Last synced: about 2 months ago
JSON representation
AZD Template for deploying LiteLLM running in Azure Container Apps
- Host: GitHub
- URL: https://github.com/build5nines/azd-litellm
- Owner: Build5Nines
- License: mit
- Created: 2025-03-13T22:26:24.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-13T22:29:51.000Z (over 1 year ago)
- Last Synced: 2025-03-13T23:27:42.966Z (over 1 year ago)
- Size: 1.95 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# azd-litellm 
An `azd` template to deploy [LiteLLM](https://www.litellm.ai/) running in Azure Container Apps using an Azure PostgreSQL database.
To use this template, follow these steps using the [Azure Developer CLI](https://learn.microsoft.com/azure/developer/azure-developer-cli/overview):
1. Log in to Azure Developer CLI. This is only required once per-install.
```bash
azd auth login
```
2. Initialize this template using `azd init`:
```bash
azd init --template build5nines/azd-litellm
```
3. Use `azd up` to provision your Azure infrastructure and deploy the web application to Azure.
```bash
azd up
```
4. `azd up` will prompt you to enter these additional secret and password parameters used to configure LiteLLM:
- `databaseAdminPassword`: The Admin password use to connect to the PostgreSQL database.
- `litellm_master_key`: The LiteLLM Master Key. This is the LiteLLM proxy admin key.
- `litellm_salt_key`: The LiteLLM Salt Key. This cannot be changed once set, and is used to encrypt model keys in the database.
Be sure to save these secrets and passwords to keep them safe.
5. Once the template has finished provisioning all resources, and Azure Container Apps has completed deploying the LiteLLM container _(this can take a minute or so after `azd up` completes to finish)_, you can access both the Swagger UI and Admin UI for LiteLLM.
This can be done by navigating to the `litellm` service **Endpoint** returned from the `azd` deployment step using your web browser. _You can also find this endpoint by navigating to the **Container App** within the **Azure Portal** then locating the **Application Url**._
Navigating to the Endpoint URL will access Swagger UI:
Navigating to `/ui` on the Endpoint URL will access the LiteLLM Admin UI where Models and other things can be configured:
## Architecture Diagram
In addition to deploying Azure Resources to host LiteLLM, this template includes a `DOCKERFILE` that builds a LiteLLM docker container that builds the LiteLLM proxy server with Admin UI using Python from the `litellm` pip package.
## Azure Resources
These are the Azure resources that are deployed with this template:
- **Container Apps Environment** - The environment for hosting the Container App
- **Container App** - The hosting for the [LiteLLM](https://www.litellm.ai) Docker Container
- **Azure Database for PostgreSQL flexible server** - The PostgreSQL server to host the LiteLLM database
- **Log Analytics** and **Application Insights** - Logging for the Container Apps Environment
- **Container Registry** - Used to deploy the custom Docker container for LiteLLM
## How to use Specific Version of LiteLLM
By default, this project uses the latest version of LiteLLM. There may be reasons you want to run a specific version of LiteLLM. This project deploys LiteLLM using its Python package, and you can update the version referenced to target a specific version of LiteLLM if necessary.
To do so, you can edit the [`/src/litellm/requirements.txt`](/src/litellm/requirements.txt) file. This file is the Python `pip` requirements file that specifies the PIP packages and their versions to use. The `litellm[proxy]` package is the package that is LiteLLM. You can find the available release versions on the [`litellm` PIP package page.](https://pypi.org/project/litellm/)
Here's an example of the `requirements.txt` file reference of `litellm` package specifying the v1.65.1 release:
```text
litellm[proxy]==1.65.1
```
By default, this project does not specify a version; which tells PIP to pull down the latest release. I hope this helps if you find yourself needing to run a specific version of LiteLLM.
## Author
This `azd` template was written by [Chris Pietschmann](https://pietschsoft.com), founder of [Build5Nines](https://build5nines.com), Microsoft MVP, HashiCorp Ambassador, and Microsoft Certified Trainer (MCT).