Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Azure-Samples/azure-openai-terraform-deployment-sample

Last synced: 14 days ago
JSON representation

Host: GitHub
URL: https://github.com/Azure-Samples/azure-openai-terraform-deployment-sample
Owner: Azure-Samples
License: mit
Created: 2023-05-16T11:53:04.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2024-09-13T09:20:15.000Z (2 months ago)
Last Synced: 2024-09-13T22:59:17.916Z (about 2 months ago)
Language: HCL
Homepage:
Size: 217 KB
Stars: 45
Watchers: 16
Forks: 22
Open Issues: 4
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

# Azure OpenAI Terraform deployment for sample chatbot

This sample application deploys an AI-powered document search using Azure OpenAI Service, Azure Kubernetes Service (AKS), and a Python application leveraging the [Llama index](https://gpt-index.readthedocs.io/en/latest/) ans [Streamlit](https://docs.streamlit.io/library/get-started). The application will be deployed within a virtual network to ensure security and isolation. Users will be able to upload documents and ask questions based on the content of the uploaded documents.

![diagram](./images/diagram.png)

## Prerequisites

- Azure subscription. If you don't have an Azure subscription, [create a free account](https://azure.microsoft.com/free/?ref=microsoft.com&utm_source=microsoft.com&utm_medium=docs&utm_campaign=visualstudio) before you begin.
- Subscription access to Azure OpenAI service. Request Access to Azure OpenAI Service [here](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUOFA5Qk1UWDRBMjg0WFhPMkIzTzhKQ1dWNyQlQCN0PWcu).
- [Terraform](https://learn.microsoft.com/azure/developer/terraform/quickstart-configure).

The easiest way to run this sample is to run it creating a new [GitHub Codespace](https://codespaces.new/Azure-Samples/azure-openai-terraform-deployment-sample)

[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/Azure-Samples/azure-openai-terraform-deployment-sample)

## Quickstart

### (Optional) configure GitHub Codespaces secrets to access your Azure subscription

- Run the following command to create a service principal with the "Owner" role for a specific subscription, and outputs its information in JSON format.

```bash
az ad sp create-for-rbac --role="Owner" --scopes="/subscriptions/" -o json
```

- In your github account go to Codespaces and Create a new Codespace with "Azure-Sample/azure-openai-terraform-deployment-sample" repository and select the main branch.

![codespace_create](./images/codespace-create.png)

- In your github account, go to Settings. On the left pane, select Codespaces tab and create a secret for `ARM_CLIENT_ID`, `ARM_CLIENT_SECRET`, `ARM_SUBSCRIPTION_ID` and `ARM_TENANT_ID` values, as shown in the image below. For each secret, on the Repository access section, click on the "Select repositories" dropdown menu and select "Azure-Samples/azure-openai-terraform-deployment-sample".

![codespace_secrets](./images/codespace_secrets.png)

- Follow this link to create a new [GitHub Codespace](https://codespaces.new/Azure-Samples/azure-openai-terraform-deployment-sample).

[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/Azure-Samples/azure-openai-terraform-deployment-sample)

### Run the Terraform

- Clone or fork this repository. (Skip if using GitHub codespaces)
```
git clone https://github.com/Azure-Samples/azure-openai-terraform-deployment-sample/
cd azure-openai-terraform-deployment-sample
```

- Go to the `infra` folder and run the following command to initialize your working directory.

```bash
cd infra
terraform init
```

- Run terraform apply to deploy all the necessary resources on Azure.

```bash
terraform apply
```

- Run the following command. This script retrieves the AKS cluster credentials, logs in to the ACR, builds and pushes a Docker image, creates a federated identity, and deploys resources to the Kubernetes cluster using a YAML manifest.

```bash
terraform output -raw installation-script | bash
```

- Get the external ip address of the service by running the command bellow.

```bash
kubectl get services -n chatbot
```

- Copy the external ip address and paste it in your browser. The application should load in a few seconds.

![app](/images/application.png)

- You can test the application by uploading a sample `file.txt` with the following content:

```
Saverio and Sofia both work at Microsoft.
They live in Switzerland.
Saverio is Italian.
Sofia is from Portugal.
```

And then ask a question like the following screenshot:

![app](/images/Who_is_Sofia.png)

- Access the Grafana dashboard by running the following command.

```bash
kubectl port-forward svc/grafana 8080:80
```
And point your browser to `http://localhost:8080`.

You can browse to the metrics explorer and see the traces collected by the application. In the traces you are able to inspect the actual prompt that the llama_index library is using to query the OpenAI service.

![app](/images/grafana_tempo.png)

## Resources

- [Azure OpenAI](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/overview)
- [Azure OpenAI Terraform verified module](https://registry.terraform.io/modules/Azure/openai/azurerm/latest).