Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/BastinFlorian/RAG-on-GCP-with-VertexAI
Create a Chatbot app on your own data with GCP tools
https://github.com/BastinFlorian/RAG-on-GCP-with-VertexAI
cloudrun confluence gcp google-cloud-platform google-drive helpdesk llm rag vertexai
Last synced: 2 days ago
JSON representation
Create a Chatbot app on your own data with GCP tools
- Host: GitHub
- URL: https://github.com/BastinFlorian/RAG-on-GCP-with-VertexAI
- Owner: BastinFlorian
- License: apache-2.0
- Created: 2023-11-27T18:33:02.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2023-12-01T12:36:24.000Z (about 1 year ago)
- Last Synced: 2024-04-18T04:12:03.590Z (9 months ago)
- Topics: cloudrun, confluence, gcp, google-cloud-platform, google-drive, helpdesk, llm, rag, vertexai
- Language: Python
- Homepage:
- Size: 1.45 MB
- Stars: 17
- Watchers: 1
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome_ai_agents - Rag-On-Gcp-With-Vertexai - Create a Chatbot app on your own data with GCP tools (Building / Tools)
- awesome_ai_agents - Rag-On-Gcp-With-Vertexai - Create a Chatbot app on your own data with GCP tools (Building / Tools)
README
# Retrieval Augmented Generation with GCP Vertex AI Search
This repository contains the everything you need to deploy a smart and private question answering Chatbot from your Confluence data
Here is the demo of the app:
It uses the following architecture:
# How to use
* 1. [Prerequisites](#Prerequisites)
* 2. [Installation](#Installation)
* 2.1. [Bootstrap of the project](#Bootstrapoftheproject)
* 2.1.1. [Configure Terraform backend](#ConfigureTerraformbackend)
* 2.1.2. [Bootstrap the project](#Bootstraptheproject)
* 2.2. [Push the docker image to the Artefact Registry](#PushthedockerimagetotheArtefactRegistry)
* 2.2.1. [Configure service account impersonation](#Configureserviceaccountimpersonation)
* 2.2.2. [Add the secrets in GCP secret manager](#AddthesecretsinGCPsecretmanager)
* 2.3. [Deploy GCP ressources](#DeployGCPressources)
* 2.3.1. [Deploy GCP Ressources](#DeployGCPRessources)
* 2.4. [Add data to your a Vertex Index](#AdddatatoyouraVertexIndex)
* 2.4.1. [Add environment variables for a local data intialization](#Addenvironmentvariablesforalocaldataintialization)
* 2.4.2. [Deploy the Vertex Index Endpoint](#DeploytheVertexIndexEndpoint)
* 3. [Local development](#Localdevelopment)
* 3.1. [Prerequisites](#Prerequisites-1)
* 3.2. [Install dependencies](#Installdependencies)
* 3.3. [Run the app locally](#Runtheapplocally)
* 3.4. [Understand the Chatbot logic with the notebooks](#UnderstandtheChatbotlogicwiththenotebooks)
* 3.5. [Use docker and try the app on a container:](#Usedockerandtrytheapponacontainer:)
* 4. [Delete ressources](#Deleteressources)
* 4.1. [Delete the Vertex AI ressources](#DeletetheVertexAIressources)
* 4.2. [Delete the terraform ressources](#Deletetheterraformressources)
* 5. [Troubleshooting](#Troubleshooting)
* 6. [Roadmap](#Roadmap)
* 7. [References](#References)
- **Google Cloud CLI** : Install Google Cloud SDK following the instructions [here](https://cloud.google.com/sdk/docs/quickstart)
- **Terraform v>=1.5** : Follow the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli) to install the Terraform CLI locally
- a GCP project with enabled billing
In the following commands, you will have to replace these variables with your own values:
- `my-project-id`
- `my-tfstate-bucket` : useful to share the tf.state file between multiple users
- `my-user-email` : the email you used to create your GCP account
- `my-region` : the region where you want to deploy the GCP ressources (ex: `europe-west1`)
- `my-service-account` : the name of the service account you want to create (ex: `project-sa`)
- `my-registry-name` : the name of the artificat registry that will be created (ex: `rag-api`)
### 2.1. Bootstrap of the project
| **Prerequisite** : GCP Admin Account
This step consistes of creating Terraform Service Account with the required roles (we used to deploy GCP ressources) and enable GCP APIs.
If you are not owner of your project because you are using your company GCP account, you can ask your GCP admin to create a SA and give you the serviceAccountTokenCreator role.
#### 2.1.1. Configure Terraform backend
To configure the Terraform backend first you need to:
- Create the GCS bucket to store the tf.state file
The bucket name must be unique because the namespace is shared across all GCP projects.
```bash
gcloud init # Use your email and project settings
gcloud auth application-default login # Authenticate on GCP with admin account
gcloud storage buckets create gs:// --project= --location= # You need storage.buckets.create permission (ex: roles/editor)
```
Update GCS bucket name & prefix :
```bash
cd terraform/bootstrap
vim backend.tf
# Edit with the name of the bucket created above
vim terraform.tfvars
# Edit the values of project_id, location, artifactregistry_name and service_account_name
```
#### 2.1.2. Bootstrap the project
```bash
terraform init # You need storage.buckets.list permission (ex: roles/storage.objectUser)
terraform apply -var gcp_account_email=""
```
### 2.2. Push the docker image to the Artefact Registry
#### 2.2.1. Configure service account impersonation
```bash
export [email protected]
```
```bash
# Docker authentication
gcloud auth configure-docker -docker.pkg.dev
# Build and push the image:
cd ../../ # Go back to the root directory
docker buildx build --platform linux/amd64 --push -t -docker.pkg.dev///gen-ai:latest .
vim terraform/terraform.tfvars
# Change the image name in the terraform/terraform.tfvars file with the one you created
```
#### 2.2.2. Add the secrets in GCP secret manager
To secure the Confluence identifiers, you need to add them in [GCP secret manager](https://console.cloud.google.com/security/secret-manager).
You need to store 4 secrets:
- `CONFLUENCE_PRIVATE_API_KEY`: the private API key of your Confluence account. [Create a key here](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/)
- `CONFLUENCE_SPACE_NAMES`: the list of Confluence spaces names, useful for filters. You can find the space name in the URL of the space. For example, if the URL of your space is `https://my-company.atlassian.net/wiki/spaces/SPACE_NAME/overview`, the space name is `SPACE_NAME`.
- `CONFLUENCE_URL`
- `CONFLUENCE_EMAIL_ADRESS`
### 2.3. Deploy GCP ressources
#### 2.3.1. Deploy GCP Ressources
```bash
cd terraform
vim ./terraform.tfvars # Follow the instructions in the file to fill the variables
vim ./backend.tf # Update with the name of the bucket created above
terraform init
terraform apply # It will take approximatively 30 minutes to create the index
```
Now your Cloud Run service is deployed and you can access it with the URL displayed in the terraform output.
We have now to add data to the Vertex AI Search Index and deploy an endpoint to be able to query it.
Keep the two variables from the terraform output:
- `my-index-id` : the Vertex AI Search Index ID
- `my-index-endpoint-id` : the Vertex AI Search Index Endpoint ID
They will be used in the next step.
### 2.4. Add data to your a Vertex Index
The vector index is empty.
We need to add data to it and deploy an endpoint to be able to query it.
It will:
- collect urls from the Confluence Space you mentionned in the Confluence space names variable
- load each valid space url text and create chunks
- generate embeddings and save it to GCS
#### 2.4.1. Add environment variables for a local data intialization
The data initialization in GCP compute engine will be available soon
```bash
cd ../src
cp ~/.config/gcloud/application_default_credentials.json ./credentials.json
cp ../.env.template .env
vim .env # add the required variables
# Install poetry and dependencies before ?
poetry run python -m upload_data.data_init
# You may have to wait up to an hour for the index to be updated.
# You can check the GCP Vector Search console and check the number of vectors in the index to see when it is updated.
```
#### 2.4.2. Deploy the Vertex Index Endpoint
Once the index is not empty and contains our data, we can deploy an endpoint to be able to query it.
```bash
gcloud ai index-endpoints deploy-index \
--index= \
--deployed-index-id=ragdeployedindex \
--display-name=ragendpoint \
--region=
```
You can go now go to the Cloud Run console and try the chatbot.
#### 3.1. Prerequisites
- [Poetry](https://python-poetry.org/)
To work on a local environement, you first need to follow the preivous step or at least deploy a vector index and a deployed endpoint.
#### 3.2. Install dependencies
Create a virtual environment using poetry:
```bash
poetry install
```
You won't be able to use the secret manager in local, you have to copy the secrets in the `.env` file.
If not already done
```bash
cd src
cp ~/.config/gcloud/application_default_credentials.json ./credentials.json
cp ../.env.template .env
vim .env # add the required variables
```
#### 3.3. Run the app locally
```bash
# To run the app in local:
poetry run python -m streamlit run src/chatbot/app.py
# At initialization, add new data:
poetry run python src/upload_data/data_init.py
# To update the index with new/modified/deleted Confluence data
poetry run python src/upload_data/data_update.py
```
#### 3.4. Understand the Chatbot logic with the notebooks
```bash
make open-notebook
```
#### 3.5. Use docker and try the app on a container:
```bash
make demo
# Open http://localhost:8501/ in your browser
```
#### 4.1. Delete the Vertex AI ressources
```bash
gcloud ai index-endpoints undeploy-index \
--project= \
--deployed-index-id=ragdeployedindex \
--region=europe-west1 \
```
#### 4.2. Delete the terraform ressources
```bash
terraform destroy
```
> Cannot import Tensorflow
This can be caused in mac M1. Tensorflow requires to install tensorflow-metal with:
`poetry add tensorflow-metal`.
Be careful when using make build-docker after this command because tensorflow-metal cannot be installed in the docker image.
To remove this dependency if needed in local:
`poetry remove tensorflow-metal`
> ERROR: (gcloud.ai.index-endpoints.deploy-index) FAILED_PRECONDITION: Deploying empty Index is not allowed.
This error can be caused by an empty index. You need to add data to the index before deploying an endpoint.
If you have data in your storage, you have to update the index: `make update-index`.
If the index vector number is not 0 in the GCP console, try creating another index endpoint and deploy it with this new
endpoint.
- [ ] Switch to Hexagonale Architecture
- [ ] Design a Cloud Build CI/CD
- [ ] Initialize data on GCP Compute Engine
- [ ] Add the Google Drive loader with LangChain or [our custom implementation](https://github.com/pprados/langchain-googledrive)
- [ ] Improve Retriever with [our custom implementation](https://github.com/pprados/langchain-rag)
- [ ] Integrate the evaluation process directly in GCP
- [ ] Export the dashboard from LangSmith to monitor RAG process in GCP
[Article: Understand the RAG process with an example](https://blog.octo.com/le-chatbot-docto-langchain-rag-et-code-associe#:~:text=Le%20RAG%20est%20une%20technique,langages%20open%20ou%20closed%2Dsource)
[Code: GCP notebooks to understand the ingestion process and Vector Search usage](https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main/notebooks/official/matching_engine)
[Doc: GCP Vector Search documentation on ingestion data format](https://cloud.google.com/vertex-ai/docs/vector-search/setup/format-structure)
[Doc: Understand the Vector Search filtering process](https://cloud.google.com/vertex-ai/docs/vector-search/filtering)
[Doc: Connect a PubSub subscription to BigQuery with a schema to bypass Dataflow](https://cloud.google.com/pubsub/docs/bigquery)
[Vidéo: How to implement IAP in your Cloud Run service](https://www.youtube.com/watch?v=ayTGOuCaxuc)