{"id":24972975,"url":"https://github.com/weaviate-tutorials/scalable-ai-workshop","last_synced_at":"2025-08-28T17:34:38.724Z","repository":{"id":262327343,"uuid":"870648500","full_name":"weaviate-tutorials/scalable-ai-workshop","owner":"weaviate-tutorials","description":null,"archived":false,"fork":false,"pushed_at":"2025-04-03T12:54:38.000Z","size":9993,"stargazers_count":4,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T13:38:48.520Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/weaviate-tutorials.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-10T12:23:24.000Z","updated_at":"2025-04-03T12:54:41.000Z","dependencies_parsed_at":"2024-12-17T14:25:37.496Z","dependency_job_id":"0f384207-3876-4480-9e2e-d344d1e7538b","html_url":"https://github.com/weaviate-tutorials/scalable-ai-workshop","commit_stats":null,"previous_names":["weaviate-tutorials/scalable-ai-workshop"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2Fscalable-ai-workshop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2Fscalable-ai-workshop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2Fscalable-ai-workshop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2Fscalable-ai-workshop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/weaviate-tutorials","download_url":"https://codeload.github.com/weaviate-tutorials/scalable-ai-workshop/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248359596,"owners_count":21090554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-03T18:11:01.967Z","updated_at":"2025-08-28T17:34:38.711Z","avatar_url":"https://github.com/weaviate-tutorials.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scalable AI applications with Weaviate\n\n\u003e [!NOTE]\n\u003e Please see the official [Weaviate documentation](https://weaviate.io/developers/weaviate) for the latest information on Weaviate.\n\n![App screenshot](assets/app.png)\n\n# Step 1: Preparation \u0026 Setup\n\nClone this repo and navigate into it. This will be your working directory for the workshop.\n\n```shell\ngit clone git@github.com:weaviate-tutorials/scalable-ai-workshop.git\ncd scalable-ai-workshop\n```\n\n## 1.1 Install Python \u0026 set up a virtual environment\n\n\u003e [!NOTE]\n\u003e If you have a preferred setup (e.g. Conda/Poetry), please feel free to use that. Otherwise, we recommend following these steps.\n\nInstall `Python 3.9` or higher (e.g. from the [Python website](https://python.org/), or `pyenv`).\n\nThen, create \u0026 activate a virtual environment:\n\n```shell\npython -m venv .venv  # Or use `python3` if `python` is Python 2\nsource .venv/bin/activate\n```\n\nCheck that you are using the correct Python:\n\n```shell\nwhich python\n```\n\nThis should point to the Python binary in the `.venv` directory.\n\nInstall the required Python packages:\n\n```shell\npip install -r requirements.txt\n```\n\n\u003e [!TIP]\n\u003e If you have network connectivity issues, the installation may time out part way through. If this happens, just try running the command again. It will re-used cached data, so you will make further\n\n\u003e [!TIP]\n\u003e If you open new terminal or shell window, you will need to activate the virtual environment again. Navigate to the project directory and run `source .venv/bin/activate`.\n\n## 1.3 Install containerization tools\n\n### 1.3.1 Docker (Required)\n\nInstall Docker Desktop: https://www.docker.com/products/docker-desktop/\nDocker will also be used as a container runtime for Minikube.\n\n### 1.3.2 Minikube \u0026 Helm (Optional, only if you want to use Kubernetes)\n\nFor running Weaviate in a Kubernetes cluster, you can use Minikube \u0026 Helm.\n\nMinikube (https://minikube.sigs.k8s.io/docs/start/)\nHelm (https://helm.sh/docs/intro/install/)\n\nYou will get the most out of the workshop if you have Minikube \u0026 Helm installed.\n\nNow, you are ready to start running Weaviate!\n\n\u003e [!NOTE]\n\u003e If you are waiting, you can check out the introductory materials. Take a look at the slide deck PDF, or the `intro_workshop.ipynb` notebook.\n\n# Part 2: Cluster setup\n\n\u003e [!INFO]\n\u003e For simplicity, we will use Docker in the online workshop.\n\u003e\n\u003e But for a setup closer to production, try the Kubernetes setup following [section 2.2](#22-minikube--helm) instead.\n\n## 2.1 Docker\n\nStart up a single-node Weaviate cluster with the following command:\n\n```shell\ndocker compose up -d\n```\n\nThis will start a single-node Weaviate cluster.\n\nCheck an Weaviate endpoint:\n\n```shell\ncurl http://localhost:8080/v1/meta | jq\n```\n\nYou should see a response - this means Weaviate is running!\n\nYou should be able to see the memory usage of the Weaviate pod by running:\n\n```shell\ngo tool pprof -top http://localhost:6060/debug/pprof/heap\n```\n\nNow, go to [Step 2](#step-3-work-with-weaviate)\n\n## 2.2 Minikube \u0026 Helm\n\n\u003e [!NOTE]\n\u003e This is not an actual production setup. But, we can approximate a production set-up by using a local cluster and using Kubernetes to orchestrate our local cluster, just like we would a cluster in production.\n\n### 2.2.1 Set up the cluster\n\nUpdate helm chart \u0026 add the Weaviate repository:\n\n```shell\nhelm repo update weaviate\nhelm repo add weaviate https://weaviate.github.io/weaviate-helm\n```\n\nGet the default values:\n\n```shell\nhelm show values weaviate/weaviate \u003e values-default.yaml\n```\n\nThis file shows you the set of available options for the Weaviate helm chart. (Optionally - take a look at the file with your favorite text editor.)\n\nYou could edit this file, or create a new one with just the values you want to change.\n\nWe will use a pre-configured values file (`values.yaml`) that sets up a single-node Weaviate cluster.\n\nStart a small cluster with Minikube:\n\n```shell\nminikube config set memory 2048\nminikube start --nodes 1\n```\n\n### 2.2.2 Install Weaviate onto our cluster\n\nInstall Weaviate:\n\n```shell\nkubectl create namespace weaviate\nhelm upgrade --install \"weaviate\" weaviate/weaviate --namespace \"weaviate\" --values ./values.yaml\n```\n\nCheck the status:\n\n```shell\nkubectl get pods -n weaviate\n```\n\nShortly, you should see the Weaviate pods running. This should take about a minute.\n\n```shell\n❯ kubectl get pods -n weaviate\nNAME         READY   STATUS    RESTARTS   AGE\nweaviate-0   0/1     Running   0          48s\n```\n\nCan we access the cluster? Try running this, for example:\n\n```shell\ncurl http://localhost:80/v1/meta | jq\n```\n\nWe can't access the Weaviate service yet, because it's not exposed to the outside world.\n\nTo expose the service, we can use `minikube tunnel`:\n\n```shell\nminikube tunnel\n```\n\n\u003e [!TIP]\n\u003e Note that this command will block the terminal. Open a new terminal and continue on.\n\u003e If you want to stop the tunnel, you can do so by pressing `Ctrl+C` or closing the terminal.\n\nNow, you can connect to Weaviate - e.g. to check an Weaviate endpoint:\n\n```shell\ncurl http://localhost:80/v1/meta | jq\n```\n\nYou should see a response - this means Weaviate is running!\n\nWe'll also expose the pprof service, which we can use to monitor the memory usage of the Weaviate pod. (To delete, run `kubectl delete service pprof-service -n weaviate`)\n\n```shell\nkubectl apply -f pprof-service.yaml\n```\n\nYou should be able to see the memory usage of the Weaviate pod by running:\n\n```shell\ngo tool pprof -top http://localhost:6060/debug/pprof/heap\n```\n\nThe Docker-based Weaviate is configured to run on port 8080, whereas the Kubernetes deployment uses port 80. Open `helpers.py` and update `port=8080` to `port=80` and restart the Streamlit app.\n\nNow, go to [Step 3](#step-3-work-with-weaviate)\n\n# Step 3: Work with Weaviate\n\n\u003e [!TIP]\n\u003e If you are new to Weaviate, you can take a look at the introductory materials. Take a look at the slide deck PDF, or the `intro_workshop.ipynb` notebook.\n\u003e For complete examples, see `intro_workshop_finished.ipynb` notebook.\n\n## 3.0 Choose your embedding \u0026 LLM provider\n\nThe workshop is set up to use Cohere by default. We provide helper CLI to download the data \u0026 prepare your project for you:\n\n### 3.0.1 Default Option: Cohere\n\n\u003e [!NOTE]\n\u003e - Recommended if you want to use an API-based solution\n\u003e - Cohere account \u0026 API key required\n\u003e     - Once you have a key, set it as an environment variable: `export COHERE_API_KEY=your-key`\n\u003e     - We will use pre-embedded data for this workshop, so the expense will be for queries only \u0026 minimal\n\u003e     - Still, please set a budget limit in your account for extra safety\n\nRun the following command to set up the workshop, and delete your existing collection data.\n\n```shell\npython workshop_setup.py --provider cohere\n```\n\nIf you have already downloaded the data file, it will use a cached version. To overwrite the file, specify this flag::\n\n```shell\npython workshop_setup.py --provider cohere --use-cache False\n```\n\nWhen the download finishes, continue to the [next section](#31-run-the-demo-streamlit-app).\n\n## 3.1 Run the demo Streamlit app\n\nWe have a Streamlit app that will help you to visualise basic cluster statistics, and to make use of the data. (Remember to navigate to the project directory and activate the virtual environment.) Run it with:\n\n```shell\nstreamlit run app.py\n```\n\nThis will throw an error, but that's OK. We'll fix that in the next step.\n\n## 3.2 Use Weaviate\n\nNow, let's ingest data and play with Weaviate. You will follow through the `workshop.ipynb` notebook in the session. But you can also see `workshop_finished.ipynb` for a complete example.\n\nWe don't want to spoil the whole workshop for you, so we'll leave it here for now. But - if you find yourself ahead of the group, you can try playing with the following sections and ideas:\n\nENJOY THE WORKSHOP!\n\n# Step 4: Additional exercises to try\n\n## 4.1 Scale out Weaviate\n\n### 4.1.1 Docker\n\nFirst, shut down the existing Docker-based Weaviate.\n\n```shell\ndocker compose down\n```\n\nNow, spin up a three-node Weaviate cluster:\n\n```shell\ndocker compose -f docker-compose-three-nodes.yml up -d\n```\n\n(Earlier, we were using the default `docker-compose.yml` file, so we did not need to specify it. Now, we need to specify the file with the `-f` flag.)\n\nThen, run the scripts to create the collection and add data.\n\nNote that this is a slightly different exercise to the Kubernetes-based one. The reason is that the Kubernetes pods were configured with an artificially small amount of RAM, to showcase the benefits of scaling up or out.\n\n### 4.1.2 Kubernetes\n\nTo scale Weaviate out with Minikube, you need to do two things.\n\nFirst, shut down minikube. So, close the tunnel, and stop minikube:\n\n```shell\nminikube stop\n```\n\nThen, start minikube with more nodes:\n\n```shell\nminikube start --nodes 3\n```\n\nUpdate your values file to have more replicas:\n\n```yaml\nreplicas: 3\n```\n\nThen, install Weaviate again:\n\n```shell\nhelm upgrade --install \"weaviate\" weaviate/weaviate --namespace \"weaviate\" --values ./values.yaml\n```\n\nNow, once the pods are ready, go through the same steps as before.\n\nExpose the service with `minikube tunnel`.\n\nThen, run the scripts to create the collection and add data.\n\nHow do the results change? Do you notice the same limitations?\n\n## 4.2 Update the vector index \u0026 quantization settings (Any deployment)\n\nIn `1_create_collection.py`, you can change the settings for the vector index and quantization.\n\nNotice the commented out lines for `quantization`. Try each one, and see how it affects the memory usage. Do you notice changes to the search results? Would you expect it to?\n\nTry changing `.hnsw` to `.flat`. How does this affect the memory usage and the search performance of the system?\n- Note: The `.flat` index can only be used with the `bq` quantization.\n\n## 4.3 Try a larger dataset (Any deployment)\n\nIf you want to experiment with even larger (or smaller) datasets, you can run the following command:\n\n```shell\npython workshop_setup.py --provider \u003cYOUR_PROVIDER\u003e --dataset-size \u003cSIZE\u003e\n```\n\nWhere `\u003cSIZE\u003e` is one of `10000`, `50000` (default), `100000` or `200000`.\n\nThey are pre-vectorized datasets, so you can experiment with different sizes without having to wait for the data to be vectorized, or spend money on the inference.\n\n## 4.4 Increase the pod memory (Kubernetes users only)\n\nIncrease its memory, e.g. to:\n\n```yaml\nresources:\n  requests:\n    cpu: '500m'\n    memory: '1000Mi'\n  limits:\n    cpu: '1000m'\n    memory: '1000Mi'\n```\n\nActually, we have a pre-configured values file for this. Run the following command to upgrade Weaviate with the new values:\n\n```shell\nhelm upgrade --install \"weaviate\" weaviate/weaviate --namespace \"weaviate\" --values ./values-larger.yaml\n```\n\n## Finish up\n\n### Kubernetes\n\nClose the tunnel by closing the terminal on which it is running, and stop minikube:\n\n```shell\nminikube stop\n```\n\n### Docker\n\nWhen finished with the workshop, you can stop the cluster with:\n\n```shell\ndocker compose -f docker-compose.yml down\n```\n\nOr\n\nAnd a three-node cluster with:\n\n```shell\ndocker compose -f docker-compose-three-nodes.yml down\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweaviate-tutorials%2Fscalable-ai-workshop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fweaviate-tutorials%2Fscalable-ai-workshop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweaviate-tutorials%2Fscalable-ai-workshop/lists"}