{"id":18647380,"url":"https://github.com/googleforgames/genai-quickstart","last_synced_at":"2025-04-11T13:30:47.566Z","repository":{"id":196883119,"uuid":"694837008","full_name":"googleforgames/GenAI-quickstart","owner":"googleforgames","description":"Google for Games Generative AI Quickstart","archived":false,"fork":false,"pushed_at":"2024-04-19T13:24:44.000Z","size":1217,"stargazers_count":18,"open_issues_count":10,"forks_count":13,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-04-19T18:14:10.560Z","etag":null,"topics":["games","generative-ai","google","google-cloud"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/googleforgames.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-09-21T19:49:57.000Z","updated_at":"2024-04-19T18:14:10.561Z","dependencies_parsed_at":null,"dependency_job_id":"2146e746-a9d7-4783-95f7-b311be5b0060","html_url":"https://github.com/googleforgames/GenAI-quickstart","commit_stats":null,"previous_names":["googleforgames/genai-quickstart"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleforgames%2FGenAI-quickstart","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleforgames%2FGenAI-quickstart/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleforgames%2FGenAI-quickstart/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleforgames%2FGenAI-quickstart/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/googleforgames","download_url":"https://codeload.github.com/googleforgames/GenAI-quickstart/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223468536,"owners_count":17150162,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["games","generative-ai","google","google-cloud"],"created_at":"2024-11-07T06:26:00.570Z","updated_at":"2024-11-07T06:26:01.167Z","avatar_url":"https://github.com/googleforgames.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GenAI Quickstart for Games\n\nThis project provides set of quickstarts aimed at accelerating GenAI integration and personalization within of live game enviroments using Google Cloud products and open source gaming solutions.\n\nIn today's gaming industry, providing a personalized and engaging experience for players is crucial. This project offers game developers a set of quickstart resources to help them integrate GenAI capabilities into their live game environments. By leveraging Google Cloud products and open-source gaming solutions, you can enhance player engagement, unlock new use cases with Generative AI, and create memorable gaming experiences.\n\nNOTE: This is a rapidly evolving repo and is being adapted for a variety of use cases. If you would like to contribute or notice any bugs, please open an issue and/or feel free to submit a PR for review.\n\nIf you’re using this project, please ★Star this repository to show your interest!\n\n## Project Structure\n\n| Folder                        | Description                   |\n|-------------------------------|-------------------------------|\n| [terraform](./terraform)      | Infrastructure deployment scripts based on Terraform |\n| [examples](./examples)        | Individual quickstarts that can be tested and deployed based on your use case |\n| [src](./src)                  | Core source code that is used as part of our quickstarts |\n\n## Architecture\n\n![Architecture](images/genai-api-arch.png)\n\n## Prerequisites\n\n- [Terraform](https://www.terraform.io/downloads.html)\n- [gcloud](https://cloud.google.com/sdk/docs/install)\n- [kubectl](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl#install_kubectl)\n- [Skaffold](https://skaffold.dev/docs/)\n- [Helm](https://helm.sh/docs/intro/install/)\n\n## Getting started\n\nThe following steps below will walk you through the setup guide for *GenAI Quickstart*. The process will walk through enabling the proper **Google Cloud APIs**, creating the resources via **Terraform**, and deployment of the **Kubernetes manifests** needed to run the project.\n\n\u003e __Note:__ These steps assume you already have a running project in Google Cloud for which you have IAM permissions to deploy resources into.\n\n### 1) Clone this git repository\n\n```\ngit clone https://github.com/googleforgames/GenAI-quickstart.git\n\ncd GenAI-quickstart\n```\n\n### 2) Set ENV variable\n\nSet your unique Project ID for Google Cloud\n\n```\n# To just use your current project\nexport PROJECT_ID=$(gcloud config list --format 'value(core.project)' 2\u003e/dev/null)\n\n# Otherwise set it to the project you wish to use.\n```\n\nSet default location for Google Cloud\n\n```\nexport LOCATION=us-central1\n```\n\nTo better follow along with this quickstart guide, set `CUR_DIR` env variable\n\n```\nexport CUR_DIR=$(pwd)\n```\n\n### 3) Confirm user authentication to Google Cloud project\n\n```\ngcloud auth list\n```\n\nCheck if your authentication is ok and your `PROJECT_ID` is valid.\n\n```\ngcloud projects describe ${PROJECT_ID:?}\n```\n\nYou should see the your `PROJECT_ID` listed with an `ACTIVE` state.\n\n### 4) Enable Google Cloud APIs\n\n```\ngcloud services enable --project ${PROJECT_ID:?} \\\n  aiplatform.googleapis.com \\\n  artifactregistry.googleapis.com \\\n  cloudbuild.googleapis.com \\\n  cloudresourcemanager.googleapis.com \\\n  compute.googleapis.com \\\n  container.googleapis.com \\\n  containerfilesystem.googleapis.com \\\n  containerregistry.googleapis.com \\\n  iam.googleapis.com \\\n  servicecontrol.googleapis.com \\\n  spanner.googleapis.com\n```\n\n### 5) Deploy infrastructure with Terraform\n\n```\ncd ${CUR_DIR:?}/terraform\n\ncat terraform.example.tfvars | sed -e \"s:your-unique-project-id:${PROJECT_ID:?}:g\" \u003e terraform.tfvars\n\nterraform init\n\nterraform plan\n\nterraform apply\n```\n\nThe deployment of cloud resources can take between 5 - 10 minutes. For a detailed view of the resources deployed see [README](terraform/README.md) in `terraform` directory.\n\n### 6) Setup GKE credentials\n\nAfter cloud resources have successfully been deployed with Terraform, get newly created GKE cluster credentials.\n\n```\ngcloud container clusters get-credentials genai-quickstart --region us-central1 --project ${PROJECT_ID:?}\n```\n\nTest your Kubernetes client credentials.\n\n```\nkubectl get nodes\n```\n\n### 7) Deploy GenAI workloads on GKE\n\n\nSwitch to the `genai` directory and render common templates that use your unique project id.\n```\n# Find all files named .template.yaml, replace `your-unique-project-id` with PROJECT_ID, and output to .yaml.\ncd ${CUR_DIR:?}/genai \u0026\u0026 find common -type f -name \"*.template.yaml\" -exec \\\n  bash -c \"template_path={}; sed \\\"s:your-unique-project-id:${PROJECT_ID:?}:g\\\" \u003c \\${template_path} \u003e \\${template_path/%.template.yaml/.yaml} \" \\;\n```\n\nBuild and run GenAI workloads with **Skaffold**\n\n```\ngcloud auth configure-docker ${LOCATION:?}-docker.pkg.dev\n\nexport SKAFFOLD_DEFAULT_REPO=${LOCATION:?}-docker.pkg.dev/${PROJECT_ID:?}/repo-genai-quickstart\n\ncd ${CUR_DIR:?}/genai\n\n# To run all apis and models (requires a GPU node for stable-diffusion)\nskaffold run --build-concurrency=0\n```\n\nAfter workloads are deployed, you can swap to using GPU deployments instead:\n```\n# Scale up a 2xL4 Mixtral 8x7B Deployment:\nkubectl scale -n genai deployment huggingface-tgi-mixtral-small --replicas=1\n\n# Or scale up a 8xL4 Mixtral 8x7B Deployment:\nkubectl scale -n genai deployment huggingface-tgi-mixtral-big --replicas=1\n\n# Scale down CPU Deployment:\nkubectl scale -n genai deployment huggingface-tgi-mistral-cpu --replicas=0\n\n# Note that the `huggingface-tgi-api` Service matches all of the huggingface-tgi-*\n# Deployments, so if you have multiple replicas running, it will load balance\n# between them.\n```\n\nYou can also run the individual backends in isolation:\n```\n# To run only stable-diffusion (requires a GPU node)\n#skaffold run --module stable-diffusion-api-cfg,stable-diffusion-endpt-cfg\n\n# To run only Vertex chat (Vertex AI is required)\n#skaffold run --module vertex-chat-api-cfg\n```\n\n### 8) Tests\n\nAccess the API - You can test the application and all the APIs from here  :)\n\nThe cluster creates an internal passthrough Network Load Balancer (ILB). To access the APIs run:\n```\nkubectl port-forward svc/genai-api -n genai 8080:80\n```\nthen in another window run:\n```\nexport EXT_IP=localhost:8080\necho \"Browse to http://${EXT_IP}/genai_docs to try out the GenAI APIs!\"\n```\nand then navigate to the URL in your browser.\n\nTest the API using `curl`:\n\n```\ncurl -X 'POST' \"http://${EXT_IP}/genai/text\" \\\n  -H 'accept: application/json' \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"prompt\": \"Who are the founders of Google?\"}'\n```\n\nOr test the API using the `api-caller` container inside the cluster:\n```\n# See available service endpoints. The `genai` endpoint wraps them all.\nkubectl get svc -ngenai\n\n# Start `api-caller` pod interactively\nkubectl run -it -ngenai --rm --restart=Never api-caller --image=${SKAFFOLD_DEFAULT_REPO}/api-caller:latest\n\n# Examples:\n\n# See available example scripts\nroot@api-caller:/app# ls\nembeddings.py  genai_api.py  huggingface_tgi.py  npc_chat_api.py  stable_diffusion_api.py  vertex_chat_api.py  vertex_code_api.py  vertex_gemini_api.py  vertex_image_api.py  vertex_text_api.py\n\n# The genai_api script works for text prompts\nroot@api-caller:/app# python3 genai_api.py --endpoint=http://genai-api/genai/text --prompt \"Describe a wombat\"\nINFO:root:Status Code: 200\nINFO:root:Response:    \"A wombat is a marsupial native to Australia. [...]\"\n\n# To try the Smart NPC, first reset the world data:\nroot@api-caller:/app# python3 npc_chat_api.py --endpoint http://genai-api/genai/npc_chat/reset_world_data --empty\nINFO:root:Status Code: 200\nINFO:root:Response:    {\"status\":\"ok\"}\n\n# Then you can use the interactive chat:\nroot@api-caller:/app# python3 npc_chat_api.py --endpoint http://genai-api/genai/npc_chat --chat\n\u003e\u003e\u003e hey, how are you?\n\u003c\u003c\u003c I am doing my best here at the distribution center. It's a tough situation, but I am staying focused on helping those in need. How about you? How are you holding up?\n\n# You can also interact with the services underneath, e.g.: Hugging Face TGI supports an interactive chat\nroot@api-caller:/app# python3 huggingface_tgi.py --endpoint=http://huggingface-tgi-api:8080/v1\n\u003e\u003e\u003e hello!\nINFO:httpx:HTTP Request: POST http://huggingface-tgi-api:8080/v1/chat/completions \"HTTP/1.1 200 OK\"\n\u003c\u003c\u003c  Hello! How can I help you today? If you have any questions or need assistance with something, feel free to ask and I'll do my best to help. If you just want to chat, we can talk about pretty much anything. What's on your mind?\n\n```\n\n## Project cleanup\n\n### Remove Kubernetes resources\nIn `genai` directory\n\n```\ncd ${CUR_DIR:?}/genai\n\nskaffold delete\n```\n\n### Remove infrastructure\nIn `terraform` directory\n\n```\ncd ${CUR_DIR:?}/terraform\n\nterraform destroy\n```\n\n## Troubleshooting\n\n### Not authenticated with Google Cloud project\n\nIf you are not running the above project in Google Cloud shell, make sure you are logged in and authenticated with your desired project:\n\n```\ngcloud auth application-default login\n\ngcloud config set project ${PROJECT_ID:?}\n```\n\nand follow the authentication flow.\n\n\n---\n\n## Contributing\n\nThe entire repo can be cloned and used as-is, or in many cases you may choose to fork this repo and keep the code base that is most useful and relevant for your use case. If you'd like to contribute, then more info can be found witin our [contributing guide](./CONTRIBUTING.md).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogleforgames%2Fgenai-quickstart","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogleforgames%2Fgenai-quickstart","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogleforgames%2Fgenai-quickstart/lists"}