{"id":28394618,"url":"https://github.com/tkestack/tke-ai-playbook","last_synced_at":"2025-06-27T00:32:22.522Z","repository":{"id":294871800,"uuid":"971201140","full_name":"tkestack/tke-ai-playbook","owner":"tkestack","description":null,"archived":false,"fork":false,"pushed_at":"2025-06-26T07:35:03.000Z","size":1044,"stargazers_count":3,"open_issues_count":3,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-06-26T08:36:55.061Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tkestack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"Contributing.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-23T07:02:56.000Z","updated_at":"2025-06-26T07:35:06.000Z","dependencies_parsed_at":"2025-05-31T09:34:25.992Z","dependency_job_id":"6ecb8424-d26b-49b2-8a06-752286590647","html_url":"https://github.com/tkestack/tke-ai-playbook","commit_stats":null,"previous_names":["tkestack/tke-ai-playbook"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tkestack/tke-ai-playbook","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tkestack%2Ftke-ai-playbook","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tkestack%2Ftke-ai-playbook/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tkestack%2Ftke-ai-playbook/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tkestack%2Ftke-ai-playbook/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tkestack","download_url":"https://codeload.github.com/tkestack/tke-ai-playbook/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tkestack%2Ftke-ai-playbook/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262166153,"owners_count":23268993,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-31T18:37:55.574Z","updated_at":"2025-06-27T00:32:22.513Z","avatar_url":"https://github.com/tkestack.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TKE AI Playbook\n\n[English](README.md) | [中文](README_zh.md)\n\n## Project Overview\nThis project provides Kubernetes-based scripts for AI large language model (LLM) operations, including model downloading, inference service deployment, and performance benchmarking, enabling end-to-end AI workflows on Tencent Kubernetes Engine (TKE).\n\n### Prerequisites\n1. Kubernetes cluster (recommended v1.28+)\n2. Tencent Cloud CFS storage (or compatible storage solution)\n3. GPU nodes (3 * H20 nodes used in this project)\n\n### Capabilities\n\n#### Model Download\n\nUse the [Model Download Utility](./helm-charts/model-fetch/README.md) to download models to CFS storage for reuse across inference services.\n\n#### Inference Service Deployment\n\n- [dynamo](./helm-charts/dynamo/README.md): NVIDIA's distributed inference framework (open-sourced at GTC 2025), supporting multiple inference engines including TRT-LLM, vLLM, and SGLang.\n\n#### Performance Benchmarking\n- [LLM Benchmark Suite](./helm-charts/benchmark/README.md)\n\n\n### Examples\n\n#### dynamo\n\n##### Single-Node PD Disaggregation\n\nPrerequisites:\n- 1 x 8 GPU Node.\n\nDeploys:\n- 1 x VllmWorker (4 GPUs for decode phase)\n- 4 x PrefillWorker (1 GPU each for prefill phase)\n\n```bash\nbash examples/dynamo/single-node/deploy.sh\n```\n\n##### 3 Nodes PD Disaggregation\n\n[TODO]\n\n### Scripts\n\n#### Quick Test for OpenAI-format API Endpoint\n1. Script: [test-openai-api](./scripts/test-openai-api.sh)\n2. Usage:\n\n```bash\nAPI_ENDPOINT=\u003cyour-api-endpoint\u003e bash scripts/test-openai-api.sh\n\n# Test localhost:8080\nAPI_ENDPOINT=http://localhost:8080 bash scripts/test-openai-api.sh\n```\n\n#### Generate Model Download Job\n1. Script: [tke-llm-downloader](./scripts/tke-llm-downloader.sh)\n2. Usage:\n\n```bash\n# Download 'deepseek-ai/DeepSeek-R1' model to PVC 'ai-model' with 6 concurrency\nbash scripts/tke-llm-downloader.sh --pvc ai-model --completions 6 --parallelism 6 --model deepseek-ai/DeepSeek-R1\n\n# Download 'Qwen/Qwen3-32B' model to PVC 'ai-model' with 3 concurrency\nbash scripts/tke-llm-downloader.sh --pvc ai-model --completions 3 --parallelism 3 --model Qwen/Qwen3-32B\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftkestack%2Ftke-ai-playbook","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftkestack%2Ftke-ai-playbook","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftkestack%2Ftke-ai-playbook/lists"}