{"id":34590772,"url":"https://github.com/nullxjx/vllm-docker-compose","last_synced_at":"2026-04-24T13:32:22.874Z","repository":{"id":262598104,"uuid":"887762982","full_name":"nullxjx/vllm-docker-compose","owner":"nullxjx","description":"docker-compose for vllm, support sticky-sessions using traefik to enable the prefix-caching feature of vllm","archived":false,"fork":false,"pushed_at":"2024-12-21T09:02:34.000Z","size":8,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-21T10:18:14.655Z","etag":null,"topics":["docker-compose","traefik","vllm"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nullxjx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-13T08:40:59.000Z","updated_at":"2024-12-21T09:02:37.000Z","dependencies_parsed_at":null,"dependency_job_id":"a4645ecc-ef9b-45d6-8d69-1e3db0cee4fd","html_url":"https://github.com/nullxjx/vllm-docker-compose","commit_stats":null,"previous_names":["nullxjx/vllm-docker-compose"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nullxjx/vllm-docker-compose","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullxjx%2Fvllm-docker-compose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullxjx%2Fvllm-docker-compose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullxjx%2Fvllm-docker-compose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullxjx%2Fvllm-docker-compose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nullxjx","download_url":"https://codeload.github.com/nullxjx/vllm-docker-compose/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullxjx%2Fvllm-docker-compose/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32225733,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T13:21:15.438Z","status":"ssl_error","status_checked_at":"2026-04-24T13:21:15.005Z","response_time":64,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker-compose","traefik","vllm"],"created_at":"2025-12-24T10:50:57.940Z","updated_at":"2026-04-24T13:32:22.868Z","avatar_url":"https://github.com/nullxjx.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# vllm-docker-compose\ndocker-compose for [vllm](https://github.com/vllm-project/vllm/), support [sticky-sessions]( https://doc.traefik.io/traefik/routing/services/#sticky-sessions) using traefik to enable the [prefix-caching](https://docs.vllm.ai/en/v0.5.5/automatic_prefix_caching/apc.html) feature of vllm\n\n\n## Usage\nstart services\n```bash\ndocker-compose up -d\n``` \n\nstop services\n```bash\ndocker-compose down\n```\n\ntest sticky-sessions\n```bash\nbash request.sh\n```\n\nYou can observe that if you do not delete the cookie, your request will be load balanced to the same vllm instance every time.\nIf you delete the cookies, or if you don't use cookies, then load balancing between different instances will work normally.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnullxjx%2Fvllm-docker-compose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnullxjx%2Fvllm-docker-compose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnullxjx%2Fvllm-docker-compose/lists"}