{"id":15853752,"url":"https://github.com/ivangabriele/docker-fastchat","last_synced_at":"2025-03-15T21:31:00.741Z","repository":{"id":164129962,"uuid":"639586759","full_name":"ivangabriele/docker-fastchat","owner":"ivangabriele","description":"[Work In Progress] Server/Cloud-ready FastChat Docker images.","archived":false,"fork":false,"pushed_at":"2024-03-04T02:26:46.000Z","size":25,"stargazers_count":4,"open_issues_count":6,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-07T02:35:38.582Z","etag":null,"topics":["ai","ai-chatbot","artificial-intelligence","chat-gpt","chatgpt","cloud","data-science","docker","docker-compose","fastchat","llama","lmsys","script","server","vicuna","vicuna-13b","vicuna-7b"],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ivangabriele.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-11T19:17:42.000Z","updated_at":"2024-03-14T17:41:26.000Z","dependencies_parsed_at":null,"dependency_job_id":"2e6423fc-3fcf-4d91-88c1-818a1fdc927f","html_url":"https://github.com/ivangabriele/docker-fastchat","commit_stats":{"total_commits":4,"total_committers":1,"mean_commits":4.0,"dds":0.0,"last_synced_commit":"c08c22620e17a91bf164f6e6fcf515fd62d96496"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivangabriele%2Fdocker-fastchat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivangabriele%2Fdocker-fastchat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivangabriele%2Fdocker-fastchat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivangabriele%2Fdocker-fastchat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ivangabriele","download_url":"https://codeload.github.com/ivangabriele/docker-fastchat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243792339,"owners_count":20348623,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-chatbot","artificial-intelligence","chat-gpt","chatgpt","cloud","data-science","docker","docker-compose","fastchat","llama","lmsys","script","server","vicuna","vicuna-13b","vicuna-7b"],"created_at":"2024-10-05T19:22:46.048Z","updated_at":"2025-03-15T21:31:00.248Z","avatar_url":"https://github.com/ivangabriele.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FastChat Docker Image\n\n[![img-docker]][link-docker]\n[![img-github-actions]][link-github-actions]\n\nServer/Cloud-ready [FastChat][link-fastchat] Docker image based either on [Nvdidia CUDA][link-docker-nvidia-cuda]\nUbuntu image including:\n\n- Python 3\n- Rust\n- [FastChat][link-fastchat]\n- Vicuna-7B\n\nThis image is intented for people who want to run FastChat using on their own server or cloud instance.\n\n[Docker Hub prebuilt images][link-docker] are automatically updated on a daily basis.\n\n---\n\n- [Caveats](#caveats)\n- [Supported tags and respective `Dockerfile` links](#supported-tags-and-respective-dockerfile-links)\n- [Docker Environment Variables](#docker-environment-variables)\n  - [`PORT`](#port)\n- [Roadmap](#roadmap)\n- [Convenience Server Scripts (no Docker)](#convenience-server-scripts-no-docker)\n- [Contribute](#contribute)\n  - [Prerequisites](#prerequisites)\n  - [Build](#build)\n  - [Run](#run)\n\n---\n\n## Caveats\n\n- **⚠️ The web application is NOT secured.** It's up to you to secure your public HTTP port connection.\n- You need a GPU server or instance. The default entrypoint uses a single GPU run but you can override it\n  using `docker run --entrypoint ...` or a Docker Compose file via `entrypoint: [...]`.\n- This image is **BIG** since it includes a 9GB data model.\n\n\n## Supported tags and respective `Dockerfile` links\n\n- [`latest-cuda12`](https://github.com/ivangabriele/docker-fastchat/blob/main/dockerfiles/lastest-cuda12.Dockerfile)\n\n## Docker Environment Variables\n\n### `PORT`\n\n**REQUIRED**\n\nThis is the server exposed port to emit and listen socket events.\n\n## Roadmap\n\n- Add a CPU-only version\n\n## Convenience Server Scripts (no Docker)\n\nThis repository provides a few convenience scripts to install and run FastChat without using Docker,\nfor better performances:\n\n- `debian.sh`: Copy this file in your `/home/[username]` directory and run it from this directory via `sh ./debian.sh`.\n\n## Contribute\n\n`X` is FastChat version:\n\n- `latest`: clone the `main` branch from [FastChat repository](https://github.com/lm-sys/FastChat)\n\n`Z` is the variant tag:\n\n- `cuda12`: NVidia GPUs\n\n### Prerequisites\n\n- Docker ([Desktop](https://docs.docker.com/desktop/) or [Engine](https://docs.docker.com/engine/install/))\n- Make (installation depends on your OS)\n\n### Build\n\n```sh\nmake build-X-Z\n```\n\nExample: `make build-latest-cuda12`.\n\n### Run\n\n```sh\nmake run-X\n# or:\nmake run-X-Z\n```\n\nExample: `make run-latest-cuda12`.\n\n---\n\n[img-docker]: https://img.shields.io/docker/pulls/ivangabriele/docker-fastchat?style=for-the-badge\n[img-github-actions]:\n  https://img.shields.io/github/actions/workflow/status/ivangabriele/docker-fastchat/main.yml?branch=main\u0026label=Build\u0026style=for-the-badge\n\n[link-docker-nvidia-cuda]: https://hub.docker.com/r/nvidia/cuda\n[link-docker]: https://hub.docker.com/r/ivangabriele/fastchat\n[link-fastchat]: https://github.com/lm-sys/FastChat#readme\n[link-github-actions]: https://github.com/ivangabriele/docker-fastchat/actions/workflows/main.yml?query=branch%3Amain\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivangabriele%2Fdocker-fastchat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fivangabriele%2Fdocker-fastchat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivangabriele%2Fdocker-fastchat/lists"}