{"id":13797885,"url":"https://github.com/noco-ai/spellbook-docker","last_synced_at":"2025-05-13T04:32:44.457Z","repository":{"id":207234525,"uuid":"715248012","full_name":"noco-ai/spellbook-docker","owner":"noco-ai","description":"AI stack for interacting with LLMs, Stable Diffusion, Whisper, xTTS and many other AI models","archived":false,"fork":false,"pushed_at":"2024-05-01T15:46:58.000Z","size":2504,"stargazers_count":157,"open_issues_count":2,"forks_count":11,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-30T02:38:15.821Z","etag":null,"topics":["automatic-speech-recognition","bark","llama2","llm-inference","mixtral","musicgeneration","stable-diffusion","text-to-speech","whisper","xttsv2"],"latest_commit_sha":null,"homepage":"https://github.com/noco-ai/spellbook-docker/wiki","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"osl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/noco-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-06T19:01:58.000Z","updated_at":"2025-04-28T19:30:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"c6e354fc-621b-48b3-bd26-f66b987cde40","html_url":"https://github.com/noco-ai/spellbook-docker","commit_stats":null,"previous_names":["noco-ai/spellbook-docker"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noco-ai%2Fspellbook-docker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noco-ai%2Fspellbook-docker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noco-ai%2Fspellbook-docker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noco-ai%2Fspellbook-docker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/noco-ai","download_url":"https://codeload.github.com/noco-ai/spellbook-docker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253501674,"owners_count":21918316,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automatic-speech-recognition","bark","llama2","llm-inference","mixtral","musicgeneration","stable-diffusion","text-to-speech","whisper","xttsv2"],"created_at":"2024-08-04T00:00:35.721Z","updated_at":"2025-05-13T04:32:39.446Z","avatar_url":"https://github.com/noco-ai.png","language":"Shell","funding_links":[],"categories":["📚 Contents"],"sub_categories":[],"readme":"# Spellbook Docker Compose\n\nThe repository contains the Docker Compose files for running the Spellbook AI Assistant stack. The function calling features\nrequire ExLlama and a Nvidia Ampere or better GPU for real-time results.\n\n![UI demo](https://github.com/noco-ai/spellbook-docker/blob/master/ui-demo.gif)\n\n## Stack Architecture\n\n![Software stack diagram](https://github.com/noco-ai/spellbook-docker/blob/master/stack.png)\n\n## Ubuntu 22 Install Instructions\n\nThese instructions should work to get the SpellBook framework up and running on Ubuntu 22. A Nvidia video card supported by ExLlama is required for routing.\n\n- Default username: admin\n- Default password: admin\n\n### Docker Installation\n\n```bash\n# add Dockers official GPG key:\nsudo apt-get update\nsudo apt-get install ca-certificates curl gnupg\nsudo install -m 0755 -d /etc/apt/keyrings\ncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg\nsudo chmod a+r /etc/apt/keyrings/docker.gpg\n\n# add the repository to apt sources:\necho \\\n  \"deb [arch=\"$(dpkg --print-architecture)\" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \\\n  \"$(. /etc/os-release \u0026\u0026 echo \"$VERSION_CODENAME\")\" stable\" | \\\n  sudo tee /etc/apt/sources.list.d/docker.list \u003e /dev/null\nsudo apt-get update\n\n# install docker, create user and let current user access docker\nsudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin\nsudo groupadd docker\nsudo usermod -aG docker $USER\nsudo newgrp docker\nsudo shutdown -r now\n```\n\n### Build and Start Containers (No GPU)\n\nThe docker-compose-nogpu.yml is useful for running the UI and middleware in a situation where you want another backend handling you GPUs and LLMs. For example\nif you are also using Text Generation UI and do not want to mess with it settings this compose file can be used to just run the UI, allowing you then to then connect it to the endpoint provided by Oobabooga or any other OpenAI compatible backend.\n\n```bash\ndocker compose -f docker-compose-nogpu.yml build\ndocker compose -f docker-compose-nogpu.yml up\n```\n\n### Build and Start Additional Workers (No GPU)\n\nIf you have more than one server you can run additional Elemental Golem workers to give the UI access to more resources. A few steps need to be taken on the\nprimary Spellbook server that is running the UI, middleware and other resources like Vault.\n\n- Run these command on the primary server that is running the UI and middleware software in Docker.\n- Copy the read token to a temp file to copy it to the worker server.\n- Make note of the LAN IP address of the primary server. It is needed for the GOLEM_VAULT_HOST and GOLEM_VAULT_HOST variables.\n- Make sure ports for RabbitMQ and Vault are open.\n\n```bash\nsudo more /var/lib/docker/volumes/spellbook-docker_vault_share/_data/read-token \nip address\nsudo ufw allow 5671\nsudo ufw allow 5672\nsudo ufw allow 8200\n```\n\n- Run these command on the server running the worker. The GOLEM_ID needs to be unique for every server and golem1 is used by the primary.\n- The first time you run the container it will timeout, git CTL + C or wait then copy the Vault token.\n```bash\ndocker compose -f docker-compose-worker-nogpu.yml build\nGOLEM_VAULT_HOST=10.10.10.X GOLEM_AMQP_HOST=10.10.10.X GOLEM_ID=golem2 docker compose -f docker-compose-worker-nogpu.yml up\nsudo su\necho \"TOKEN FROM PRIMARY SERVER\" \u003e /var/lib/docker/volumes/spellbook-docker_vault_share/_data/read-token\nexit\nGOLEM_VAULT_HOST=10.10.10.X GOLEM_AMQP_HOST=10.10.10.X GOLEM_ID=golem2 docker compose -f docker-compose-worker-nogpu.yml up\n```\n\n### Nvidia Driver Installation\n\n```bash\n# make sure system see's the Nvidia graphic(s) card\nlspci | grep -e VGA\n\n# check available drivers\nubuntu-drivers devices\n\n# install the latest driver\nsudo apt install nvidia-driver-535\n\n# restart the server\nsudo shutdown -h now\n\n# confirm driver was installed\nnvidia-smi\n\n# install the Nvidia docker toolkit\ncurl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \\\n  \u0026\u0026 curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \\\n    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \\\n    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list \\\n  \u0026\u0026 \\\n    sudo apt-get update\n\nsudo apt-get install -y nvidia-container-toolkit\nsudo nvidia-ctk runtime configure --runtime=docker\nsudo systemctl restart docker\n\n# verify see the output of nvidia-smi for inside a container\nsudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi\n```\n\n### Build and Start Containers (Nvidia GPU)\n\n```bash\ndocker compose build\ndocker compose up\n```\n\n### Build and Start Additional Workers (Nvidia GPU)\n\nFollow the directions under the *Build and Start Additional Workers (No GPU)* section substituting the build and up lines for the ones found below.\n```bash\ndocker compose -f docker-compose-worker.yml build\nGOLEM_VAULT_HOST=10.10.10.X GOLEM_AMQP_HOST=10.10.10.X GOLEM_ID=golem2 docker compose -f docker-compose-worker.yml up\n```\n\n### Port Forwarding\n\nThis repository assumes you are running the docker containers on your local system if this is not the case make sure ports **3000** and **4200** are forwarded to the host running the docker containers.\n\n### Fresh Install\n\nFor a fresh install of the stack run the following commands, this will remove all downloaded models and all conversation and configuration records.\n\n```bash\ncd spellbook-docker\ndocker compose down\ndocker volume rm spellbook-docker_models_share\ndocker volume rm spellbook-docker_vault_share\ngit pull origin master\ndocker compose build\ndocker compose up\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoco-ai%2Fspellbook-docker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnoco-ai%2Fspellbook-docker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoco-ai%2Fspellbook-docker/lists"}