{"id":14964725,"url":"https://github.com/remixer-dec/botality-ii","last_synced_at":"2025-10-25T08:30:21.771Z","repository":{"id":140443613,"uuid":"612741834","full_name":"remixer-dec/botality-ii","owner":"remixer-dec","description":"telegram bot for self-hosted local inference of stable diffusion, text-to-speech and large language models, such as llama3","archived":false,"fork":false,"pushed_at":"2024-05-13T00:43:39.000Z","size":386,"stargazers_count":40,"open_issues_count":0,"forks_count":10,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-10-21T20:00:47.036Z","etag":null,"topics":["ai","alpaca","gpt-2","gpt-j","llama","llama3","llamacpp","lora","m1-mac","mps","multimodal","self-hosted","stable-diffusion","stt","telegram-bot","tta","tts"],"latest_commit_sha":null,"homepage":"https://github.com/remixer-dec/botality-ii/wiki/Overview","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/remixer-dec.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-11T20:52:50.000Z","updated_at":"2025-10-04T14:16:49.000Z","dependencies_parsed_at":"2023-11-19T19:26:26.745Z","dependency_job_id":"42705ffd-8ca0-4785-8f92-896455c46e33","html_url":"https://github.com/remixer-dec/botality-ii","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/remixer-dec/botality-ii","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/remixer-dec%2Fbotality-ii","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/remixer-dec%2Fbotality-ii/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/remixer-dec%2Fbotality-ii/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/remixer-dec%2Fbotality-ii/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/remixer-dec","download_url":"https://codeload.github.com/remixer-dec/botality-ii/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/remixer-dec%2Fbotality-ii/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280870852,"owners_count":26405598,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-24T02:00:06.418Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","alpaca","gpt-2","gpt-j","llama","llama3","llamacpp","lora","m1-mac","mps","multimodal","self-hosted","stable-diffusion","stt","telegram-bot","tta","tts"],"created_at":"2024-09-24T13:33:41.730Z","updated_at":"2025-10-25T08:30:21.444Z","avatar_url":"https://github.com/remixer-dec.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Botality II  \n  \nThis project is an implementation of a modular **telegram bot** based on [aiogram](https://github.com/aiogram/aiogram), designed for local ML Inference with remote service support. Currently integrated with:\n-  **Stable Diffusion** (using [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui) API),\n-  **TTS** text-to-speech engine (using [TTS (VITS)](https://github.com/coqui-ai/TTS) and [so-vits-SVC](https://github.com/svc-develop-team/so-vits-svc/tree/4.0)) as well as OS voices.  \n-  **STT** integrated with multiple speech recognition engines, including [whisper.cpp](https://github.com/ggerganov/whisper.cpp)[\u003csup\u003e1\u003c/sup\u003e](https://github.com/stlukey/whispercpp.py), [whisperS2T](https://github.com/shashikg/WhisperS2T), [silero](https://github.com/snakers4/silero-models), [wav2vec2](https://ai.meta.com/blog/wav2vec-20-learning-the-structure-of-speech-from-raw-audio/)  \n-  **LLMs** such as [llama (1-3)](https://github.com/facebookresearch/llama), [gpt-j](https://github.com/kingoflolz/mesh-transformer-jax#gpt-j-6b), [gpt-2](https://huggingface.co/gpt2) with support for assistant mode via instruct-tuned lora models and multimodality via [adapter-model](https://github.com/OpenGVLab/LLaMA-Adapter) \n- **TTA** experimental text-to-audio support via [audiocraft](https://github.com/facebookresearch/audiocraft)  \n\nAccelerated LLM inference support: [llama.cpp](https://github.com/ggerganov/llama.cpp), [mlc-llm](https://github.com/mlc-ai/mlc-llm) and [llama-mps](https://github.com/remixer-dec/llama-mps/)  \nRemote LLM inference support: [oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui/), [LostRuins/koboldcpp](https://github.com/LostRuins/koboldcpp) and [llama.cpp server](https://github.com/ggerganov/llama.cpp/tree/master/examples/server)  \nCompatibility table is available [here](COMPATIBILITY.md)  \n  \nEvolved from predecessor [Botality I](https://github.com/remixer-dec/ru-gpt3-telegram-bot)  \nShipped with an easy-to-use webui, you can run commands and talk with the bot right in the webui.\n\n\u003cimg src=\"https://i.imgur.com/cnLXvTa.png\" alt=\"preview\" height=\"500\"\u003e\n\n### Documentation\n\nYou can find it [here](https://github.com/remixer-dec/botality-ii/wiki/Getting-started) (coming soon)\n\n### Changelog\nSome versions have breaking changes, see [Changelog file](CHANGELOG.md) for more information\n\n### Features\n[Bot]\n- User-based queues and delayed task processing\n- Multiple modes to filter access scopes (WL/BL/Both/Admin-only)\n- Support of accelerated inference on M1 Macs\n- Memory manager, keeps track of models loaded at the same time and loads/unloads them on demand.\n\n[LLM]\n- Supports dialog mode casually playing a role described in a character file, keeping chat history with all users in group chats or with each user separately\n- Character files can be easily localized for any language for non-english models\n- Assistant mode via /ask command or with direct replies (configurable)\n- Single-reply short-term memory for assistant feedback\n- Supports visual question answering, when multimodal-adapter is available\n\n[SD]\n- CLI-like way to pass stable diffusion parameters\n- pre-defined prompt wrappers\n- lora integration with easy syntax: lora_name100 =\u003e \u0026lt;lora:lora_name:1.0\u0026gt; and custom lora activators\n\n[TTS]\n- can be run remotely, or on the same machine\n- tts output is sent as voice messages\n- can be used on voice messages (speech and acapella songs) to dub them with a different voice \n\n[STT]\n- can be activated as a speech recognition tool via /stt command replying to voice messages  \n- if `stt_autoreply_mode` parameter is not `none`, it recognizes voice messages and replies to them with LLM and TTS modules  \n\n[TTA]\n- can be used with /sfx and /music commands after adding `tta` to `active_modules`  \n\n\u003cimg src=\"https://i.imgur.com/eCEcgCc.jpg\" alt=\"multimodality demo\" title=\"multimodality demo\" height=\"400\"\u003e  \n  \n### Setup:\n- copy `.env.example` file and rename the copy to `.env`, do NOT add the .env file to your commits! \n- set up your telegram bot token and other configuration options in `.env` file\n- install requirements `pip install -r requrements.txt`\n- install optional requirements if you want to use tts and tts_server `pip install -r requrements-tts.txt` and `pip install -r requrements-llm.txt` if you want to use llm, you'll probably also need a fresh version of [pytorch](https://pytorch.org/get-started/locally/). For speech-to-text run `pip install -r requrements-stt.txt`, for text-to-audio run `pip install -U git+https://git@github.com/facebookresearch/audiocraft#egg=audiocraft`\n- you can continue configuration in the webui, it has helpful tips about each configuration option\n- for stable diffusion module, make sure that you have webui installed and it is running with `--api` flag\n- for text-to-speech module download VITS models, put their names in `tts_voices` configuration option and path to their directory in `tts_path`\n- for llm module, see LLM Setup section bellow\n- if you want to use webui + api, run it with `python dashboard.py`, otherwise run the bot with `python bot.py` \n  \npython3.10+ is recommended, due to aiogram compatibility, if you are experiencing problems with whisper or logging, please update numpy.\n### Supported language models (tested):  \n#### Python/Pytorch backend  \n- [original llama](https://github.com/facebookresearch/llama/blob/main/example.py) (7b version was tested on [llama-mps fork](https://github.com/remixer-dec/llama-mps/tree/multimodal-adapter) for macs), requires running the bot with `python3.10 -m torch.distributed.launch --use_env bot.py`  \nassistant mode for original llama is available with [LLaMa-Adapter](https://github.com/ZrrSkywalker/LLaMA-Adapter), to use both chat and assistant mode, some changes[[1]](https://github.com/remixer-dec/llama-mps/commit/a9b319a927461e4d9b5d74789b3b4a079cb90620)[[2]](https://github.com/remixer-dec/llama-mps/commit/74e9734eefaba721d03974924d0a43175237f32c) are necessary for non-mac users.\n- [hf llama](https://huggingface.co/decapoda-research/llama-7b-hf/tree/main) (tests outdated) + [alpaca-lora](https://github.com/tloen/alpaca-lora) / [ru-turbo-alpaca-lora](https://huggingface.co/IlyaGusev/llama_7b_ru_turbo_alpaca_lora)\n- [gpt-2](https://huggingface.co/gpt2) (tested on [ru-gpt3](https://github.com/ai-forever/ru-gpts)), nanoGPT (tested on [minChatGPT](https://github.com/ethanyanjiali/minChatGPT) [[weights](https://huggingface.co/ethanyanjiali/minChatGPT/blob/main/final_ppo_model_gpt2medium.pt)])\n- [gpt-j](https://github.com/kingoflolz/mesh-transformer-jax#gpt-j-6b) (tested on a custom model)\n  \n#### C++ / TVM backend  \n- [llama.cpp](https://github.com/abetlen/llama-cpp-python) (tested on a lot of models)[[models]](https://huggingface.co/models?sort=downloads\u0026search=GGUF)]\n- [mlc-llm-chat](https://mlc.ai/mlc-llm/#windows-linux-mac) (tested using prebuilt binaries on demo-vicuna-v1-7b-int3 model, M1 GPU acceleration confirmed, integrated via [mlc-chatbot](https://github.com/XinyuSun/mlc-chatbot))\n  \n#### Remote api backend  \n- [oobabooga webui](https://github.com/oobabooga/text-generation-webui/) \n- [kobold.cpp](https://github.com/LostRuins/koboldcpp/) with the same `remote_ob` backend\n- [llama.cpp server](https://github.com/ggerganov/llama.cpp/tree/master/examples/server) with `remote_lcpp` llm backend option (Obsidian model w/ multimodality tested)\n\n\n### LLM Setup\n- Make sure that you have enough RAM / vRAM to run models.\n- Download the weights (and the code if needed) for any large language model\n- in .env file, make sure that `\"llm\"` is in `active_modules`, then set:  \n`llm_paths` - change the path(s) of model(s) that you downloaded  \n`llm_backend` - select from `pytorch`, `llama.cpp`, `mlc_pb`, `remote_ob`, `remote_lcpp`\n`llm_python_model_type` = if you set `pytorch` in the previous option, set the model type that you want to use, it can be `gpt2`,`gptj`,`llama_orig`, `llama_hf` and `auto_hf`.  \n`llm_character` = a character of your choice, from `characters` directory, for example `characters.gptj_6B_default`, character files also have prompt templates and model configuration options optimal to specific model, feel free to change the character files, edit their personality and use with other models.  \n`llm_assistant_chronicler` = a input/output formatter/parser for assistant task, can be `instruct` or `raw`, do not change if you do not use `mlc_pb`.  \n`llm_history_grouping` = `user` to store history with each user separately or `chat` to store group chat history with all users in that chat  \n`llm_assistant_use_in_chat_mode` = `True`/`False` when False, use /ask command to ask the model questions without any input history, when True, all messages are treated as questions.  \n  \n- For llama.cpp: make sure that you have a c++ compiler, then put all necessary flags to enable GPU support, and install it `pip install llama-cpp-python`, download model weights and change the path in `llm_paths`.\n- For mlc-llm, follow the installation instructions from the docs, then clone [mlc-chatbot](https://github.com/XinyuSun/mlc-chatbot), and put 3 paths in `llm_paths`. Use with `llm_assistant_use_in_chat_mode=True` and with `raw` chronicler.  \n- For oobabooga webui and kobold.cpp, instead of specifying `llm_paths`, set `llm_host`, set `llm_active_model_type` to `remote_ob` and set the `llm_character` to one that has the same prompt format / preset as your model. Run the server with --api flag.\n- For llama.cpp c-server, start the `./server`, set its URL in `llm_host` and set `llm_active_model_type` to `remote_lcpp`, for multimodality please refer to this [thread](https://www.reddit.com/r/LocalLLaMA/comments/17jus3h/obsidian_worlds_first_3b_multimodal_opensource_llm/)\n  \n  \n### Bot commands\nSend a message to your bot with the command **/tti -h** for more info on how to use stable diffusion in the bot, and **/tts -h** for tts module. The bot uses the same commands as voice names in configuration file for tts. Try **/llm** command for llm module details. LLM defaults to chat mode for models that support it, assistant can be called with /ask command\n  \nLicense: the code of this project is currently distributed under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) license, third party libraries might have different licenses.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fremixer-dec%2Fbotality-ii","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fremixer-dec%2Fbotality-ii","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fremixer-dec%2Fbotality-ii/lists"}