{"id":14964845,"url":"https://github.com/innightwolfsleep/llm_telegram_bot","last_synced_at":"2025-04-05T16:04:22.667Z","repository":{"id":154890769,"uuid":"628147351","full_name":"innightwolfsleep/llm_telegram_bot","owner":"innightwolfsleep","description":"LLM telegram bot","archived":false,"fork":false,"pushed_at":"2025-02-18T16:03:37.000Z","size":1216,"stargazers_count":118,"open_issues_count":11,"forks_count":24,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-29T15:11:18.598Z","etag":null,"topics":["aiogram","bot","exllama","large-language-models","llamacpp","llm","telegram","telegram-bot","text-generation-webui"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/innightwolfsleep.png","metadata":{"files":{"readme":"readme.md","changelog":"history/telegram_history_here.txt","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-15T03:41:07.000Z","updated_at":"2025-03-22T19:51:58.000Z","dependencies_parsed_at":"2023-12-30T18:32:42.212Z","dependency_job_id":"7bd6c8d7-a52e-4010-b200-1145c9c9a5f0","html_url":"https://github.com/innightwolfsleep/llm_telegram_bot","commit_stats":{"total_commits":247,"total_committers":5,"mean_commits":49.4,"dds":0.05668016194331982,"last_synced_commit":"7668521f44793fc14c0cee225cd0e79d1394cdbf"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innightwolfsleep%2Fllm_telegram_bot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innightwolfsleep%2Fllm_telegram_bot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innightwolfsleep%2Fllm_telegram_bot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innightwolfsleep%2Fllm_telegram_bot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/innightwolfsleep","download_url":"https://codeload.github.com/innightwolfsleep/llm_telegram_bot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247198468,"owners_count":20900081,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiogram","bot","exllama","large-language-models","llamacpp","llm","telegram","telegram-bot","text-generation-webui"],"created_at":"2024-09-24T13:33:51.435Z","updated_at":"2025-04-05T16:04:22.639Z","avatar_url":"https://github.com/innightwolfsleep.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\r\n![Image1](https://raw.githubusercontent.com/innightwolfsleep/llm_telegram_bot/refs/heads/main/manuals/images/textgen_telegram.png)\r\n\r\n- wrAPPer for llama.cpp(default), exllama or transformers. \r\n\r\n- wrAPPer for [ollama](https://github.com/ollama/ollama) (config example - config/ollama_config.json)\r\n\r\n- an EXTension for [oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui).s\r\n\r\nProvide telegram chat with various additional functional like buttons, prefixes, voice/image generation.\r\n\r\n\r\n---------------\r\nHOW TO INSTALL (**standalone app**):\r\n1) clone this repo  \r\n```git clone https://github.com/innightwolfsleep/llm_telegram_bot ```\r\n2) install requirements.  \r\n```pip install -r llm_telegram_bot\\requirements_app.txt```\r\n\r\nHOW TO RUN (**standalone app**):\r\n1) get bot token from https://t.me/BotFather \r\n2) copy bot token to `configs\\telegram_token.txt`\r\n3) move your model file to `models\\`\r\n4) set **model_path** to your model in `configs\\app_config.json`\r\n5) start `run.cmd`(windows) or `run.sh`(linux). \r\n\r\n(optional) to use exllama: \r\n```\r\ngit clone https://github.com/turboderp/exllama source\\generators\\exllama\r\npip install -r source\\generators\\exllama\\requirements.txt\r\n```\r\n\r\n(optional) to use exllamav2: \r\n```\r\ngit clone https://github.com/turboderp/exllamav2 source\\generators\\exllamav2\r\ncd source\\generators\\exllamav2\r\npython setup.py install --user\r\n```\r\n\r\n(optional) to use llama.cpp with GPU acceleration reinstall abetlen/llama-cpp-python by guide: [llama-cpp-python#installation-with-hardware-acceleration](https://github.com/abetlen/llama-cpp-python#installation-with-hardware-acceleration) or [this guide for windows](https://github.com/abetlen/llama-cpp-python/discussions/871)\r\n\r\n---------------\r\nHOW TO INSTALL (**extension mode**):\r\n1) obviously, install  [oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui) first, add model, set all options you need\r\n2) run `cmd_windows.bat` or `cmd_linux.sh` to enable venv\r\n3) clone this repo to \"text-generation-webui\\extensions\"  \r\n`git clone https://github.com/innightwolfsleep/llm_telegram_bot extensions\\telegram_bot`\r\n4) install requirements  \r\n`pip install -r extensions\\telegram_bot\\requirements_ext.txt`\r\n\r\nHOW TO USE (**extension mode**):\r\n1) get bot token from https://t.me/BotFather \r\n2) add your bot token in `extensions\\telegram_bot\\configs\\telegram_token.txt` file or oobabooga environment\r\n3) add flags to CMD_FLAGS.txt `--api --extension telegram_bot`\r\n4) run appropriate start_OS script.\r\n(direct integration is not available now)\r\n---------------\r\n\r\nHOW TO INSTALL/USE (**google collab**):\r\n1) run notebook at [manuals/llm_telegram_bot_cublas.ipynb](https://colab.research.google.com/drive/1nTX1q7WRkXwSbLLCUs3clPL5eoJXShJq?usp=sharing)\r\n2) install, set bot token, run\r\n---------------\r\n(optional) if you are facing internet issue, change `proxy_url` at `app_config.json` into your own proxy. For example: `https://127.0.0.1:10808`\r\n\r\n\r\n\r\nFEATURES:\r\n- chat templates (see [manuals/custom_prompt_templates.md])\r\n- chat and notebook modes\r\n- session for all users are separative (by chat_id)\r\n- local session history - conversation won't be lost if server restarts. Separated history between users and chars.\r\n- nice \"X typing\" during generating (users will not think that bot stuck)\r\n- buttons: impersonate, continue previous message, regenerate last message, remove last messages from history, reset history button, new char loading menu\r\n- you can load new characters from text-generation-webui\\characters with button\r\n- you can load new model during conversation with button\r\n- \"+\" or \"#\" user message prefix for impersonate: \"#Chiharu sister\" or \"+Castle guard\". Or even ask bot generate your own message \"+You\"\r\n- \"-\" or \"!\" prefix to replace last bot message\r\n- \"++\" prefix replace bot name during chat (switch conversation to another character)\r\n- \"--\" prefix replace you name during chat\r\n- \"==\" prefix to add message to context\r\n- \"📷\" prefix to make photo via SD api. Write like \"📷Chiharu Yamada\", not single \"📷\". Need to run ([StableDiffusion](https://github.com/AUTOMATIC1111/stable-diffusion-webui)) with --api key first.\r\n- save/load history in chat by downloading/forwarding to chat .json file\r\n- integrated auto-translate (you can set model/user language parameter) \r\n- receiving text files (code, text, etc)\r\n- voice generating ([silero](https://github.com/snakers4/silero-models)), en and ru variants\r\n- translation_as_hidden_text option in .cfg - if you want to learn english with bot\r\n- telegram_users.txt - list of permitted users (if empty - permit for all)\r\n- antiflood - one message per 15 sec from one user\r\n- improved group chatting mode\r\n\r\n\r\nCONFIGURATION:\r\n\r\n`app_config.json` - config for running as standalone app (`run.sh` or `run.cmd`)  \r\n`ext_config.json` - config for running as extension for [oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui)\r\n\r\n```\r\nx_config.json\r\n    bot_mode=admin  \r\n        specific bot mode. admin for personal use\r\n            - admin - bot answer for everyone in chat-like mode. All buttons, include settings-for-all are avariable for everyone. (Default)\r\n            - chat - bot answer for everyone in chat-like mode. All buttons, exclude settings-for-all are avariable for everyone. (Recommended for chatting)\r\n            - chat-restricted - same as chat, but user can't change default character\r\n            - persona - same as chat-restricted, but reset/regenerate/delete message are unavailable too. \r\n            - notebook - notebook-like mode. Prefixes wont added automaticaly, only \"\\n\" separate user and bot messages. Restriction like chat mode.\r\n            - query - same as notebook, but without history. Each question for bot is like new convrsation withot influence of previous questions\r\n    user_name_template=\r\n        user name template, useful for group chat.\r\n        if empty bot always get default name of user - You. By default even in group chats bot perceive all users as single entity \"You\"\r\n        but it is possible force bot to perceive telegram users names with templates: \r\n            FIRSTNAME - user first name (Jon)\r\n            LASTNAME - user last name (Dow)\r\n            USERNAME - user nickname (superguy)\r\n            ID - user Id (999999999)\r\n        so, user_name_template=\"USERNAME FIRSTNAME ID\" translatede to user name \"superguy Jon 999999999\"\r\n        but if you planed to use template and group chat - you shold add \"\\n\" sign to stopping_strings to prevent bot impersonating!!!\r\n    generator_script=GeneratorLlamaCpp\r\n        name of generator script (generators folder):\r\n            - generator_exllama - based on llama-cpp-python, recommended\r\n            - generator_llama_cpp - based on llama-cpp-python, recommended\r\n            - generator_langchain_llama_cpp - based in langchain+llama\r\n            - generator_transformers - based on transformers, untested\r\n            - generator_text_generator_webui_openapi - use oobabooga/text-generation-webui OpenAPI extension\r\n            - (BROKEN) generator_text_generator_webui - module to integrate in oobabooga/text-generation-webui (curently broken:( )\r\n            - (OUTDATED) generator_text_generator_webui_api - use oobabooga/text-generation-webui API (old api version)\r\n    model_path=models\\llama-13b.ggml.q4_0.gguf\r\n        path to model file or directory\r\n    characters_dir_path=characters\r\n    default_char=Example.yaml\r\n        default cahracter and path to cahracters folder\r\n    presets_dir_path=presets\r\n    default_preset=Shortwave.yaml\r\n        default generation preset and path to preset folder\r\n    model_lang=en\r\n    user_lang=en\r\n        default model and user language. User language can be switched by users, individualy.\r\n    html_tag_open=\u003cpre\u003e\r\n    html_tag_close=\u003c/pre\u003e\r\n        tags for bot answers in tg. By default - preformatted text (pre)\r\n    history_dir_path=history\r\n        directory for users history\r\n    token_file_path=configs\\\\telegram_token.txt\r\n        bot token. Ask https://t.me/BotFather\r\n    admins_file_path=configs\\\\telegram_admins.txt\r\n        users whos id's in admins_file switched to admin mode and can choose settings-for-all (generating settings and model)\r\n    users_file_path=configs\\\\telegram_users.txt\r\n        if just one id in users_file - bot will ignore all users except this id (id's). Even admin will be ignored\r\n    generator_params_file_path=configs\\\\telegram_generator_params.json\r\n        default text generation params, overwrites by choosen preset \r\n    user_rules_file_path=configs\\\\telegram_user_rules.json\r\n        user rules matrix \r\n    telegram_sd_config=configs\\\\telegram_sd_config.json\r\n        stable diffusion api config\r\n    stopping_strings=\u003cEND\u003e,\u003cSTART\u003e,end{code}\r\n        generating settings - which text pattern stopping text generating? Add \"\\n\" if bot sent too much text.\r\n    eos_token=None\r\n        generating settings\r\n    translation_as_hidden_text=on\r\n        if \"on\" and model/user lang not the same - translation will be writed under spoiler. If \"off\" - translation without spoiler, no original text in message.\r\n    sd_api_url=\"http://127.0.0.1:7860\"\r\n    stable diffusion api url, need to use \"photo\" prefixes\r\n    proxy_url\r\n        to avoid provider blocking\r\n\r\n\r\ngenerator_params.json\r\n    config for generator \r\n\r\nsd_config.json\r\n    config for stable diffusion\r\n\r\ntelegram_admins.txt\r\n    list of users id who forced to admin mode. If telegram_users not empty - must be in telegram_users too!\r\n\r\ntelegram_users.txt\r\n    list og users id (or groups id) who permitted interact with bot. If empty - everyone permitted\r\n\r\ntelegram_token.txt (you can use .env instead)\r\n    telegram bot token\r\n\r\ntelegram_user_rules.json\r\n    buttons visibility config for various bot modes\r\n\r\n```\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finnightwolfsleep%2Fllm_telegram_bot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finnightwolfsleep%2Fllm_telegram_bot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finnightwolfsleep%2Fllm_telegram_bot/lists"}