{"id":15055287,"url":"https://github.com/vemonet/libre-chat","last_synced_at":"2025-04-13T00:49:44.506Z","repository":{"id":183984835,"uuid":"671115276","full_name":"vemonet/libre-chat","owner":"vemonet","description":"🦙 Free and Open Source Large Language Model (LLM) chatbot web UI and API. Self-hosted, offline capable and easy to setup. Powered by LangChain.","archived":false,"fork":false,"pushed_at":"2024-11-12T19:57:16.000Z","size":4534,"stargazers_count":152,"open_issues_count":6,"forks_count":19,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-13T00:49:39.156Z","etag":null,"topics":["chatbot","chatgpt","langchain","large-language-models","llama2","llm","llm-inference","mixtral","no-code","open-source","openapi","question-answering","self-hosted"],"latest_commit_sha":null,"homepage":"https://vemonet.github.io/libre-chat","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vemonet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-07-26T15:13:50.000Z","updated_at":"2025-04-09T03:27:32.000Z","dependencies_parsed_at":"2024-01-17T04:00:27.494Z","dependency_job_id":"7c97483a-38d0-4ed5-98e8-a6fbb552f7e4","html_url":"https://github.com/vemonet/libre-chat","commit_stats":{"total_commits":233,"total_committers":2,"mean_commits":116.5,"dds":"0.0042918454935622075","last_synced_commit":"dbb8e3400e5258112179783d74c9cc54310cb72b"},"previous_names":["vemonet/libre-llm","vemonet/libre-chat"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vemonet%2Flibre-chat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vemonet%2Flibre-chat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vemonet%2Flibre-chat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vemonet%2Flibre-chat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vemonet","download_url":"https://codeload.github.com/vemonet/libre-chat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248650436,"owners_count":21139672,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","chatgpt","langchain","large-language-models","llama2","llm","llm-inference","mixtral","no-code","open-source","openapi","question-answering","self-hosted"],"created_at":"2024-09-24T21:40:06.590Z","updated_at":"2025-04-13T00:49:44.476Z","avatar_url":"https://github.com/vemonet.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# \u003cspan\u003e\u003cimg height=\"30\" src=\"https://raw.github.com/vemonet/libre-chat/main/docs/docs/assets/logo.png\"\u003e\u003c/span\u003e Libre Chat\n\n[![Test package](https://github.com/vemonet/libre-chat/actions/workflows/test.yml/badge.svg)](https://github.com/vemonet/libre-chat/actions/workflows/test.yml) [![Coverage](https://coverage-badge.samuelcolvin.workers.dev/vemonet/libre-chat.svg)](https://coverage-badge.samuelcolvin.workers.dev/redirect/vemonet/libre-chat)\n\n[![PyPI - Version](https://img.shields.io/pypi/v/libre-chat.svg?logo=pypi\u0026label=PyPI\u0026logoColor=silver)](https://pypi.org/project/libre-chat/)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/libre-chat.svg?logo=python\u0026label=Python\u0026logoColor=silver)](https://pypi.org/project/libre-chat/)\n[![License](https://img.shields.io/pypi/l/libre-chat)](https://github.com/vemonet/libre-chat/blob/main/LICENSE.txt) [![Pull requests welcome](https://img.shields.io/badge/pull%20requests-welcome-brightgreen)](https://github.com/vemonet/libre-chat/fork)\n\nEasily configure and deploy a **fully self-hosted chatbot web service** based on open source Large Language Models (LLMs), such as [Mixtral](https://mistral.ai/news/mixtral-of-experts) or [Llama 2](https://ai.meta.com/llama/), without the need for knowledge in machine learning.\n\n\u003c/div\u003e\n\n- 🌐 Free and Open Source chatbot web service with UI and API.\n- 🏡 Fully self-hosted, not tied to any service, and offline capable. Forget about API keys! Models and embeddings can be pre-downloaded, and the training and inference processes can run off-line if necessary.\n- 🔌 Web API described using OpenAPI specs: GET/POST operations, websocket for streaming response\n- 🪶 Chat web UI working well on desktop and mobile, with streaming response, and markdown rendering. Alternative gradio-based UI also available.\n- 🚀 Easy to setup, no need to program, just configure the service with a [YAML](https://yaml.org/) file, and start it with 1 command\n- 📦 Available as a `pip` package 🐍, or `docker` image 🐳\n- 🐌 No need for GPU, this will work even on your laptop CPU! That said, just running on CPUs can be quite slow (up to 1min to answer a documents-base question on recent laptops).\n- 🦜 Powered by [`LangChain`](https://python.langchain.com) and [`llama.cpp`](https://github.com/ggerganov/llama.cpp) to perform inference locally.\n- 🤖 Various types of agents can be deployed:\n  - **💬 Generic conversation**: do not need any additional training, just configure settings such as the template prompt\n  - **📚 Documents-based question answering** (experimental): automatically build similarity vectors from documents uploaded through the API UI, the chatbot will use them to answer your question, and return which documents were used to generate the answer (PDF, CSV, HTML, JSON, markdown, and more supported).\n- 🔍 Readable logs to understand what is going on.\n\n## 📖 Documentation\n\nFor more details on how to use Libre Chat check the documentation at **[vemonet.github.io/libre-chat](http://vemonet.github.io/libre-chat)**\n\n\n![UI screenshot](https://raw.github.com/vemonet/libre-chat/main/docs/docs/assets/screenshot.png)\n\n![UI screenshot](https://raw.github.com/vemonet/libre-chat/main/docs/docs/assets/screenshot-light.png)\n\n## 🏗️ Work in progress\n\n\u003e [!WARNING]\n\u003e This project is a work in progress, use it with caution.\n\nThose checkpoints are features we plan to work on in the future, feel free to let us know in the issues if you have any comment or request.\n\n- [x] Stream response to the websocket to show words as they are generated\n- [ ] Add button to let the user stop the chatbot generation\n- [ ] Add authentication mechanisms? (OAuth/OpenID Connect) https://github.com/vemonet/libre-chat/issues/5\n- [ ] Add conversational history? https://milvus.io/blog/conversational-memory-in-langchain.md\n- [ ] Add an admin dashboard web UI to enable users to upload/inspect/delete documents for QA, see/edit the config of the chatbot.\n- [ ] Kubernetes deployment (Helm chart?)\n\n## 🐳 Deploy with docker\n\nIf you just want to quickly deploy it using the pre-trained model `Mixtral-8x7B-Instruct`, you can use docker:\n\n```bash\ndocker run -it -p 8000:8000 ghcr.io/vemonet/libre-chat:main\n```\n\nYou can configure the deployment using environment variables. For this using a `docker compose` and a `.env` file is easier, first create the `docker-compose.yml` file:\n\n```yaml\nversion: \"3\"\nservices:\n  libre-chat:\n    image: ghcr.io/vemonet/libre-chat:main\n    volumes:\n      # ⚠️ Share folders from the current directory to the /data dir in the container\n      - ./chat.yml:/data/chat.yml\n      - ./models:/data/models\n      - ./documents:/data/documents\n      - ./embeddings:/data/embeddings\n      - ./vectorstore:/data/vectorstore\n    ports:\n      - 8000:8000\n```\n\nAnd create a `chat.yml` file with your configuration in the same folder as the `docker-compose.yml`:\n\n```yaml\nllm:\n  model_path: ./models/mixtral-8x7b-instruct-v0.1.Q2_K.gguf\n  model_download: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF/resolve/main/mixtral-8x7b-instruct-v0.1.Q2_K.gguf\n  temperature: 0.01    # Config how creative, but also potentially wrong, the model can be. 0 is safe, 1 is adventurous\n  max_new_tokens: 1024 # Max number of words the LLM can generate\n  # Always use input for the human input variable with a generic agent\n  prompt_variables: [input, history]\n  prompt_template: |\n    Your are an assistant, please help me\n\n    {history}\n    User: {input}\n    AI Assistant:\n\nvector:\n  vector_path: null # Path to the vectorstore to do QA retrieval, e.g. ./vectorstore/db_faiss\n  # Set to null to deploy a generic conversational agent\n  vector_download: null\n  embeddings_path: ./embeddings/all-MiniLM-L6-v2 # Path to embeddings used to generate the vectors, or use directly from HuggingFace: sentence-transformers/all-MiniLM-L6-v2\n  embeddings_download: https://public.ukp.informatik.tu-darmstadt.de/reimers/sentence-transformers/v0.2/all-MiniLM-L6-v2.zip\n  documents_path: ./documents # Path to documents to vectorize\n  chunk_size: 500             # Maximum size of chunks, in terms of number of characters\n  chunk_overlap: 50           # Overlap in characters between chunks\n  chain_type: stuff           # Or: map_reduce, reduce, map_rerank. More details: https://docs.langchain.com/docs/components/chains/index_related_chains\n  search_type: similarity     # Or: similarity_score_threshold, mmr. More details: https://python.langchain.com/docs/modules/data_connection/retrievers/vectorstore\n  return_sources_count: 2     # Number of sources to return when generating an answer\n  score_threshold: null       # If using the similarity_score_threshold search type. Between 0 and 1\n\ninfo:\n  title: \"Libre Chat\"\n  version: \"0.1.0\"\n  description: |\n    Open source and free chatbot powered by [LangChain](https://python.langchain.com) and [llama.cpp](https://github.com/ggerganov/llama.cpp)\n  examples:\n  - What is the capital of the Netherlands?\n  - Which drugs are approved by the FDA to mitigate Alzheimer symptoms?\n  - How can I create a logger with timestamp using python logging?\n  favicon: https://raw.github.com/vemonet/libre-chat/main/docs/docs/assets/logo.png\n  repository_url: https://github.com/vemonet/libre-chat\n  public_url: https://chat.semanticscience.org\n  contact:\n    name: Vincent Emonet\n    email: vincent.emonet@gmail.com\n  license_info:\n    name: MIT license\n    url: https://raw.github.com/vemonet/libre-chat/main/LICENSE.txt\n```\n\nFinally start your chat service with:\n\n```bash\ndocker compose up\n```\n\n## 📦️ Usage with pip\n\nThis package requires Python \u003e=3.8, simply install it with `pipx` or `pip`:\n\n```bash\npip install libre-chat\n```\n\n### ⌨️ Use as a command-line interface\n\nYou can easily start a new chat web service including UI and API using your terminal:\n\n```bash\nlibre-chat start\n```\n\nProvide a specific config file:\n\n```bash\nlibre-chat start config/chat-vectorstore-qa.yml\n```\n\nFor re-build of the vectorstore:\n\n```bash\nlibre-chat build --vector vectorstore/db_faiss --documents documents\n```\n\nGet a full rundown of the available options with:\n\n```bash\nlibre-chat --help\n```\n\n### 🐍 Use with python\n\nOr you can use this package in python scripts:\n\n```python\nimport logging\n\nimport uvicorn\nfrom libre_chat import ChatConf, ChatEndpoint, Llm\n\nlogging.basicConfig(level=logging.getLevelName(\"INFO\"))\nconf = ChatConf(\n  model_path=\"./models/mixtral-8x7b-instruct-v0.1.Q2_K.gguf\",\n  vector_path=None\n)\nllm = Llm(conf=conf)\nprint(llm.query(\"What is the capital of the Netherlands?\"))\n\n# Create and deploy a FastAPI app based on your LLM\napp = ChatEndpoint(llm=llm, conf=conf)\nuvicorn.run(app)\n```\n\n## 🤝 Credits\n\nInspired by:\n\n- https://github.com/kennethleungty/Llama-2-Open-Source-LLM-CPU-Inference\n- https://github.com/lm-sys/FastChat\n- https://github.com/liltom-eth/llama2-webui\n\n\u003ca href=\"https://www.flaticon.com/free-icons/llama\" title=\"llama icons\"\u003eLlama icons created by Freepik - Flaticon\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvemonet%2Flibre-chat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvemonet%2Flibre-chat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvemonet%2Flibre-chat/lists"}