{"id":19283038,"url":"https://github.com/chuloai/brainchulo","last_synced_at":"2025-04-22T01:32:27.337Z","repository":{"id":155704845,"uuid":"630972834","full_name":"ChuloAI/BrainChulo","owner":"ChuloAI","description":"Harnessing the Memory Power of the Camelids","archived":false,"fork":false,"pushed_at":"2023-10-19T06:38:12.000Z","size":1205,"stargazers_count":145,"open_issues_count":10,"forks_count":11,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-11-08T21:40:18.539Z","etag":null,"topics":["chromadb","fastapi","langchain","llama","llm","microsoft-guidance","python","retrieval-augmented","sqlmodel","vector-store"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ChuloAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["Karajan421","paolorechia","iGavroche"]}},"created_at":"2023-04-21T15:37:51.000Z","updated_at":"2024-08-23T16:30:04.000Z","dependencies_parsed_at":"2023-06-25T23:07:38.069Z","dependency_job_id":"41d99a75-08de-412b-9294-a11901e0973b","html_url":"https://github.com/ChuloAI/BrainChulo","commit_stats":{"total_commits":302,"total_committers":8,"mean_commits":37.75,"dds":"0.39072847682119205","last_synced_commit":"cacf3acff407ed131ef02196dfede54d8ff216da"},"previous_names":["chuloai/brainchulo"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChuloAI%2FBrainChulo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChuloAI%2FBrainChulo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChuloAI%2FBrainChulo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChuloAI%2FBrainChulo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ChuloAI","download_url":"https://codeload.github.com/ChuloAI/BrainChulo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223887120,"owners_count":17219867,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chromadb","fastapi","langchain","llama","llm","microsoft-guidance","python","retrieval-augmented","sqlmodel","vector-store"],"created_at":"2024-11-09T21:29:30.420Z","updated_at":"2024-11-09T21:29:31.021Z","avatar_url":"https://github.com/ChuloAI.png","language":"Python","readme":"# BrainChulo: Harnessing the Memory Power of the Camelids\n\nBrainChulo is a powerful Chat application with an emphasis on its memory management system. Our work is inspired by the camel family's resilience and adaptability. The repository contains a custom LTM (Long-Term Memory) approach for Llama-based LLMs (Large Language Models), and is designed to help users enhance their experience when using Open-Source LLMs.\n\n## Current Look-n-Feel\n\n![image](https://github.com/ChuloAI/BrainChulo/assets/95258328/2a6a44ab-0597-431a-b85d-902c44eefa6b)\n\n\n## Features\n\n- Custom LTM implementation based on Open Source Large Language Models\n- Memory retrieval algorithms for quick recall\n- Easy-to-use interface for storing and managing memories\n- Modern User Interface\n- One-Shot, Few-Shot, and Tool-Capable Agents with Full Vector-Based Memory Integration\n- Ability to handle multiple conversations\n- Ability to upvote or downvote AI answers for fine-tuning\n\n## Installation\n### Oobabooga UI (not officially supported currently)\n**Update 06.06.2023** As of today, we're dropping support to [Oobabooga Text Generation WebUI](https://github.com/oobabooga/text-generation-webui).The reason being is it does not offer enough support for the [guidance library](https://github.com/microsoft/guidance) features.\n\n**Update 08.06.2023** We've raised a PR to add an extension that would enable us to again use oobabooga. If that's important for you, [leave a thumbs up](https://github.com/oobabooga/text-generation-webui/pull/2554) so it gets merged.\n\n\nNote that currently, we only support models on GPU (GPTQ or Hugging Face), because the guidance does not (yet) fully support llama cpp bindings, or any GGML model.\nThis would be the case until [this PR is merged on guidance library](https://github.com/microsoft/guidance/pull/70).\n\nWe plan to reintroduce CPU support through GGML / llamacpp-bindings - at least for the LLaMA models - when this PR is fully merged into guidance.\n\n\n### Using docker-compose\nTo use BrainChulo, there are four required steps.\n### 1 - Install requirements (docker)\n\n1. [Docker Engine](https://docs.docker.com/engine/)\n2. [Docker Compose v2](https://docs.docker.com/compose/)\n3. [nvidia-docker](https://github.com/NVIDIA/nvidia-docker)\n\nIf you want to use docker-compose v1, you might run into an issue with the port binding - though there's an easy workaround here:\nhttps://github.com/ChuloAI/BrainChulo/issues/39\n\n\n### 2 - Clone your desired model from Hugging Face\n\nChoose a model that either implements the Hugging Face API, if you're downloading them from TheBloke, typically they have the `-HF` suffix, or a GPTQ one (`-GPTQ`) \n\nThere a couple of scripts to help you with this:\n- `bootstrap_models.py`\n- `download-model.py` (highly-inspired script from Oobabooga's repo)\n\nIf you simply execute `python bootstrap_models.py` it will create the required directory and download a pair of recommended models to get started. You can override the default model by passing additional arguments:\n\nFor instance, downloading WizardLM-30B-GPTQ\n```bash\npython bootstrap_models.py TheBloke/WizardLM-30B-GPTQ\n```\n\nUnder the hood, this is roughly equivalent to this:\n\n```bash\nmkdir models\ncd models\ngit clone https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2\ngit clone https://huggingface.co/TheBloke/wizardLM-7B-HF\ncd ..\n```\n\n### 3 - Update the environment variables\nCopy the `.env.example` file. In Linux, that is done:\n\n```bash\ncp .env.example .env\n```\n\nYou should then override the following variables to match your downloaded model:\n```\nMODEL_PATH=/models/vicuna-AlekseyKorshuk-7B-GPTQ-4bit-128g\n# General Settings\nGENERAL_LOADING_METHOD=GPTQ\nGENERAL_BOOL_CPU_OFFLOADING=true\n# HuggingFace\nHF_BOOL_USE_QUANT=true\nHF_BOOL_USE_4_BIT=true\n# GPTQ\nGPTQ_INT_WBITS=4\nGPTQ_INT_GROUP_SIZE=128\nGPTQ_INT_PRE_LOADED_LAYERS=20\nGPTQ_DEVICE=\"cuda\"\n```\nNote that you only need to set the variables according to your desired model loading method.\n`GENERAL_LOADING_METHOD` expects either `GPTQ` or `HUGGING_FACE`. Most of these parameters are self-explanatory,\nexcept `GPTQ_INT_PRE_LOADED_LAYERS` which only changes how many layers are preloaded for the `CPU OFFLOADING` when loading a GPTQ model.\nSetting a number too high for the model e.g., 50 for a 7B model triggers an error.\n\n\n\n### 4 - Download, build and start Docker containers through the docker-compose\n```\ndocker-compose up --build\n```\n\n## Usage\n### BrainChulo\n#### Starting the service\nThe easiest way to start BrainChulo is using Docker with `docker-compose`:\n\n```\n# from the root directory of the project, start with:\ndocker-compose up --build\n\n# To shut it down\ndocker compose down\n\n# After pulling the latest code, make sure to run the database migrations\ndocker-compose exec backend alembic upgrade head\n```\n\n## Enabling Flow Agents\nWe have a experimental feature that enables what we call flow agents. To activate it, set the feature flag to true in the .env file:\n```\nUSE_FLOW_AGENTS=true\n```\n\n## Creating a Plugin\n\n1. Create a new directory for your plugin, e.g. `/app/plugins/my_plugin`\n\n2. Inside the new directory, create a new Python module, e.g. `myplugin_main.py`\n\n3. In `myplugin_main.py`, define your plugin routes using FastAPI. See the `/app/plugins/sample_plugin` directory for an example.\n\n4. Create a `database.py` file to define your SQLModel models. Data for hese models will be persisted in the main database.\n\n5. Once you have defined your models, run the following command **with your own message** to generate a new Alembic migration file. Migration files allow the application to add your models to the database schema:\n\n```\nalembic revision --autogenerate -m \"Add Sample Plugin Demo Model\"\n```\n\n6. Run `alembic upgrade head` to run your migration. This will update your database.\n\n**Note:** If you wish to run these commands using the stood-up containers, prefix them as such:\n\n```\n# To create a migration\ndocker-compose exec backend alembic revision --autogenerate -m \"Add Sample Plugin Demo Model\"\n\n# To run a migration\ndocker-compose exec backend alembic upgrade head\n```\n\n\n**Developers:** While you may use the container-based approach since it is a development container with hot reloading you may also wish to start BrainChulo's services manually. To do so, run the `main.py` script from within the `app` directory, then start the frontend:\n\n```\ncd ./app\npython main.py\n\n# if within the ./app directory\ncd ../frontend\nnpm run dev\n```\n\n\nThis will eventually launch the BrainChulo application. Point your web browser to the following default URL:\n\n```\nhttp://localhost:5173/\n```\n\nThis interface allows you to chat or load text documents which will be used as context in the BrainChulo application.\n\n### \u0026#9888; WARNING \u0026#9888;\nAs we develop BrainChulo the application is gaining abilities which allow it to access and change files on the filesystem. This is a major security risk. We strongly recommend that you run BrainChulo in a Docker container, using the provided `Dockerfile`.\n\nTo run BrainChulo in a Docker container, you just need to run:\n\n`docker-compose up`\n\nTo shut down the container:\n\n`docker-compose down`\n\n\n## Roadmap\n\nHere's a detailed roadmap for the BrainChulo project:\n\n1. **Create an End-to-End capable of creating, persisting, and using an index which can be loaded as context to a conversation using `langchain memory` module or `llama-index` package.** This will allow the agent to maintain context and continuity between conversations and ensure that information is not lost over time.\n\n2. **Implement a Long-Term Memory (LTM) mechanism using a Vector Database layer with fine-tuning/training capabilities for LTM Management.** This will allow the agent to remember past conversations and information for longer periods of time, and provide a more personalized experience to the user.\n\n3. **Implement Tools for the agent (ability to browse the web, send a tweet, read a file, use the REPL).** These tools will enable the agent to perform various tasks and make it more versatile.\n\n4. **Create an Integration point for external systems.** This will allow other systems to interact with the BrainChulo agent and expand its capabilities.\n\n5. **Integrate into existing Text Generation Systems such as oobabooga and Kobold.** This will enable the BrainChulo agent to work with other text generation systems and provide a wider range of outputs to the user.\n\nPlease note that this roadmap is subject to change based on community feedback and contributions. We welcome your input and ideas as we work together to add long-term memory to custom LLMs!\n\n\n\n## Contributing\n\nWe welcome contributions to BrainChulo from the open source community! If you would like to contribute to the project, please fork the repository and submit a pull request. For more information on how to contribute, please see the [CONTRIBUTING.md](CONTRIBUTING.md) file.\n\n\n\u003ca href=\"https://discord.gg/9prDPY2rpU\" target=\"_blank\" title=\"Join us on Discord\"\u003e\u003cimg src=\"https://user-images.githubusercontent.com/95258328/235766839-47336cbb-f338-4939-b2ec-b33363020f95.png\" width=\"150\" alt=\"Join BrainChulo on Discord\" /\u003e\u003c/a\u003e\n\n## License\nBrainChulo is licensed under the MIT license. See [LICENSE.md](LICENSE.md) for more information.\n\n\n## Notes for developers\nMore commands are available on Docker. They are not necessary but can help developers\nBuild the Docker image: \n\n`docker build -t brainchulo .`\n\nRun the Docker container: \n\n`docker run -p 7865:7865 --name brainchulo brainchulo`\n\n\nIf you wish to mount the BrainChulo local repository to the running container, run the following command:\n\n`docker run -p 7865:7865 -v /path/to/local/code:/app brainchulo`\n\nIf you want hot reloading when coding, start the app with the following command:\n```\ngradio main.py\n```\n\n## Further resources\n\n### Medium Articles\n\n#### Introduction to Langchain\nhttps://medium.com/@paolorechia/creating-my-first-ai-agent-with-vicuna-and-langchain-376ed77160e3\n#### Q/A with Sentence Transformer + Vicuna\nhttps://medium.com/@paolorechia/building-a-question-answer-bot-with-langchain-vicuna-and-sentence-transformers-b7f80428eadc\n#### Fine tuning for Python REPL\nhttps://medium.com/@paolorechia/fine-tuning-my-first-wizardlm-lora-ca75aa35363d\n#### Deep dive into Guidance\nhttps://betterprogramming.pub/a-deep-dive-into-guidances-source-code-16681a76fb20\n#### Prototyping a Small Guidance Select Clone With ExLLama Logits Bias\nhttps://betterprogramming.pub/prototyping-a-small-guidance-clone-with-exllama-logits-bias-de26f85623ab\n\n","funding_links":["https://github.com/sponsors/Karajan421","https://github.com/sponsors/paolorechia","https://github.com/sponsors/iGavroche"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchuloai%2Fbrainchulo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchuloai%2Fbrainchulo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchuloai%2Fbrainchulo/lists"}