https://github.com/evilfreelancer/moda
Is a framework designed to enhance the performance and flexibility of large language models by dynamically selecting and integrating specialized LoRA adapters based on the input query.
https://github.com/evilfreelancer/moda
adapters gpt llama2 lora router transformers
Last synced: 3 months ago
JSON representation
Is a framework designed to enhance the performance and flexibility of large language models by dynamically selecting and integrating specialized LoRA adapters based on the input query.
- Host: GitHub
- URL: https://github.com/evilfreelancer/moda
- Owner: EvilFreelancer
- License: mit
- Created: 2024-06-09T10:13:46.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-06-16T16:15:17.000Z (almost 2 years ago)
- Last Synced: 2025-10-07T10:41:41.352Z (7 months ago)
- Topics: adapters, gpt, llama2, lora, router, transformers
- Language: Jupyter Notebook
- Homepage:
- Size: 197 KB
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.en.md
- License: LICENSE
Awesome Lists containing this project
README
# Mixture of dynamically connected adapters (MoDA)
Mixture of Dynamically Connected Adapters (MoDA) is a framework designed to enhance the performance and flexibility of
large language models by dynamically selecting and integrating specialized adapters based on the input query. This
project facilitates seamless integration and execution of various adapters to optimize responses for different types of
queries.
[Русский](./README.md) | **English**
## Features
* **Dynamic Adapter Selection** - automatically selects the most appropriate adapter based on the
input query conditions.
* **Multiple Adapter Integration** - supports multiple adapters with distinct functionalities, such
as language-specific or domain-specific tasks.
* **Efficient Resource Utilization** - leverages efficient models and techniques like loading
models in 8-bit to save computational resources.
* **Extensible Framework** - easily add and configure new adapters via a YAML configuration file.
## Configuration
MoDA uses a `config.yml` file to specify the models and adapters. Below is the schema for the configuration file.
```yaml
caller:
name:
load_in_8bit:
socket:
name:
load_in_8bit:
functions:
- name:
prompt:
description:
condition:
chat_template:
```
For a full example, refer to the [config.yml](./config.yml) file located in the root of this repository.
## Usage
To understand how to use MoDA, refer to the [moda.ipynb](./moda.ipynb) Jupyter notebook in the root of this repository.
This notebook provides a comprehensive example of loading models, initializing a conversation, handling function calls,
and dynamically switching between different adapters to process user queries efficiently.
## License
This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for details.
## Citation
```
@misc{moda2024sources,
title={Mixture of dynamically connected adapters (MoDA)},
author={Pavel Rykov},
year={2024}
}
```