https://github.com/cyberagentailab/condlora
Code for our paper "A Single Linear Layer Yields Task-Adapted Low-Rank Matrices"
https://github.com/cyberagentailab/condlora
Last synced: 9 months ago
JSON representation
Code for our paper "A Single Linear Layer Yields Task-Adapted Low-Rank Matrices"
- Host: GitHub
- URL: https://github.com/cyberagentailab/condlora
- Owner: CyberAgentAILab
- License: apache-2.0
- Created: 2024-04-04T01:41:38.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2024-04-04T01:56:58.000Z (about 2 years ago)
- Last Synced: 2025-09-10T07:42:52.037Z (9 months ago)
- Language: Python
- Homepage: https://arxiv.org/abs/2403.14946
- Size: 152 KB
- Stars: 3
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# CondLoRA
## How to use
### Environment
- OS: Debian GNU/Linux 10 (buster)
- Python version: 3.8.5
- CUDA version: 11.3.109
### Install libraries
```
pip install -r requirements.txt
cd ConditionalLoRA
python setup.py install
```
### Hyperparameter (learning rate) search
```
cd experiments
bash hp_search.sh [GPU number] [GLEU task name (e.g. sst2, mnli, etc.)] [lora type (e.g. lora, adalora, conditional_lora)] [seed number]
```
### Train Model
```
cd experiments/[GLEU task name (e.g. sst2, mnli, etc.)]
bash train.sh [GPU number] [lora type (e.g. lora, adalora, conditional_lora)] [seed number]
```
### Evaluate Model
```
python src/evaluate.py --model_path [path to trained lora model] --task [GLEU task name (e.g. sst2, mnli, etc.)] --batch_size [batch size] --max_length [max length of input text]
```