https://github.com/dataesr/llm-finetuning

https://github.com/dataesr/llm-finetuning

Last synced: 5 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/dataesr/llm-finetuning
• Owner: dataesr
• License: mit
• Created: 2025-03-27T11:30:30.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2025-11-28T15:18:56.000Z (7 months ago)
• Last Synced: 2025-11-29T13:44:47.943Z (7 months ago)
• Language: Python
• Size: 165 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
> build docker image

```make docker-build-finetuning``` --> build the finetuning image

```make docker-build-inference``` --> build the inference image

```make docker-build-inference-app``` --> build the inference app image

```make docker-push-finetuning``` --> push the finetuning image

```make docker-push-inference``` --> push the inference image

```make docker-push-inference-app``` --> push the inference app image

> launch finetuning job on ovh

`ovhai job run  --gpu 1 --volume llm-datasets@1azgra:/workspace/datasets:ro --volume llm-jobs@1azgra:/workspace/jobs:rwd --env HF_TOKEN=  ghcr.io/dataesr/llm-finetuning:latest -- uv run main.py  --mode ["train", "push"] --model_name  --pipeline ["causallm", "custom"] --dataset_name  --dataset_config  --dataset_format  --push_model_dir `

examples:

- Simple Fine-Tuning

`ovhai job run --gpu 1 --volume llm-datasets@1azgra:/workspace/datasets:ro --volume llm-jobs@1azgra:/workspace/jobs:rwd --env HF_TOKEN=hf_abcdefg ghcr.io/dataesr/llm-finetuning:latest -- uv run main.py --model_name meta-llama/Llama-3.2-1B --dataset_name test.json`

- Fine-tuning and push to hub

`ovhai job run --gpu 1 --volume llm-datasets@1azgra:/workspace/datasets:ro --volume llm-jobs@1azgra:/workspace/jobs:rwd --env HF_TOKEN=hf_abcdefg HF_PUSH_REPO=nanani/nanana ghcr.io/dataesr/llm-finetuning:latest -- uv run main.py --model_name meta-llama/Llama-3.2-1B --dataset_name test.json`

- Push trained model to hub

`ovhai job run --cpu 1 --volume llm-jobs@1azgra:/workspace/jobs:ro --env HF_TOKEN=hf_abcdefg ghcr.io/dataesr/llm-finetuning:latest HF_PUSH_REPO=nanani/nanana -- uv run main.py --mode push --push_model_dir llama-3.2-1b/finetuned`

- Delete all jobs files

`ovhai bucket object delete llm-jobs@1azgra --all --yes`

> Launch inference job on ovh

`ovhai job run  --gpu 1 --volume llm-datasets@1azgra:/workspace/datasets:ro --volume llm-completions@1azgra:/workspace/completions:rwd --env HF_TOKEN=  ghcr.io/dataesr/llm-inference:latest -- uv run main.py --model_name  --dataset_name  --dataset_split  --dataset_config `

> Launch inference app on ovh

`ovhai app run --gpu 1 --env HF_TOKEN= --env MODEL_NAME= --default-http-port 8000 --unsecure-http ghcr.io/dataesr/llm-inference-app:latest`

> Environment variables

General

- HF_TOKEN            = HuggingFace token

- HF_PUSH_REPO        = HuggingFace repository (will try to push finetuned model to repo if defined)

- MLFLOW_TRACKING_URI = Mlflow tracking url (mlflow tracking disabled if not set)

- MLFLOW_RUN_NAME     = Mlflow tracking run name

- MLFLOW_RUN_NAME_TAG = Suffix for auto generated run name if MLFLOW_RUN_NAME not set

- MLFLOW_EXPERIMENT_NAME = Mlflow tracking experiment name

- MLFLOW_MODEL_NAME = Name for model logged into Mlflow

Training params (finetuning)

- MAX_SEQ_LENGTH, NUM_TRAIN_EPOCHS, MAX_STEPS, BATCH_SIZE, GRAD_ACC_STEPS, OPTIM, LEARNING_RATE, LR_SCHEDULER, WEIGHT_DECAY, MAX_GRAD_NORM, WARMUP_RATIO, SAVE_STEPS, LOG_STEPS

- LORA_R, LORA_ALPHA, LORA_DROPOUT, TASK_TYPE

- BNB_4BIT, BNB_QUANT_TYPE, BNB_DOUBLE_QUANT, BNB_COMPUTE_DTYPE