https://github.com/0xzee/finetune_model_unsloth
⚙️ Fine-Tune 🦙 Llama 3.1, Phi-3.. Models on custom DataSet using 🕴️ unsloth & Saving to HuggingFace Hub
https://github.com/0xzee/finetune_model_unsloth
colab-notebook finetuned-model huggingface huggingface-models huggingface-transformers llama3-8b llama3-finetune training unsloth
Last synced: about 2 months ago
JSON representation
⚙️ Fine-Tune 🦙 Llama 3.1, Phi-3.. Models on custom DataSet using 🕴️ unsloth & Saving to HuggingFace Hub
- Host: GitHub
- URL: https://github.com/0xzee/finetune_model_unsloth
- Owner: 0xZee
- Created: 2024-07-25T21:27:58.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2024-07-27T20:17:35.000Z (10 months ago)
- Last Synced: 2025-02-01T20:09:38.738Z (4 months ago)
- Topics: colab-notebook, finetuned-model, huggingface, huggingface-models, huggingface-transformers, llama3-8b, llama3-finetune, training, unsloth
- Language: Jupyter Notebook
- Homepage: https://huggingface.co/0xZee/llama3-1_0xZee_model
- Size: 55.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# 🕴️ finetune `LLM` 🦙 model on custom `DataSet`, using `unsloth`
## 🕴️ ONESTEP CODE : finetune.py 🦙
1. Prepare `DataSet` to match the `unsloth` format :
> [process_dataset_to_unsloth.ipynb](process_dataset_to_unsloth.ipynb)
```bash
Process Dataset -> dataset_to_unsloth.ipynb
```
2. Adapt the `config arguments` : `finetune.py` in finetune.md
> [finetune.md](finetune.md)
```python
def finetune(
# -- PARAMETERS CONFIG --
SOURCE_MODEL = "unsloth/Phi-3-mini-4k-instruct",
DATASET = "0xZee/arxiv-math-Unsloth-tune-50k",
#DATASET = "ArtifactAI/arxiv-math-instruct-50k",
MAX_STEPS = 444,
FINETUNED_LOCAL_MODEL = "Phi-3-mini_ft_arxiv-math",
FINETUNED_ONLINE_MODEL = "0xZee/Phi-3-mini_ft_arxiv-math",
TEST_PROMPT = "Which compound is antiferromagnetic?", # response : common magnetic ordering in various materials.
):
```
3. Run the onestep file : `finetune.py` in finetune.md
> [finetune.md](finetune.md)
```bash
python finetune.py
```
---
## 🕴️ finetune llama3.1 🦙 model on custom DataSet
- 🏬 FineTunning Framework : `Unsloth` on `GPU Tesla T4`
- 🦙 Source Model : `models--unsloth--meta-llama-3.1-8b-bnb-4bit` Model 🕴️
- 💾 Training DataSet ; "yahma/alpaca-cleaned" on HuggingFace
- ⚙️ Fine-Tuned Model : 🕴️ `llama3-1_0xZee_model`
- Model saved to : https://huggingface.co/0xZee/llama3-1_0xZee_model