https://github.com/arjuntheprogrammer/finetune_codellama_alpaca20k

https://github.com/arjuntheprogrammer/finetune_codellama_alpaca20k

Last synced: about 1 year ago
JSON representation

• Host: GitHub
• URL: https://github.com/arjuntheprogrammer/finetune_codellama_alpaca20k
• Owner: arjuntheprogrammer
• Created: 2024-05-31T10:47:54.000Z (almost 2 years ago)
• Default Branch: master
• Last Pushed: 2024-05-31T14:14:55.000Z (almost 2 years ago)
• Last Synced: 2025-01-26T20:29:22.506Z (about 1 year ago)
• Language: Jupyter Notebook
• Size: 7.97 MB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# CODE LLAMA TRAINING

- GPU TYPE: A100 80GB

- VRAM USED: 11.6GB

- Training Dataset: HuggingFaceH4/CodeAlpaca_20K

- Dataset Link: 

- Dataset Size: 20K rows (train: 18K, test: 2K)

- Pretrained Model: codellama/CodeLlama-7b-hf

- Pretrained Model Link: 

- Finetuning Training Time: 1 hour 47 mins

- HF Uploaded Model: 

- Wandb Logs Link: 

## DATASET EXAMPLE

Prompt: `Use a loop in Java to print out the squares of numbers from 1 to 10.`

Completion: `for (int i = 1; i <= 10; i++) { System.out.println(i * i); }`

## TRAINING LOGS

![alt text](image.png)

![alt text](image-1.png)

![alt text](image-2.png)

![alt text](image-3.png)

---

## YT RECORDING

[![CodeLlama Finetuning](https://img.youtube.com/vi/s6uqbqxXXao/0.jpg)](https://youtu.be/s6uqbqxXXao)

---