https://github.com/dvgodoy/finetuningllms101_odsc_europe2024

https://github.com/dvgodoy/finetuningllms101_odsc_europe2024

Last synced: about 1 month ago
JSON representation

• Host: GitHub
• URL: https://github.com/dvgodoy/finetuningllms101_odsc_europe2024
• Owner: dvgodoy
• License: mit
• Created: 2024-08-17T16:14:44.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-09-06T17:29:25.000Z (4 months ago)
• Last Synced: 2024-10-14T00:52:47.712Z (3 months ago)
• Language: Jupyter Notebook
• Size: 3.52 MB
• Stars: 1
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Fine-Tuning LLMs 101 ODSC Europe 2024

In this short workshop, you'll get to fine-tune a language model on a custom dataset. We'll cover the main challenges and the building blocks of the fine-tuning procedure: model quantization, parameter-efficient fine-tuning (PEFT) and low-rank adapters (LoRA), chat templates and dataset formatting, and training arguments such as gradient checkpointing, gradient accumulation, sequence length, and optimizers. We'll use Google Colab, BitsAndBytes, and several Hugging Face libraries (peft, datasets, and transformers).

[Open in Colab](https://colab.research.google.com/github/dvgodoy/FineTuningLLMs101_ODSC_Europe2024/blob/main/FineTuningLLMs101.ipynb)