https://github.com/hasibzunair/generative-models-tutorial

Tutorial materials for Ericsson AI & ML Upskill Program 2023-2024.
https://github.com/hasibzunair/generative-models-tutorial

Last synced: about 2 months ago
JSON representation

Tutorial materials for Ericsson AI & ML Upskill Program 2023-2024.

• Host: GitHub
• URL: https://github.com/hasibzunair/generative-models-tutorial
• Owner: hasibzunair
• Created: 2023-10-04T00:27:53.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-06-23T04:46:02.000Z (6 months ago)
• Last Synced: 2024-06-24T04:33:06.366Z (6 months ago)
• Language: Jupyter Notebook
• Homepage:
• Size: 395 KB
• Stars: 0
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Building and Applying Generative Models using PyTorch

Tutorial materials for **Ericsson AI & ML Upskill Program 2023-2024**. It contains code for finetuning a GPT-like large language model (LLM) in PyTorch. 

Materials for the previous program can be found [Ericsson AI & ML Upskill Program 2021](https://github.com/hasibzunair/intro-ml-tutorial).

### Learning resources

* [Intro to Large Language Models](https://youtu.be/zjkBMFhNj_g?si=bIVqXiQratJdqvhj)

* [Understanding Large Language Models](https://magazine.sebastianraschka.com/p/understanding-large-language-models)

* [QLoRA is all you need (Fast and lightweight model fine-tuning)](https://youtu.be/J_3hDqSvpmg?si=b0j8O2puV42z18UL)

* [Parameter-efficient Finetuning with LoRA](https://github.com/rasbt/LLMs-from-scratch/blob/main/appendix-E/01_main-chapter-code/appendix-E.ipynb)

* [QLoRA: Efficient Finetuning of Quantized LLMs](https://arxiv.org/abs/2305.14314)

* [Run Mixtral-8x7B models in Colab or consumer desktops](https://github.com/dvmazur/mixtral-offloading)

### Acknowledgements

This repository was built using [Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step](https://github.com/rasbt/LLMs-from-scratch).