https://github.com/sovit-123/lm_sft
Various LMs/LLMs below 3B parameters (for now) trained using SFT (Supervised Fine Tuning) for several downstream tasks
https://github.com/sovit-123/lm_sft
gemma gpt gpt2 large-language-models llms supervised-finetuning
Last synced: 4 months ago
JSON representation
Various LMs/LLMs below 3B parameters (for now) trained using SFT (Supervised Fine Tuning) for several downstream tasks
- Host: GitHub
- URL: https://github.com/sovit-123/lm_sft
- Owner: sovit-123
- Created: 2024-03-03T15:31:31.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2024-04-13T18:26:05.000Z (about 2 years ago)
- Last Synced: 2024-04-13T20:55:03.066Z (about 2 years ago)
- Topics: gemma, gpt, gpt2, large-language-models, llms, supervised-finetuning
- Language: Jupyter Notebook
- Homepage:
- Size: 1.19 MB
- Stars: 2
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# README
Various LMs/LLMs below 3B parameters (for now) trained using SFT (Supervised Fine Tuning) for several downstream tasks like:
* Following instructions (the general instruction fine tuning)
* Question Answering
* Summarization
* Headline generation
* Sentiment analysis
* Text classification
* Language translation
* Code generation
* And more coming...
## Setup
* Create a Conda environment:
```
conda create -n env_name python=3.11
```
* Installing `packaging`:
```
pip install packaging
```
* Install PyTorch (you can install the version of your choice from [here](https://pytorch.org/get-started/locally/))
```
conda install pytorch==2.2.0 pytorch-cuda=12.1 -c pytorch -c nvidia
```
* Install rest of the requirements:
```
pip install -r requirements.txt
```