https://github.com/gindachen/ring-attention-pytorch

https://github.com/gindachen/ring-attention-pytorch

Last synced: 30 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/gindachen/ring-attention-pytorch
• Owner: GindaChen
• License: mit
• Created: 2024-08-15T09:03:05.000Z (6 months ago)
• Default Branch: feat-add-llama
• Last Pushed: 2024-08-15T16:52:11.000Z (6 months ago)
• Last Synced: 2024-08-15T19:02:54.131Z (6 months ago)
• Language: Python
• Size: 1.03 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.env-setup.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Setup Environment

Setup environment 

```bash

mamba create -p ./env python=3.10

mamba activate ./env

mamba install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia

pip install -e .

pip install vllm 'torch==2.4.0'

```

Test 

```bash

MASTER_ADDR=localhost MASTER_PORT=51234 torchrun --nproc_per_node=8 test_llama.py --tp_size 1 --sp_size 8 --seq_len $((2 ** 10 * 64)) --model_name "meta-llama/Meta-Llama-3-8B"

```