https://github.com/Lightning-AI/lightning-colossalai

Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
https://github.com/Lightning-AI/lightning-colossalai

lightning python pytorch

Last synced: 2 months ago
JSON representation

Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI

• Host: GitHub
• URL: https://github.com/Lightning-AI/lightning-colossalai
• Owner: Lightning-Universe
• License: apache-2.0
• Archived: true
• Created: 2023-02-14T22:42:00.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2023-09-01T17:22:21.000Z (over 1 year ago)
• Last Synced: 2024-05-17T03:12:13.214Z (7 months ago)
• Topics: lightning, python, pytorch
• Language: Python
• Homepage:
• Size: 1.09 MB
• Stars: 58
• Watchers: 13
• Forks: 2
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE
  • Codeowners: .github/CODEOWNERS

Awesome Lists containing this project

• awesome-llm-and-aigc - Lightning-AI/lightning-colossalai - AI/lightning-colossalai?style=social"/> : Efficient Large-Scale Distributed Training with [Colossal-AI](https://colossalai.org/) and [Lightning AI](https://lightning.ai/). (Summary)
• awesome-llm-and-aigc - Lightning-AI/lightning-colossalai - AI/lightning-colossalai?style=social"/> : Efficient Large-Scale Distributed Training with [Colossal-AI](https://colossalai.org/) and [Lightning AI](https://lightning.ai/). (Summary)

README

        
# Lightning ⚡ Colossal-AI

**Efficient Large-Scale Distributed Training with [Colossal-AI](https://colossalai.org/) and [Lightning AI](https://lightning.ai)**

[![lightning](https://img.shields.io/badge/-Lightning_2.0+-792ee5?logo=pytorchlightning&logoColor=white)](https://lightning.ai/)

[![PyPI Status](https://badge.fury.io/py/lightning-colossalai.svg)](https://badge.fury.io/py/lightning-colossalai)

[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/lightning-colossalai)](https://pypi.org/project/lightning-colossalai/)

[![PyPI - Downloads](https://img.shields.io/pypi/dm/lightning-ColossalAI)](https://pepy.tech/project/lightning-colossalai)

[![Deploy Docs](https://github.com/Lightning-AI/lightning-ColossalAI/actions/workflows/docs-deploy.yml/badge.svg)](https://lightning-ai.github.io/lightning-ColossalAI/)

[![General checks](https://github.com/Lightning-Universe/lightning-ColossalAI/actions/workflows/ci-checks.yml/badge.svg?event=push)](https://github.com/Lightning-Universe/lightning-ColossalAI/actions/workflows/ci-checks.yml)

[![CI testing](https://github.com/Lightning-Universe/lightning-ColossalAI/actions/workflows/ci-testing.yml/badge.svg?event=push)](https://github.com/Lightning-Universe/lightning-ColossalAI/actions/workflows/ci-testing.yml)

[![Build Status](https://dev.azure.com/Lightning-AI/compatibility/_apis/build/status%2Fstrategies%2FLightning-Universe.lightning-ColossalAI?branchName=main)](https://dev.azure.com/Lightning-AI/compatibility/_build/latest?definitionId=66&branchName=main)

[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/Lightning-Universe/lightning-ColossalAI/main.svg)](https://results.pre-commit.ci/latest/github/Lightning-Universe/lightning-ColossalAI/main)

______________________________________________________________________

## Installation

```bash

pip install -U lightning-colossalai

```

## Usage

Simply set the strategy argument in the Trainer:

```py

import lightning as L

trainer = L.Trainer(strategy="colossalai", precision="16-mixed", devices=...)

```

For more fine-grained tuning of Colossal-AI's parameters, pass the strategy object to the Trainer:

```py

import lightning as L

from lightning_colossalai import ColossalAIStrategy

strategy = ColossalAIStrategy(...)

trainer = L.Trainer(strategy=strategy, precision="16-mixed", devices=...)

```

Find all configuration options [in the docs](https://lightning.ai/docs/pytorch/latest/advanced/third_party/colossalai.html)!