Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tunib-ai/oslo
OSLO: Open Source framework for Large-scale model Optimization
https://github.com/tunib-ai/oslo
Last synced: 8 days ago
JSON representation
OSLO: Open Source framework for Large-scale model Optimization
- Host: GitHub
- URL: https://github.com/tunib-ai/oslo
- Owner: tunib-ai
- License: other
- Archived: true
- Created: 2021-12-21T05:04:48.000Z (almost 3 years ago)
- Default Branch: master
- Last Pushed: 2022-08-25T19:28:59.000Z (about 2 years ago)
- Last Synced: 2024-10-31T04:21:14.957Z (12 days ago)
- Language: Python
- Homepage: https://tunib-ai.github.io/oslo
- Size: 5.15 MB
- Stars: 306
- Watchers: 20
- Forks: 30
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE.3rd-party-library
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
- awesome-huggingface - OSLO - A library that supports various features to help you train large-scale models. (🌗 Model Scalability)
README
## O S L O
**O**pen **S**ource framework for **L**arge-scale transformer **O**ptimization
### What's New:
* December 30, 2021 [Add Deployment Launcher](https://github.com/tunib-ai/oslo/releases/tag/v1.0).
* December 21, 2021 [Released OSLO 1.0](https://github.com/tunib-ai/oslo/releases/tag/v1.0).
## What is OSLO about?
OSLO is a framework that provides various GPU based optimization technologies for large-scale modeling. 3D Parallelism and Kernel Fusion which could be useful when training a large model like [EleutherAI/gpt-j-6B](https://huggingface.co/EleutherAI/gpt-j-6B) are the key features. OSLO makes these technologies easy-to-use by magical compatibility with [Hugging Face Transformers](https://github.com/huggingface/transformers) that is being considered as a de facto standard in 2021. Currently, the architectures such as GPT2, GPTNeo, and GPTJ are supported, but we plan to support more soon.
## Installation
OSLO can be easily installed using the pip package manager.
All the dependencies such as [torch](https://pypi.org/project/torch/), [transformers](https://pypi.org/project/transformers/), [dacite](https://pypi.org/project/dacite/),
[ninja](https://pypi.org/project/ninja/) and [pybind11](https://pypi.org/project/pybind11/) should be installed automatically with the following command.
Be careful that the 'core' is in the PyPI project name.
```console
pip install oslo-core
```
Some of features rely on the C++ language.
So we provide an option, `CPP_AVAILABLE`, to decide whether or not you install them.
- If the C++ is available:
```console
CPP_AVAILABLE=1 pip install oslo-core
```
- If the C++ is not available:
```console
CPP_AVAILABLE=0 pip install oslo-core
```
Note that the default value of `CPP_AVAILABLE` is 0 in Windows and 1 in Linux.
## Key Features
```python
import deepspeed
from oslo import GPTJForCausalLM
# 1. 3D Parallelism
model = GPTJForCausalLM.from_pretrained_with_parallel(
"EleutherAI/gpt-j-6B", tensor_parallel_size=2, pipeline_parallel_size=2,
)
# 2. Kernel Fusion
model = model.fuse()
# 3. DeepSpeed Support
engines = deepspeed.initialize(
model=model.gpu_modules(), model_parameters=model.gpu_parameters(), ...,
)
# 4. Data Processing
from oslo import (
DatasetPreprocessor,
DatasetBlender,
DatasetForCausalLM,
...
)
# 5. Deployment Launcher
model = GPTJForCausalLM.from_pretrained_with_parallel(..., deployment=True)
```
OSLO offers the following features.
- **3D Parallelism**: The state-of-the-art technique for training a large-scale model with multiple GPUs.
- **Kernel Fusion**: A GPU optimization method to increase training and inference speed.
- **DeepSpeed Support**: We support [DeepSpeed](https://github.com/microsoft/DeepSpeed) which provides ZeRO data parallelism.
- **Data Processing**: Various utilities for efficient large-scale data processing.
- **Deployment Launcher**: A launcher for easily deploying a parallelized model to the web server.
See [USAGE.md](USAGE.md) to learn how to use them.
## Administrative Notes
### Citing OSLO
If you find our work useful, please consider citing:
```
@misc{oslo,
author = {Ko, Hyunwoong and Kim, Soohwan and Park, Kyubyong},
title = {OSLO: Open Source framework for Large-scale transformer Optimization},
howpublished = {\url{https://github.com/tunib-ai/oslo}},
year = {2021},
}
```
### Licensing
The Code of the OSLO project is licensed under the terms of the [Apache License 2.0](LICENSE.apache-2.0).
Copyright 2021 TUNiB Inc. http://www.tunib.ai All Rights Reserved.
### Acknowledgements
The OSLO project is built with GPU support from the [AICA (Artificial Intelligence Industry Cluster Agency)](http://www.aica-gj.kr).