https://github.com/azure/ms-amp

Microsoft Automatic Mixed Precision Library
https://github.com/azure/ms-amp

amp deep-learning fp8 gpu mixed-precision pytorch transformer

Last synced: about 2 hours ago
JSON representation

Microsoft Automatic Mixed Precision Library

• Host: GitHub
• URL: https://github.com/azure/ms-amp
• Owner: Azure
• License: mit
• Created: 2023-01-30T09:29:17.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-09-29T07:39:18.000Z (3 months ago)
• Last Synced: 2025-01-11T07:06:32.644Z (about 2 hours ago)
• Topics: amp, deep-learning, fp8, gpu, mixed-precision, pytorch, transformer
• Language: Python
• Homepage: https://azure.github.io/MS-AMP/
• Size: 1.85 MB
• Stars: 542
• Watchers: 11
• Forks: 45
• Open Issues: 22
• Metadata Files:
  • Readme: README.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Security: SECURITY.md
  • Support: SUPPORT.md

Awesome Lists containing this project

README

        
# MS-AMP: Microsoft Automatic Mixed Precision

__MS-AMP__ is an automatic mixed precision package for deep learning developed by Microsoft.

📢 [v0.4.0](https://github.com/Azure/MS-AMP/releases/tag/v0.4.0) has been released!

## _Check [aka.ms/msamp/doc](https://aka.ms/msamp/doc) for more details._

## Publication

FP8-LM: Training FP8 Large Language Models [bib]

```bibtex

@misc{fp8lm,

      title={FP8-LM: Training FP8 Large Language Models},

      author={Houwen Peng and Kan Wu and Yixuan Wei and Guoshuai Zhao and Yuxiang Yang and Ze Liu and Yifan Xiong and Ziyue Yang and Bolin Ni and Jingcheng Hu and Ruihang Li and Miaosen Zhang and Chen Li and Jia Ning and Ruizhe Wang and Zheng Zhang and Shuguang Liu and Joe Chau and Han Hu and Peng Cheng},

      year={2023},

      eprint={2310.18313},

      archivePrefix={arXiv},

      primaryClass={cs.LG}

}

```

## Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft

trademarks or logos is subject to and must follow

[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).

Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.

Any use of third-party trademarks or logos are subject to those third-party's policies.