Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/NVIDIA/nccl

Optimized primitives for collective multi-GPU communication
https://github.com/NVIDIA/nccl

Last synced: 22 days ago
JSON representation

Optimized primitives for collective multi-GPU communication

Host: GitHub
URL: https://github.com/NVIDIA/nccl
Owner: NVIDIA
License: other
Created: 2015-11-14T00:12:04.000Z (over 8 years ago)
Default Branch: master
Last Pushed: 2024-04-04T15:49:45.000Z (about 2 months ago)
Last Synced: 2024-04-04T17:11:00.389Z (about 2 months ago)
Language: C++
Size: 2.65 MB
Stars: 2,783
Watchers: 147
Forks: 723
Open Issues: 521
Metadata Files:
- Readme: README.md
- License: LICENSE.txt

Lists

awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
fucking-awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp-completed - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp-completed - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-from-stars - NVIDIA/nccl - GPU communication (HarmonyOS / Windows Manager)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cpp - NCCL - Optimized primitives for collective multi-GPU communication. [BSD] (Concurrency)
awesome-cuda-tensorrt-fpga - NVIDIA/nccl - GPU communication. (Frameworks)
my-awesome-stars - NVIDIA/nccl - Optimized primitives for collective multi-GPU communication (C++)
awesome-stars - NVIDIA/nccl - Optimized primitives for collective multi-GPU communication (C++)

README

# NCCL

Optimized primitives for inter-GPU communication.

## Introduction

NCCL (pronounced "Nickel") is a stand-alone library of standard communication routines for GPUs, implementing all-reduce, all-gather, reduce, broadcast, reduce-scatter, as well as any send/receive based communication pattern. It has been optimized to achieve high bandwidth on platforms using PCIe, NVLink, NVswitch, as well as networking using InfiniBand Verbs or TCP/IP sockets. NCCL supports an arbitrary number of GPUs installed in a single node or across multiple nodes, and can be used in either single- or multi-process (e.g., MPI) applications.

For more information on NCCL usage, please refer to the [NCCL documentation](https://docs.nvidia.com/deeplearning/sdk/nccl-developer-guide/index.html).

## Build

Note: the official and tested builds of NCCL can be downloaded from: https://developer.nvidia.com/nccl. You can skip the following build steps if you choose to use the official builds.

To build the library :

```shell
$ cd nccl
$ make -j src.build
```

If CUDA is not installed in the default /usr/local/cuda path, you can define the CUDA path with :

```shell
$ make src.build CUDA_HOME=
```

NCCL will be compiled and installed in `build/` unless `BUILDDIR` is set.

By default, NCCL is compiled for all supported architectures. To accelerate the compilation and reduce the binary size, consider redefining `NVCC_GENCODE` (defined in `makefiles/common.mk`) to only include the architecture of the target platform :
```shell
$ make -j src.build NVCC_GENCODE="-gencode=arch=compute_70,code=sm_70"
```

## Install

To install NCCL on the system, create a package then install it as root.

Debian/Ubuntu :
```shell
$ # Install tools to create debian packages
$ sudo apt install build-essential devscripts debhelper fakeroot
$ # Build NCCL deb package
$ make pkg.debian.build
$ ls build/pkg/deb/
```

RedHat/CentOS :
```shell
$ # Install tools to create rpm packages
$ sudo yum install rpm-build rpmdevtools
$ # Build NCCL rpm package
$ make pkg.redhat.build
$ ls build/pkg/rpm/
```

OS-agnostic tarball :
```shell
$ make pkg.txz.build
$ ls build/pkg/txz/
```

## Tests

Tests for NCCL are maintained separately at https://github.com/nvidia/nccl-tests.

```shell
$ git clone https://github.com/NVIDIA/nccl-tests.git
$ cd nccl-tests
$ make
$ ./build/all_reduce_perf -b 8 -e 256M -f 2 -g
```

## Copyright