{"id":24271481,"url":"https://github.com/uxlfoundation/oneccl","last_synced_at":"2025-05-16T10:07:51.116Z","repository":{"id":43410740,"uuid":"207412282","full_name":"uxlfoundation/oneCCL","owner":"uxlfoundation","description":"oneAPI Collective Communications Library (oneCCL)","archived":false,"fork":false,"pushed_at":"2025-05-06T18:41:15.000Z","size":229197,"stargazers_count":233,"open_issues_count":15,"forks_count":79,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-05-16T10:07:37.224Z","etag":null,"topics":["ai-machine-learning","ai-training","deep-learning","libraries","oneapi","swrepo","sycl"],"latest_commit_sha":null,"homepage":"https://oneapi-src.github.io/oneCCL","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uxlfoundation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-09-09T21:57:46.000Z","updated_at":"2025-05-13T11:24:50.000Z","dependencies_parsed_at":"2024-01-17T15:35:36.319Z","dependency_job_id":"a1b5e985-b6ae-4a80-a77e-31e73b24ed23","html_url":"https://github.com/uxlfoundation/oneCCL","commit_stats":null,"previous_names":["uxlfoundation/oneccl","oneapi-src/oneccl"],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uxlfoundation%2FoneCCL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uxlfoundation%2FoneCCL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uxlfoundation%2FoneCCL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uxlfoundation%2FoneCCL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uxlfoundation","download_url":"https://codeload.github.com/uxlfoundation/oneCCL/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254509476,"owners_count":22082891,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-machine-learning","ai-training","deep-learning","libraries","oneapi","swrepo","sycl"],"created_at":"2025-01-15T17:30:15.466Z","updated_at":"2025-05-16T10:07:46.106Z","avatar_url":"https://github.com/uxlfoundation.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# oneAPI Collective Communications Library (oneCCL) \u003c!-- omit in toc --\u003e \u003cimg align=\"right\" width=\"200\" height=\"100\" src=\"https://raw.githubusercontent.com/uxlfoundation/artwork/e98f1a7a3d305c582d02c5f532e41487b710d470/foundation/uxl-foundation-logo-horizontal-color.svg\"\u003e\n\n[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/oneapi-src/oneCCL/badge)](https://scorecard.dev/viewer/?uri=github.com/oneapi-src/oneCCL)\n[Installation](#installation)\u0026nbsp;\u0026nbsp;\u0026nbsp;|\u0026nbsp;\u0026nbsp;\u0026nbsp;[Usage](#usage)\u0026nbsp;\u0026nbsp;\u0026nbsp;|\u0026nbsp;\u0026nbsp;\u0026nbsp;[Release Notes](https://www.intel.com/content/www/us/en/developer/articles/release-notes/oneapi-collective-communication-library-ccl-release-notes.html)\u0026nbsp;\u0026nbsp;\u0026nbsp;|\u0026nbsp;\u0026nbsp;\u0026nbsp;[Documentation](https://oneapi-src.github.io/oneCCL/)\u0026nbsp;\u0026nbsp;\u0026nbsp;|\u0026nbsp;\u0026nbsp;\u0026nbsp;[How to Contribute](CONTRIBUTING.md)\u0026nbsp;\u0026nbsp;\u0026nbsp;|\u0026nbsp;\u0026nbsp;\u0026nbsp;[License](LICENSE)\n\noneAPI Collective Communications Library (oneCCL) provides an efficient implementation of communication patterns used in deep learning.\n\noneCCL is integrated into:\n* [Horovod\\*](https://github.com/horovod/horovod) (distributed training framework). Refer to [Horovod with oneCCL](https://github.com/horovod/horovod/blob/master/docs/oneccl.rst) for details.\n* [PyTorch\\*](https://github.com/pytorch/pytorch) (machine learning framework). Refer to [PyTorch bindings for oneCCL](https://github.com/intel/torch-ccl) for details.\n\noneCCL is governed by the [UXL Foundation](http://www.uxlfoundation.org) and is an implementation of the [oneAPI specification](https://spec.oneapi.io).\n\n## Table of Contents \u003c!-- omit in toc --\u003e\n\n- [Prerequisites](#prerequisites)\n- [Installation](#installation)\n- [Usage](#usage)\n  - [Launching Example Application](#launching-example-application)\n    - [Using external MPI](#using-external-mpi)\n  - [Setting workers affinity](#setting-workers-affinity)\n    - [Automatic setup](#automatic-setup)\n    - [Explicit setup](#explicit-setup)\n  - [Using oneCCL package from CMake](#using-oneccl-package-from-cmake)\n    - [oneCCLConfig files generation](#onecclconfig-files-generation)\n- [Additional Resources](#additional-resources)\n  - [Blog Posts](#blog-posts)\n  - [Workshop Materials](#workshop-materials)\n- [Notice of Deprecation](#notice-of-deprecation)\n  \n## Prerequisites \n\nSee [System Requirements](https://www.intel.com/content/www/us/en/developer/articles/system-requirements/oneapi-collective-communication-library-system-requirements.html) to learn about hardware and software requirements before getting started with oneCCL.\n\n## Installation\n\nGeneral installation scenario:\n\n```\ncd oneCCL\nmkdir build\ncd build\ncmake ..\nmake -j install\n```\n\nIf you need a clean build, create a new build directory and invoke `cmake` within it.\n\nYou can also do the following during installation:\n- [Specify installation directory](INSTALL.md#specify-installation-directory)\n- [Specify the compiler](INSTALL.md#specify-the-compiler)\n- [Specify `SYCL` cross-platform abstraction level](INSTALL.md#specify-sycl-cross-platform-abstraction-level)\n- [Specify the build type](INSTALL.md#specify-the-build-type)\n- [Enable `make` verbose output](INSTALL.md#enable-make-verbose-output)\n\n## Usage\n\n### Launching Example Application\n\nUse the command:\n```bash\n$ source \u003cinstall_dir\u003e/env/setvars.sh\n$ mpirun -n 2 \u003cinstall_dir\u003e/examples/benchmark/benchmark\n```\n\n#### Using external mpi\n\nThe ccl-bundled-mpi flag in vars.sh can take values \"yes\" or \"no\" to control if bundled Intel MPI should be used or not. Current default is \"yes\", which means that oneCCL temporarily overrides the mpi implementation in use.\n\nIn order to suppress the behavior and use user-supplied or system-default mpi use the following command *instead* of sourcing `setvars.sh`:\n\n```bash\n$ source \u003cinstall_dir\u003e/env/vars.sh --ccl-bundled-mpi=no\n```\n\nThe mpi implementation will not be overridden. Please note that, in this case, user needs to assure the system finds all required mpi-related binaries.\n\n### Setting workers affinity\n\nThere are two ways to set worker threads (workers) affinity: [automatically](#setting-affinity-automatically) and [explicitly](#setting-affinity-explicitly).\n\n#### Automatic setup\n\n1. Set the `CCL_WORKER_COUNT` environment variable with the desired number of workers per process.\n2. Set the `CCL_WORKER_AFFINITY` environment variable with the value `auto`.\n\nExample:\n```\nexport CCL_WORKER_COUNT=4\nexport CCL_WORKER_AFFINITY=auto\n```\nWith the variables above, oneCCL will create four workers per process and the pinning will depend from process launcher.\n\nIf an application has been launched using `mpirun` that is provided by oneCCL distribution package then workers will be automatically pinned to the last four cores available for the launched process. The exact IDs of CPU cores can be controlled by `mpirun` parameters.\n\nOtherwise, workers will be automatically pinned to the last four cores available on the node.\n\n---\n\n#### Explicit setup\n\n1. Set the `CCL_WORKER_COUNT` environment variable with the desired number of workers per process.\n2. Set the `CCL_WORKER_AFFINITY` environment variable with the IDs of cores to pin local workers.\n\nExample:\n```\nexport CCL_WORKER_COUNT=4\nexport CCL_WORKER_AFFINITY=3,4,5,6\n```\nWith the variables above, oneCCL will create four workers per process and pin them to the cores with the IDs of 3, 4, 5, and 6 respectively.\n\n### Using oneCCL package from CMake\n\n`oneCCLConfig.cmake` and `oneCCLConfigVersion.cmake` are included into oneCCL distribution.\n\nWith these files, you can integrate oneCCL into a user project with the [find_package](https://cmake.org/cmake/help/latest/command/find_package.html) command. Successful invocation of `find_package(oneCCL \u003coptions\u003e)` creates imported target `oneCCL` that can be passed to the [target_link_libraries](https://cmake.org/cmake/help/latest/command/target_link_libraries.html) command.\n\nFor example:\n\n```cmake\nproject(Foo)\nadd_executable(foo foo.cpp)\n\n# Search for oneCCL\nfind_package(oneCCL REQUIRED)\n\n# Connect oneCCL to foo\ntarget_link_libraries(foo oneCCL)\n```\n#### oneCCLConfig files generation\n\nTo generate oneCCLConfig files for oneCCL package, use the provided [`cmake/scripts/config_generation.cmake`](/cmake/scripts/config_generation.cmake) file:\n\n```\ncmake [-DOUTPUT_DIR=\u003coutput_dir\u003e] -P cmake/script/config_generation.cmake\n```\n\n### OS File Descriptors \n\noneCCL uses [Level Zero IPC handles](https://spec.oneapi.io/level-zero/latest/core/PROG.html#memory-1) so that a process can access a memory allocation done by a different process. \nHowever, these IPC handles consume OS File Descriptors (FDs). As a result, to avoid running out of OS FDs, we recommend to increase the default limit of FDs in the system for applications running with oneCCL and GPU buffers. \n\nThe number of FDs required is application-dependent, but the recommended limit is ``1048575``. This value can be modified with the ulimit command.  \n\n## Governance\n\nThe oneCCL project is governed by the UXL Foundation and you can get involved in this project in multiple ways. It is possible to join the [Special Interest Groups (SIG)](https://github.com/uxlfoundation/foundation) meetings where the group discuss and demonstrates work using the foundation projects. Members can also join the Open Source and Specification Working Group meetings.\n\nYou can also join the mailing lists for the [UXL Foundation](https://lists.uxlfoundation.org/g/main/subgroups) to be informed of when meetings are happening and receive the latest information and discussions.\n\n## Additional Resources\n\n### Blog Posts\n\n- [Optimizing DLRM by using PyTorch with oneCCL Backend](https://pytorch.medium.com/optimizing-dlrm-by-using-pytorch-with-oneccl-backend-9f85b8ef6929)\n- [Intel MLSL Makes Distributed Training with MXNet Faster](https://medium.com/apache-mxnet/intel-mlsl-makes-distributed-training-with-mxnet-faster-7186ad245e81)\n\n### Workshop Materials\n\n- oneAPI, oneCCL and OFI: Path to Heterogeneous Architecure Programming with Scalable Collective Communications: [recording](https://www.youtube.com/watch?v=ksiZ90EtP98\u0026feature=youtu.be) and [slides](https://www.openfabrics.org/wp-content/uploads/2020-workshop-presentations/502.-OFA-Virtual-Workshop-2020-oneCCL-v5.pdf)\n\n## Notice of Deprecation \n\n### Deprecation of C++ API\n- In oneCCL version 2021.17 included with the 2025.3 oneAPI release, oneCCL will add support for a new C API that closely follows the NVIDIA Collective Communications Libary (NCCL)* API standard. The existing C++ API will remain available and will remain the default API for the 2021.17 release. Details explaining how an application may link against and use the new API will be shared in this release.\n\nSee the [oneCCL C API RFC document](https://github.com/uxlfoundation/oneCCL/tree/rfcs/rfcs/20240806-c-api) to view the proposed API and provide any feedback. \n\n- In oneCCL version 2022.0 included with the 2026.0 oneAPI release, oneCCL will use the new NCCL like C API by default. This is a breaking change. The legacy C++ API will remain available, and details explaining how an application may link against and use the legacy API will be included in the 2022.0 release.\n\nApplications cannot use both the C and C++ APIs simultaneously.\n\nSupport for the legacy C++ API shall remain in the release until future notice. The schedule for legacy API removal will be announced here.\n\n## Contribute \u003c!-- omit in toc --\u003e\n\nSee [CONTRIBUTING](CONTRIBUTING.md) for more information.\n\n## License \u003c!-- omit in toc --\u003e\n\nDistributed under the Apache License 2.0 license. See [LICENSE](LICENSE) for more\ninformation.\n\n## Security Policy \u003c!-- omit in toc --\u003e\n\nSee [SECURITY](SECURITY.md) for more information.\n\n## Notices and Disclaimers\n\nIntel technologies may require enabled hardware, software or service activation.\n\nNo product or component can be absolutely secure.\n\nYour costs and results may vary.\n\n© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.\n\nNo license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.\n\nThe products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.\n\nIntel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuxlfoundation%2Foneccl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuxlfoundation%2Foneccl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuxlfoundation%2Foneccl/lists"}