{"id":20725558,"url":"https://github.com/oneapi-src/distributed-ranges","last_synced_at":"2025-04-23T18:08:45.903Z","repository":{"id":65472531,"uuid":"578767219","full_name":"oneapi-src/distributed-ranges","owner":"oneapi-src","description":"Distributed ranges is a generalization of C++ ranges for distributed data structures.","archived":false,"fork":false,"pushed_at":"2025-04-07T14:28:04.000Z","size":8942,"stargazers_count":50,"open_issues_count":77,"forks_count":17,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-23T18:08:37.083Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oneapi-src.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSES/BSD-3-Clause.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-12-15T20:48:21.000Z","updated_at":"2025-04-18T21:39:07.000Z","dependencies_parsed_at":"2023-12-20T16:11:46.618Z","dependency_job_id":null,"html_url":"https://github.com/oneapi-src/distributed-ranges","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fdistributed-ranges","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fdistributed-ranges/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fdistributed-ranges/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fdistributed-ranges/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oneapi-src","download_url":"https://codeload.github.com/oneapi-src/distributed-ranges/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250487528,"owners_count":21438612,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T04:19:22.440Z","updated_at":"2025-04-23T18:08:45.893Z","avatar_url":"https://github.com/oneapi-src.png","language":"C++","readme":".. SPDX-FileCopyrightText: Intel Corporation\n..\n.. SPDX-License-Identifier: BSD-3-Clause\n\n===================\n Distributed Ranges\n===================\n\n.. image:: https://github.com/oneapi-src/distributed-ranges/actions/workflows/pr.yml/badge.svg\n   :target: https://github.com/oneapi-src/distributed-ranges/actions/workflows/pr.yml\n.. image:: https://www.bestpractices.dev/projects/8975/badge\n   :target: https://www.bestpractices.dev/projects/8975\n\nProductivity library for distributed and partitioned memory based on\nC++ Ranges.\n\nAbout\n-----\n\nDistributed Ranges is a C++ productivity library for distributed and partitioned memory based on C++ ranges.\nIt offers a collection of data structures, views, and algorithms for building generic abstractions\nand provides interoperability with MPI, SHMEM, SYCL and OpenMP and portability on CPUs and GPUs.\nNUMA-aware allocators and distributed data structures facilitate development of C++ applications\non heterogeneous nodes with multiple devices and achieve excellent performance and parallel scalability\nby exploiting local compute and data access.\n\nMain strength of the library\n============================\n\nIn this model one can:\n\n* create a `distributed data structure` that work with all our algorithms out of the box\n* create an `algorithm` that works with all our distributed data structures out of the box\n\nDistributed Ranges is a `glue` that makes this possible.\n\n\nDocumentation\n-------------\n\n* Usage:\n\n  * Introductory presentation: `Distributed Ranges, why you need it`_, 2024\n  * Article: `Get Started with Distributed Ranges`_, 2023\n  * Tutorial: `Distributed Ranges Tutorial`_\n\n* Design / Implementation:\n\n  * Conference paper: `Distributed Ranges, A Model for Distributed Data Structures, Algorithms, and Views`_, 2024\n  * Talk: `CppCon 2023; Benjamin Brock; Distributed Ranges`_, 2023\n  * Technical presentation: `Intel Innovation'23`_, 2023\n  * `API specification`_\n\n\nRequirements\n------------\n\n* Linux\n* cmake \u003e=3.20\n* `OneAPI HPC Toolkit`_ installed\n\nEnable `OneAPI` by::\n\n  source ~/intel/oneapi/setvars.sh\n\n... or by::\n\n  source /opt/intel/oneapi/setvars.sh\n\n... or wherever you have ``oneapi/setvars.sh`` script installed in your system.\n\nAdditional requirements for NVIDIA GPUs\n=======================================\n\n* `CUDA`_\n* `OneAPI for NVIDIA GPUs`_ plugin\n\nWhen enabling OneAPI use ``--include-intel-llvm`` option, e.g. call::\n\n  source ~/intel/oneapi/setvars.sh --include-intel-llvm\n\n... instead of ``source ~/intel/oneapi/setvars.sh``.\n\n\nBuild and run\n-------------\n\nBuild for Intel GPU/CPU\n=======================\n\nAll tests and examples can be build by::\n\n  CXX=icpx cmake -B build\n  cmake --build build -- -j\n\n\nBuild for NVidia GPU\n====================\n\n.. note::\n\n  Distributed Ranges library works in two models:\n   - Multi Process (based on SYCL and MPI)\n   - Single Process (based on pure SYCL)\n\n  On NVIDIA GPU only `Multi Process` model is currently supported.\n\nTo build multi-process tests call::\n\n  CXX=icpx cmake -B build -DENABLE_CUDA:BOOL=ON\n  cmake --build build --target mp-all-tests -- -j\n\n\nRun tests\n=========\n\nRun multi process tests::\n\n  ctest --test-dir build --output-on-failure -L MP -j 4\n\nRun single process tests::\n\n  ctest --test-dir build --output-on-failure -L SP -j 4\n\nRun all tests::\n\n  ctest --test-dir build --output-on-failure -L TESTLABEL -j 4\n\nRun benchmarks\n==============\n\nTwo binaries are build for benchmarks:\n\n* mp-bench - for benchmarking `Multi-Process` model\n* sp-bench - for benchmarking `Single-Process` model\n\nHere are examples of running single benchmarks.\n\nRunning `GemvEq_DR` strong scaling benchmark in Multi-Process model using two GPUs::\n\n  ONEAPI_DEVICE_SELECTOR='level_zero:gpu' I_MPI_OFFLOAD=1 I_MPI_OFFLOAD_CELL_LIST=0-11 \\\n  mpiexec -n 2 -ppn 2  build/benchmarks/gbench/mp/mp-bench --vector-size 1000000000 --reps 50 \\\n  --v=3 --benchmark_out=mp_gemv.txt --benchmark_filter=GemvEq_DR/ --sycl\n\nRunning `Exclusive_Scan_DR` weak scaling in Single-Process model using two GPUs::\n\n  ONEAPI_DEVICE_SELECTOR='level_zero:gpu' KMP_AFFINITY=compact \\\n  build/benchmarks/gbench/sp/sp-bench --vector-size 1000000000 --reps 50 \\\n  --v=3 --benchmark_out=sp_exclscan.txt --benchmark_filter=Exclusive_Scan_DR/ \\\n  --weak-scaling --device-memory --num-devices 2\n\n\nCheck all options::\n\n  ./build/benchmarks/gbench/mp/mp-bench --help  # see google test options help\n  ./build/benchmarks/gbench/mp/mp-bench --drhelp  # see DR specific options\n\n\n\nExamples\n--------\n\nSee `Distributed Ranges Tutorial`_ for a few well explained examples.\n\nAdding Distributed Ranges to your project\n-----------------------------------------\n\nIf your project uses CMAKE, add the following to your\n``CMakeLists.txt`` to download the library::\n\n  find_package(MPI REQUIRED)\n  include(FetchContent)\n  FetchContent_Declare(\n    dr\n    GIT_REPOSITORY https://github.com/oneapi-src/distributed-ranges.git\n    GIT_TAG main\n    )\n  FetchContent_MakeAvailable(dr)\n\nThe above will define targets that can be included in your project::\n\n  target_link_libraries(\u003capplication\u003e MPI::MPI_CXX DR::mpi)\n\nSee `Distributed Ranges Tutorial`_\nfor a live example of a cmake project that imports and uses Distributed Ranges.\n\nLogging\n-------\n\nAdd below code to your ``main`` function to enable logging.\n\nIf using `Single-Process` model::\n\n  std::ofstream logfile(\"dr.log\");\n  dr::drlog.set_file(logfile);\n\nIf using `Multi-Process` model::\n\n  int my_mpi_rank;\n  MPI_Comm_rank(MPI_COMM_WORLD, \u0026my_mpi_rank);\n  std::ofstream logfile(fmt::format(\"dr.{}.log\", my_mpi_rank));\n\nExample of adding custom log statement to your code::\n\n  DRLOG(\"my debug message with varA:{} and varB:{}\", a, b);\n\n\nContact us\n----------\n\nContact us by writing a `new issue`_.\n\nWe seek collaboration opportunities and welcome feedback on ways to extend the library,\naccording to developer needs.\n\n\nSee also\n--------\n\n* `CONTRIBUTING`_\n* `Fuzz Testing`_\n* `Spec Editing`_ - Editing the API document\n* `Print Type`_ - Print types at compile time:\n* `Testing`_ - Test system maintenance\n* `Security`_ - Security policy\n* `Doxygen`_\n\n.. _`Security`: SECURITY.md\n.. _`Testing`: doc/developer/testing\n.. _`Spec Editing`: doc/spec/README.rst\n.. _`Fuzz Testing`: test/fuzz/README.rst\n.. _`Print Type`: https://stackoverflow.com/a/14617848/2525421\n.. _`CONTRIBUTING`: CONTRIBUTING.md\n.. _`Distributed Ranges, why you need it`: https://github.com/oneapi-src/distributed-ranges/blob/main/doc/presentations/Distributed%20Ranges%2C%20why%20you%20need%20it.pdf\n.. _`Get Started with Distributed Ranges`: https://www.intel.com/content/www/us/en/developer/articles/guide/get-started-with-distributed-ranges.html\n.. _`Distributed Ranges Tutorial`: https://github.com/oneapi-src/distributed-ranges-tutorial\n.. _`Distributed Ranges, A Model for Distributed Data Structures, Algorithms, and Views`: https://dl.acm.org/doi/10.1145/3650200.3656632\n.. _`CppCon 2023; Benjamin Brock; Distributed Ranges`: https://www.youtube.com/watch?v=X_dlJcV21YI\n.. _`Intel Innovation'23`: https://github.com/oneapi-src/distributed-ranges/blob/main/doc/presentations/Distributed%20Ranges.pdf\n.. _`API specification`: https://oneapi-src.github.io/distributed-ranges/spec/\n.. _`Doxygen`: https://oneapi-src.github.io/distributed-ranges/doxygen/\n.. _`new issue`: issues/new\n.. _`OneAPI HPC Toolkit`: https://www.intel.com/content/www/us/en/developer/tools/oneapi/hpc-toolkit-download.html\n.. _`OneAPI for NVIDIA GPUs`: https://developer.codeplay.com/products/oneapi/nvidia/home/\n.. _`CUDA`: https://developer.nvidia.com/cuda-toolkit\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Fdistributed-ranges","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foneapi-src%2Fdistributed-ranges","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Fdistributed-ranges/lists"}