{"id":34082269,"url":"https://github.com/pylops/pylops-mpi","last_synced_at":"2026-04-29T06:02:51.284Z","repository":{"id":167513756,"uuid":"638831778","full_name":"PyLops/pylops-mpi","owner":"PyLops","description":"MPI-powered PyLops with MPI4Py","archived":false,"fork":false,"pushed_at":"2026-04-16T08:32:25.000Z","size":121369,"stargazers_count":24,"open_issues_count":5,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-26T18:06:03.461Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://pylops.github.io/pylops-mpi/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PyLops.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-05-10T07:47:18.000Z","updated_at":"2026-04-16T08:28:08.000Z","dependencies_parsed_at":null,"dependency_job_id":"f8618bc9-b1da-4e96-b090-c4621db405a2","html_url":"https://github.com/PyLops/pylops-mpi","commit_stats":null,"previous_names":["pylops/pylops-mpi"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/PyLops/pylops-mpi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PyLops%2Fpylops-mpi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PyLops%2Fpylops-mpi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PyLops%2Fpylops-mpi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PyLops%2Fpylops-mpi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PyLops","download_url":"https://codeload.github.com/PyLops/pylops-mpi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PyLops%2Fpylops-mpi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32412890,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T05:20:56.964Z","status":"ssl_error","status_checked_at":"2026-04-29T05:19:54.749Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-14T12:10:34.618Z","updated_at":"2026-04-29T06:02:51.279Z","avatar_url":"https://github.com/PyLops.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![PyLops-MPI](https://github.com/PyLops/pylops-mpi/blob/main/docs/source/_static/pylopsmpi_b.png)\n\n[![PyPI version](https://badge.fury.io/py/pylops-mpi.svg)](https://badge.fury.io/py/pylops-mpi)\n[![Build status](https://github.com/PyLops/pylops-mpi/actions/workflows/build.yml/badge.svg)](https://github.com/PyLops/pylops-mpi/actions/workflows/build.yml)\n[![Documentation status](https://github.com/PyLops/pylops-mpi/actions/workflows/pages/pages-build-deployment/badge.svg)](https://github.com/PyLops/pylops-mpi/actions/workflows/pages/pages-build-deployment)\n![OS-support](https://img.shields.io/badge/OS-linux,osx-850A8B.svg)\n[![Slack Status](https://img.shields.io/badge/chat-slack-green.svg)](https://pylops.slack.com)\n[![DOI](https://joss.theoj.org/papers/10.21105/joss.07512/status.svg)](https://doi.org/10.21105/joss.07512)\n\n# Distributed linear operators and solvers\nPylops-mpi is a Python library built on top of [PyLops](https://pylops.readthedocs.io/en/stable/), designed to enable distributed and parallel processing of \nlarge-scale linear algebra operations and computations.  \n\n## Installation\nTo install pylops-mpi, you need to have Message Passing Interface (MPI) and optionally Nvidia's Collective Communication Library (NCCL) installed on your system.\n\n1. **Download and Install MPI**: Visit the official MPI website to download an appropriate MPI implementation for your system. \nFollow the installation instructions provided by the MPI vendor.\n   - [Open MPI](https://www.open-mpi.org/software/ompi/v1.10/)\n   - [MPICH](https://www.mpich.org/downloads/)\n   - [Intel MPI](https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html#gs.10j8fx)\n\n2. **Verify MPI Installation**: After installing MPI, verify its installation by opening a terminal or command prompt \nand running the following command:\n   ```\n   mpiexec --version\n   ```\n\n3. **Install pylops-mpi**: Once MPI is installed and verified, you can proceed to install `pylops-mpi` via `pip`:\n   ```\n   pip install pylops-mpi\n   ```\n\n4. (Optional) To enable the NCCL backend for multi-GPU systems, install `cupy` and `nccl` via `pip`:\n   ```\n   pip install cupy-cudaXx nvidia-nccl-cuX\n   ```\n   \n   with `X=11,12`.\n\nAlternatively, if the Conda package manager is used to setup the Python environment, steps 1 and 2 can be skipped and `mpi4py` can be installed directly alongside the MPI distribution of choice:\n\n```\nconda install -c conda-forge mpi4py X\n```\n\nwith `X=mpich, openmpi, impi_rt, msmpi`. Similarly step 4 can be accomplished using:\n\n```\nconda install -c conda-forge cupy nccl \n```\n\nSee the docs ([Installation](https://pylops.github.io/pylops-mpi/installation.html)) for more information.\n\n## Run Pylops-MPI\nOnce you have installed the prerequisites and pylops-mpi, you can run pylops-mpi using the `mpiexec` command. \n\nHere is an example on how to run a python script called `\u003cscript_name\u003e.py`:\n```\nmpiexec -n \u003cNUM_PROCESSES\u003e python \u003cscript_name\u003e.py\n```\n\n## Example: A distributed finite-difference operator\nThe following example is a modified version of \n[PyLops' README](https://github.com/PyLops/pylops/blob/dev/README.md)_ starting \nexample that can handle a 2D-array distributed across ranks over the first dimension \nvia the `DistributedArray` object:\n\n```python\nimport numpy as np\nfrom pylops_mpi import DistributedArray, Partition\n\n# Initialize DistributedArray with partition set to Scatter\nnx, ny = 11, 21\nx = np.zeros((nx, ny), dtype=np.float64)\nx[nx // 2, ny // 2] = 1.0\n\nx_dist = pylops_mpi.DistributedArray.to_dist(\n            x=x.flatten(), \n            partition=Partition.SCATTER)\n\n# Distributed first-derivative\nD_op = pylops_mpi.MPIFirstDerivative((nx, ny), dtype=np.float64)\n\n# y = Dx\ny_dist = D_op @ x_dist\n\n# xadj = D^H y\nxadj_dist = D_op.H @ y_dist\n\n# xinv = D^-1 y\nx0_dist = pylops_mpi.DistributedArray(D_op.shape[1], dtype=np.float64)\nx0_dist[:] = 0\nxinv_dist = pylops_mpi.cgls(D_op, y_dist, x0=x0_dist, niter=10)[0]\n```\n\nNote that the `DistributedArray` class provides the `to_dist` class method that accepts a NumPy array as input and converts it into an instance of the `DistributedArray` class. This method is used to transform a regular NumPy array into a DistributedArray that is distributed and processed across multiple nodes or processes.\n\nMoreover, the `DistributedArray` class provides also fundamental mathematical operations, such as element-wise addition, subtraction, multiplication, dot product, and an equivalent of the [`np.linalg.norm`](https://numpy.org/doc/stable/reference/generated/numpy.linalg.norm.html) function that operate in a distributed fashion, \nthus utilizing the efficiency of the MPI/NCC; protocols. This enables efficient computation and processing of large-scale distributed arrays.\n\n## Running Tests\nThe MPI test scripts are located in the `tests` folder.\nUse the following command to run the tests:\n```\nmpiexec -n \u003cNUM_PROCESSES\u003e pytest tests/ --with-mpi\n```\nwhere the `--with-mpi` option tells pytest to enable the `pytest-mpi` plugin, allowing the tests to utilize the MPI functionality.\n\nSimilarly, to run the NCCL test scripts in the `tests_nccl` folder, \nuse the following command to run the tests:\n```\nmpiexec -n \u003cNUM_PROCESSES\u003e pytest tests_nccl/ --with-mpi\n```\n\n## Documentation \nThe official documentation of Pylops-MPI is available [here](https://pylops.github.io/pylops-mpi/).\nVisit the official docs to learn more about pylops-mpi.\n\n## Contributors\n* Rohan Babbar, rohanbabbar04\n* Yuxi Hong, hongyx11\n* Matteo Ravasi, mrava87\n* Tharit Tangkijwanichakul, tharittk\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpylops%2Fpylops-mpi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpylops%2Fpylops-mpi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpylops%2Fpylops-mpi/lists"}