{"id":27012220,"url":"https://github.com/kabicm/grid2grid","last_synced_at":"2025-04-04T11:49:05.833Z","repository":{"id":55092605,"uuid":"181563781","full_name":"kabicm/grid2grid","owner":"kabicm","description":"A library transforming between two arbitrary grid-like matrix data layouts over MPI ranks.","archived":false,"fork":false,"pushed_at":"2021-01-11T09:45:28.000Z","size":517,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-05-01T18:16:18.286Z","etag":null,"topics":["distributed-algorithm","linear-algebra","matrix","mpi","openmp"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kabicm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-15T20:51:02.000Z","updated_at":"2024-05-01T18:16:18.287Z","dependencies_parsed_at":"2022-08-14T11:40:57.462Z","dependency_job_id":null,"html_url":"https://github.com/kabicm/grid2grid","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kabicm%2Fgrid2grid","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kabicm%2Fgrid2grid/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kabicm%2Fgrid2grid/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kabicm%2Fgrid2grid/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kabicm","download_url":"https://codeload.github.com/kabicm/grid2grid/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247174397,"owners_count":20896075,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-algorithm","linear-algebra","matrix","mpi","openmp"],"created_at":"2025-04-04T11:49:05.140Z","updated_at":"2025-04-04T11:49:05.822Z","avatar_url":"https://github.com/kabicm.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# grid2grid\n\nThis is a library that transforms a matrix between two arbitrary grid-like data layouts. By layout, we mean the way in which a matrix is distributed over MPI ranks.\n\nFor example, imagine a matrix split into four blocks, such that each block resides on a different rank as shown below:\n\n```\n+---------------------+\n|       |             |\n|       |             |\n|   0   |      1      |\n|       |             |\n|       |             |\n+---------------------+\n|       |             |\n|   2   |      3      |\n|       |             |\n+---------------------+\n```\n\n Assume that we want to redistribute the matrix, to achieve the following data layout:\n \n```\n+---------------------+\n|   0   |    2    | 1 |\n|       |         |   |\n+---------------------+\n|   1   |    0    | 1 |\n|       |         |   |\n+---------------------+\n|       |         |   |\n|   2   |    2    | 0 |\n|       |         |   |\n+---------------------+\n```\n\nThis is exaclty what this library is doing!\n\n## Features:\n\n- initial and final layouts do not have to be block-cyclic, but any grid-like data layout.\n- blocks do not have to be all the same dimensions, but can have different dimensions.\n- a rank might own arbitrary number of blocks\n- the type of entries in a matrix can be arbitrary\n- blocks that belong to the same rank do not have to be consecutive in memory\n- each block can have an arbitrary stride\n- OpenMP support\n\n## Performance:\n\nWhen tested against ScaLAPACK (as provided by Intel MKL), this library outperforms the MKL's `pdgemr2d` routine that transforms between two block-cyclic data layouts by up to 50\\%, as shown below:\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/kabicm/grid2grid/blob/master/docs/performance.svg\" width=\"80%\"\u003e\u003c/p\u003e\n\n## Algorithm\n\nThe pipeline of the algorithm is roughly the following:\n- Find the intersections of the initial and final grids (called grid cover) in one pass, which decomposes initial blocks into smaller blocks.\n- Sort decomposed initial blocks based on the rank to which they should be sent. Thus, if some ranks should exchange more than one block, they will do it within a single message.\n- Copy decomposed intial blocks from a local storage (with arbitrary local data layout) to a temporary MPI buffer.\n- Perform MPI_Alltoallv.\n- Use the computed grid cover to decompose blocks from the final grid into smaller blocks.\n- Sort decomposed final blocks based on the receiver rank.\n- All smaller blocks that are received are copied from a temporary MPI buffer to the local storage (with arbitrary local data layout).\n\n## Building and Installing\n\nAssuming that you want to use the `gcc 8` compiler and `OpenMP`, you can build the project as follows:\n```bash\n# clone the repo\ngit clone --recursive https://github.com/kabicm/grid2grid\ncd grid2grid\nmkdir build\ncd build\n\n# build\nCC=gcc-8 CXX=g++-8 cmake -DCMAKE_BUILD_TYPE=Release -DWITH_OPENMP=TRUE ..\n\n# compile\nmake -j 4\n```\n\n## Example\n\nTo start with, there is a small example that transforms the matrix between two block-cyclic layouts. This example can be run from `build` directory as follows:\n```bash\nmpirun --oversubscribe -np 4 ./examples/scalapack2scalapack -m 10 -n 10 -ibm 2 -ibn 3 -fbm 3 -fbn 5 -pm 2 -pn 2\n```\nWhere flags have the following meaning:\n- `(m, n)`: Dimensions of matrix that we want to change the layout for.\n- `(ibm, ibn)`: Dimensions of initial blocks, determining the initial block-cyclic distribution.\n- `(fbm, fbn)`: Dimensions of final blocks, determining the final block-cyclic distribution that we want to reach.\n- `(pm, pn)`: Processor grid, determining the processor decomposition for the block-cyclic distribution.\n\n## Arbitrary Grid-Like Data Layouts\n\nTo transform between two arbitrary grid-like data layouts, we need to construct two `grid_layout` objects, one describing the initial layout and one describing the final layout. After that, we just invoke:\n```cpp\ngrid2grid::grid_layout initial_layout(...);\ngrid2grid::grid_layout final_layout(...);\ngrid2grid::transform(initial_layout, final_layout, MPI_COMM_WORLD);\n```\n\nIn order to create a grid_layout object, we need to provide the following information:\n- `grid2D`: small struct describing the grid, basically 2 vectors, one describing where rows are split and one describing where columns are split in a grid.\n- `owners`: a matrix that specifies the owner (i.e. the rank) of each block in `grid2D`.\n- `local_blocks`: a vector of blocks that current rank owns. Each block is defined with a pointer to the beginning of that block in the local memory of current rank, a stride (by default equal to the number of rows of a block) and dimensions.\n\nThis is done for `ScaLAPACK` block-cyclic data layout, which can serve as an example of how `grid_layout` object can be created.\n\n## Author\nMarko Kabic (marko.kabic@cscs.ch)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkabicm%2Fgrid2grid","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkabicm%2Fgrid2grid","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkabicm%2Fgrid2grid/lists"}