{"id":18422020,"url":"https://github.com/spcl/gdi-rma","last_synced_at":"2025-04-07T14:32:21.053Z","repository":{"id":206943991,"uuid":"682917743","full_name":"spcl/GDI-RMA","owner":"spcl","description":"Official Implementation of \"The Graph Database Interface: Scaling Online Transactional and Analytical Graph Workloads to Hundreds of Thousands of Cores\"","archived":false,"fork":false,"pushed_at":"2023-11-14T14:52:47.000Z","size":2223,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-04-28T02:00:17.009Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://dl.acm.org/doi/10.1145/3581784.3607068","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spcl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-08-25T07:00:51.000Z","updated_at":"2024-04-03T11:58:33.000Z","dependencies_parsed_at":"2023-11-13T10:43:38.495Z","dependency_job_id":null,"html_url":"https://github.com/spcl/GDI-RMA","commit_stats":null,"previous_names":["spcl/gdi-rma"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spcl%2FGDI-RMA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spcl%2FGDI-RMA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spcl%2FGDI-RMA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spcl%2FGDI-RMA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spcl","download_url":"https://codeload.github.com/spcl/GDI-RMA/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223283983,"owners_count":17119593,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T04:27:45.676Z","updated_at":"2024-11-06T04:27:46.323Z","avatar_url":"https://github.com/spcl.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GDI-RMA (GDA)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"paper/pics/gdi-main-pic_cases_SMALL.svg\" style=\"width: 30vw\"\u003e\n\u003c/p\u003e\n\nGDI-RMA is a reference implementation of the paper The Graph Database Interface:\nScaling Online Transactional and Analytical Graph Workloads to Hundreds of\nThousands of Cores presented at the International Conference for High\nPerformance Computing, Networking, Storage and Analysis (SC '23).\n\nGDI-RMA implements a distributed-memory graph database with the use of one-sided\nRemote Direct Memory Access (RDMA). GDI-RMA was scaled to over 7,000 servers\nwith 120,000 cores and 77 TB of distributed memory with datasets scaling up to\n34 billion vertices and 549 billion edges.\n\nGDI-RMA stores all data in the main memory and does not follow the traditional\nserver-client model, but instead uses RDMA to access remote data, which allows\nthe processes that store the graph database data also to issue and process\nqueries.\n\n## Setup Guide\n\nThe source code to build the GDI-RMA library can be found in the [src](src)\ndirectory. The library is written in C and uses MPI for its communication needs\n(one-sided/collectives). Please update the C compiler and its flags to the needs\nof your system at the beginning of the [Makefile](src/Makefile) (variables `CC`\nand `CFLAGS`). Afterwards calling `make` should result in the creation of the\nlibrary file `libgdi.a`.\n\nIf you intend to use\n[foMPI](https://spcl.inf.ethz.ch/Research/Parallel_Programming/foMPI/), please\nenable the related parameters in lines 6 to 9 of the Makefile and set the path\nto your foMPI installation in line 6.\n\n## Quickstart\n\nThe following source code provides a minimal example, that creates a simple\ngraph database with two vertices that are connected through a directed edge.\nError checking is omitted for clarity.\n\n```C\n#include \u003cstdio.h\u003e\n\n#include \"gdi.h\"\n\nint main( int argc, char* argv[] ) {\n  /* initialization */\n  MPI_Init( \u0026argc, \u0026argv );\n  GDI_Init( \u0026argc, \u0026argv );\n\n  /* graph database creation */\n  GDI_Database db;\n  GDA_Init_params parameters;\n  parameters.block_size = 256;\n  parameters.memory_size = 4096;\n  parameters.comm = MPI_COMM_WORLD;\n  GDI_CreateDatabase( \u0026parameters, sizeof(GDA_Init_params), \u0026db );\n\n  /* start transaction */\n  GDI_Transaction transaction;\n  GDI_StartTransaction( db, \u0026transaction );\n\n  /* create two vertices */\n  GDI_VertexHolder vert1, vert2;\n  uint64_t vertex_id = 1;\n  GDI_CreateVertex( \u0026vertex_id, 8, transaction, \u0026vert1 );\n  vertex_id = 2;\n  GDI_CreateVertex( \u0026vertex_id, 8, transaction, \u0026vert2 );\n\n  /* connect vertices with an edge */\n  GDI_EdgeHolder edge;\n  GDI_CreateEdge( GDI_EDGE_DIRECTED, vert1, vert2, \u0026edge );\n\n  /* commit transaction to the database */\n  int status;\n  status = GDI_CloseTransaction( \u0026transaction, GDI_TRANSACTION_COMMIT );\n  if (status == GDI_SUCCESS ) {\n    printf(\"Transaction successful.\\n\");\n  }\n\n  /* graph database deallocation and clean up */\n  GDI_FreeDatabase( \u0026db );\n  GDI_Finalize();\n  MPI_Finalize();\n\n  return 0;\n}\n```\nAssuming that the source code is stored in the main directory in the file\n`min_example.c`, it can be compiled for example with the command `mpicc -o\nmin_example min_example.c -Isrc -Lsrc -lgdi` and executed with the command\n`mpirun -n \u003c#procs\u003e ./min_example`.\n\n## Documentation\n\nThe Graph Database Interface (GDI) API is documented as a\n[specification](specification/gdi_v0.9.pdf) and a high-level overview of that\nspecification can be found in the paper with additional information found in the\nrespective [README](specification/README.md). The paper also presents an\noverview of the GDI-RMA reference implementation.\n\n## Citations\n\nAny published work which uses this software should include the following\ncitation:\n\n```bibtex\n@inproceedings{besta2023graph,\n  author = {Besta, Maciej and Gerstenberger, Robert and Fischer, Marc and Podstawski, Michal and Blach, Nils and Egeli, Berke and Mitenkov, Georgy and Chlapek, Wojciech and Michalewicz, Marek and Niewiadomski, Hubert and Mueller, Juergen and Hoefler, Torsten},\n  title = {The Graph Database Interface: Scaling Online Transactional and Analytical Graph Workloads to Hundreds of Thousands of Cores},\n  year = {2023},\n  isbn = {9798400701092},\n  publisher = {Association for Computing Machinery},\n  address = {New York, NY, USA},\n  url = {https://doi.org/10.1145/3581784.3607068},\n  doi = {10.1145/3581784.3607068},\n  booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis},\n  articleno = {22},\n  numpages = {18},\n  location = {Denver, CO, USA},\n  series = {SC '23}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspcl%2Fgdi-rma","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspcl%2Fgdi-rma","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspcl%2Fgdi-rma/lists"}