{"id":18800549,"url":"https://github.com/xtra-computing/clementi","last_synced_at":"2025-07-26T04:10:12.263Z","repository":{"id":286330306,"uuid":"960811544","full_name":"Xtra-Computing/Clementi","owner":"Xtra-Computing","description":"Clementi code repo (accepted by SIGMOD2025)","archived":false,"fork":false,"pushed_at":"2025-04-05T18:25:25.000Z","size":1502,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-06-09T03:02:36.382Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Xtra-Computing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-04-05T05:46:00.000Z","updated_at":"2025-05-12T04:08:43.000Z","dependencies_parsed_at":"2025-04-05T19:37:20.657Z","dependency_job_id":null,"html_url":"https://github.com/Xtra-Computing/Clementi","commit_stats":null,"previous_names":["xtra-computing/clementi"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Xtra-Computing/Clementi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xtra-Computing%2FClementi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xtra-Computing%2FClementi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xtra-Computing%2FClementi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xtra-Computing%2FClementi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Xtra-Computing","download_url":"https://codeload.github.com/Xtra-Computing/Clementi/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xtra-Computing%2FClementi/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267116553,"owners_count":24038624,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-26T02:00:08.937Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T22:19:00.901Z","updated_at":"2025-07-26T04:10:12.255Z","avatar_url":"https://github.com/Xtra-Computing.png","language":"C++","readme":"### Clementi: Efficient Load Balancing and Communication Overlap for Multi-FPGA Graph Processing\n\n## Introduction\nClementi is a multi-FPGA based graph processing framework designed to achieve near-linear scalability. By overlapping communication with computation, Clementi optimizes end-to-end performance. Additionally, leveraging a custom hardware architecture in FPGA, we propose an architecture-oriented performance model and a workload scheduling method to minimize execution time discrepancies among FPGAs. Our experimental results demonstrate that Clementi significantly outperforms existing multi-FPGA frameworks, achieving speedups ranging from 1.86× to 8.75×, and exhibits near-linear scalability as the number of FPGAs increases.\n\n## Update Notice\n\nPlease note that the documentation is actively being updated, send to dcsyufeng@gmail.com if you have any question.\n\n## Prerequisites\nClementi development utilizes the Xilinx Vitis toolset. Key components include:\n- **Xilinx XRT** version 2.14.354\n- **Vitis v++** at v2021.2 (64-bit)\n- **OpenMPI** at 4.1.4\n\nThe framework is executed on a public FPGA cluster: HACC cluster at NUS. For detailed information and access to this cluster, please refer to the [HACC_NUS website](https://xacchead.d2.comp.nus.edu.sg/). Each FPGA on this cluster is paired with a virtual CPU node, utilizing OpenMPI for distributed execution. The specific version used is Open MPI 4.1.4, and the system incorporates four [Xilinx U250 FPGAs](https://docs.amd.com/r/en-US/ug1120-alveo-platforms/U200-Gen3x16-XDMA-base_2-Platform).\n\n## System Overview\n![Clementi Overview](images/Clementi_overview.png)\nClementi utilizes a three-phase approach to process large graphs on multi-FPGA platforms with a ring topology:\n1. **Graph Partitioning:** The input edge list is initially partitioned into subgraphs using a 2D partitioning method that integrates interval-shard and input-aware partition.\n2. **Subgraph Assignment:** A performance model predicts execution times for each subgraph, which informs a greedy-based scheduling algorithm to ensure balanced workloads across the FPGAs.\n3. **Concurrent Processing:** Each FPGA concurrently processes its assigned subgraphs, overlapping gather-scatter and global apply stages to optimize performance.\n\n## Initialization\nIn order to generate this design you will need a valid [UltraScale+ Integrated 100G Ethernet Subsystem](https://www.xilinx.com/products/intellectual-property/cmac_usplus.html) license [installation](#Licence-Installation) in Vivado. \n\nTo begin working with Clementi, clone this repository.\n## Build Hardware\n```bash\n# Load the Xilinx Vitis and XRT settings\nsource /opt/Xilinx/Vitis/2021.2/settings64.sh\nsource /opt/xilinx/xrt/setup.sh\n\n# Build all components for the Clementi application\n# dependencies:\nsudo apt install libgraphviz-dev faketime\npip3 install graphviz\n\n# make with the default configuration (as in app makefile)\nmake TARGET=hw all APP=clementi TYPE=pr # make with specified target\n# or \nmake TARGET=hw all APP=gather_scatter TYPE=pr\n# or\nmake TARGET=hw all APP=global_apply\n# or\nmake TARGET=hw all APP=single_gas TYPE=pr\n```\n\n## Build Software\n```bash\n#If you only need to build software code, use the following command:\nmake host APP=clementi\n# or \nmake host APP=gather_scatter\n# or\nmake host APP=global_apply\n```\n\n## Test\nRun the hardware single graph processor script to test the system. Ensure that you have the correct OpenMPI settings configured before running the multiple FPGAs demo:\n```bash\n## For test gather_scatter module:\n./script/run_gather_scatter.sh\n## For test global_apply module:\n./script/run_global_apply.sh\n## For test Clementi:\n./script/run_clementi.sh R25 30 ## dataset = R25, and superstep = 30.\n\n## For test single_gas, first login a U250 FPGA node, then:\n./single_gas.app -d R25 -s 30 ## dataset = R25, and superstep = 30.\n```\n[Note] please pay attention to the graph dataset directory.\n\n## Repository Structure\n\n- `app` - Contains applications used within the project.\n- `host` - Host files for the XRT driver.\n- `images` - Images used in the README documentation.\n- `mk` - Directory containing makefiles.\n- `partition` - Includes the input-aware partition method and performance model.\n- `src` - Hardware files for each block in the Clementi framework.\n- `test` - Contains test files and scripts.\n- `host_file` - Host files used in MPI code.\n- `script` - Bash scripts in compilation.\n\n\n## Licenses\n\n**Ethernet/cmac** License: [BSD 3-Clause License](THIRD_PARTY_LIC.md)\n\n**NetLayers/100G-fpga-network-stack-core** License: [BSD 3-Clause License](THIRD_PARTY_LIC.md)\n\n## License Installation\n\nHere is the simple solution for install cmac license using linux command:\n```bash\n## Step 1. Source the Vitis settings script:\nsource /path/to/Vitis/settings64.sh\n## Step 2. Set the license file environment variable:\nexport XILINXD_LICENSE_FILE=/path/to/your/license.lic\n## Step 3. To check the IP status in your project:\n## a. Start Vivado in TCL mode:\nvivado -mode tcl\n## b. Open your project:\nopen_project /path/to/your/project.xpr\n## c. Generate a report for all IP statuses and save it to a file:\nreport_ip_status -all \u003e /path/to/ip_status_report.txt\n## d. Close the project and exit the Vivado TCL shell:\nclose_project\n```\n\n\n## 📄 Related Publication\n\nIf you find this project helpful, please consider citing our related work:\n\n\u003e **Feng Yu, Hongshi Tan, Xinyu Chen, Yao Chen, Bingsheng He, and Weng-Fai Wong.**  \n\u003e *Clementi: Efficient Load Balancing and Communication Overlap for Multi-FPGA Graph Processing*.  \n\u003e Proceedings of the ACM on Management of Data (PACMMOD), Vol. 3, No. 3 (SIGMOD), Article 138, June 2025.  \n\u003e [https://doi.org/10.1145/3725275](https://doi.org/10.1145/3725275)\n\n### BibTeX\n```bibtex\n@article{yu2025clementi,\n  title     = {Clementi: Efficient Load Balancing and Communication Overlap for Multi-FPGA Graph Processing},\n  author    = {Feng Yu and Hongshi Tan and Xinyu Chen and Yao Chen and Bingsheng He and Weng-fai Wong},\n  journal   = {Proceedings of the ACM on Management of Data (PACMMOD)},\n  volume    = {3},\n  number    = {3 (SIGMOD)},\n  article   = {138},\n  year      = {2025},\n  month     = {June},\n  doi       = {10.1145/3725275},\n  publisher = {ACM}\n}\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxtra-computing%2Fclementi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxtra-computing%2Fclementi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxtra-computing%2Fclementi/lists"}