{"id":21862849,"url":"https://github.com/tugraph-family/tugraph-antgraphlearning","last_synced_at":"2025-04-14T19:42:36.710Z","repository":{"id":193172205,"uuid":"681020404","full_name":"TuGraph-family/TuGraph-AntGraphLearning","owner":"TuGraph-family","description":"Ant Graph Learning (AGL) provides a comprehensive solution for graph learning tasks at an industrial scale.","archived":false,"fork":false,"pushed_at":"2024-01-12T08:20:38.000Z","size":2803,"stargazers_count":87,"open_issues_count":1,"forks_count":9,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-28T08:03:58.770Z","etag":null,"topics":["deep-learning","gnn","gnn-framework"],"latest_commit_sha":null,"homepage":"https://openagl.openfinai.org/en-US","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TuGraph-family.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-08-21T05:07:15.000Z","updated_at":"2025-03-27T12:11:42.000Z","dependencies_parsed_at":"2023-12-13T07:39:38.645Z","dependency_job_id":null,"html_url":"https://github.com/TuGraph-family/TuGraph-AntGraphLearning","commit_stats":null,"previous_names":["tugraph-family/tugraph-antgraphlearning"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TuGraph-family%2FTuGraph-AntGraphLearning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TuGraph-family%2FTuGraph-AntGraphLearning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TuGraph-family%2FTuGraph-AntGraphLearning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TuGraph-family%2FTuGraph-AntGraphLearning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TuGraph-family","download_url":"https://codeload.github.com/TuGraph-family/TuGraph-AntGraphLearning/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248949009,"owners_count":21187980,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","gnn","gnn-framework"],"created_at":"2024-11-28T03:17:44.406Z","updated_at":"2025-04-14T19:42:36.680Z","avatar_url":"https://github.com/TuGraph-family.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Ant Graph Learning\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](./LICENSE)\n\n[Website](https://openagl.openfinai.org/en-US)|[中文文档](./README_CN.md)\n\nAnt Graph Learning (AGL) provides a comprehensive solution for graph learning tasks at an industrial scale.\n\n[//]: # (\u003cdiv align=\"center\"\u003e)\n\n[//]: # (\u003cimg src=./doc/core/architecture.png\u003e)\n\n[//]: # (\u003cbr\u003e)\n\n[//]: # (\u003cb\u003eFigure\u003c/b\u003e: AGL Overall Architecture)\n\n[//]: # (\u003c/div\u003e)\n\n![](doc/core/English/images/architecture_EN.png)\n\nGraph learning tasks in industrial settings exhibit the following characteristics:\n\n* Complex graph data:\n    * Large-scale graphs: typically consisting of billions of nodes, tens of billions of edges, and millions of samples.\n    * Data dependencies: The computation of a node's embedding relies on the embeddings of its neighboring nodes/edges.\n    * Diverse types: homogeneous/heterogeneous/dynamic graph.\n* Complex task types:\n    * Offline: offline training, offline batch prediction, offline full-graph prediction.\n    * Online: online training, online prediction (consistent with offline results).\n* Complex usage/scenarios:\n    * Multi-tenancy.\n    * Diverse usage scenarios: GNN-only, GNN + search and recommendation/multi-modal.\n    * Heterogeneous resources: CPU/GPU clusters.\n\nAGL addresses these challenges by adopting the following approaches:\n\n* The Graph scale issue:\n    * AGL tackles the problem of data dependencies by transforming large graphs\n      into smaller subgraphs in advance.\n* Scalability/Extensibility:\n    * Graph sampling: conditional filtering (index) + sampling (random/probabilistic, TopK).\n    * Graph representation: AGL provides a graph-feature format that is capable of representing homogeneous,\n      heterogeneous, and dynamic graphs. Additionally, it supports node-level, edge-level, and graph-level subgraphs,\n      allowing for more granular analysis and learning. Furthermore, AGL provides the option to store only the structure\n      of the graph,\n      which can be beneficial for certain use cases.\n    * Graph training: AGL resolves the data dependency problem inherent in graph data, facilitating\n      large-scale distributed training through the utilization of mature deep neural network (DNN) training\n      architectures such as PS (Parameter Server) and AllReduce. These architectures enable efficient and scalable\n      training processes, ensuring the seamless handling of graph data on a distributed scale.\n* Stability:\n    * Reuse mature Spark or MapReduce (graph sampling phase) and DNN infrastructure for elasticity and fault tolerance.\n* Consistency:\n    * Sample consistency: graph samples generated offline can be reused for online/offline prediction.\n* Resource cost:\n    * Graph features can be stored on disk, thereby reducing the memory requirements.\n\nBased on these considerations, AGL has developed comprehensive solutions for graph data construction and learning,\nenabling the completion of large-scale graph learning tasks on regular machines or clusters:\n\n* Graph sampling:\n    * AGL leverages Spark (or MR) to pre-extract k-hop neighborhood information of target nodes as graph features.\n* Graph training:\n    * During the training phase, AGL incorporates parsing logic to convert graph features into essential components such\n      as the adjacency matrix, node feature matrix, and edge feature matrix, along with other necessary information for\n      the model. This seamless integration of graph learning tasks into the regular DNN learning mode allows for the\n      convenient reuse of mature technologies and infrastructure typically used in standard DNN workflows.\n\nAGL currently employs PyTorch as its backend and integrates open-source algorithm libraries like PyG to ease the\ndevelopment process for users.\nFurthermore, AGL has developed some in-house graph algorithms, including node classification, edge prediction, and\nrepresentation learning, specifically tailored for handling complex graph data in various forms such as homogeneous,\nheterogeneous, and dynamic graphs.\n\n# How to use\n\n* [Installation Guide](doc/core/English/install_EN.md)\n* [Process Workflow](doc/core/English/process_description_EN.md)\n* [Generate Graph Samples](doc/core/English/sampler/0_data_preparation_EN.md)\n* [Graph Learning Tutorial](doc/core/English/graph_learning_tutorial_EN.md)\n\n# How to Contribute\n\n* [Contribution Guidelines](doc/core/English/contribution_EN.md)\n\n# Cite\n\n```\n@article{zhang13agl,\n  title={AGL: A Scalable System for Industrial-purpose Graph Machine Learning},\n  author={Zhang, Dalong and Huang, Xin and Liu, Ziqi and Zhou, Jun and Hu, Zhiyang and Song, Xianzheng and Ge, Zhibang and Wang, Lin and Zhang, Zhiqiang and Qi, Yuan},\n  journal={Proceedings of the VLDB Endowment},\n  volume={13},\n  number={12}\n}\n\n@inproceedings{zhang2023inferturbo,\n  title={InferTurbo: A Scalable System for Boosting Full-graph Inference of Graph Neural Network over Huge Graphs},\n  author={Zhang, Dalong and Song, Xianzheng and Hu, Zhiyang and Li, Yang and Tao, Miao and Hu, Binbin and Wang, Lin and Zhang, Zhiqiang and Zhou, Jun},\n  booktitle={2023 IEEE 39th International Conference on Data Engineering (ICDE)},\n  pages={3235--3247},\n  year={2023},\n  organization={IEEE Computer Society}\n}\n```\n\n# License\n\n[Apache License 2.0](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftugraph-family%2Ftugraph-antgraphlearning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftugraph-family%2Ftugraph-antgraphlearning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftugraph-family%2Ftugraph-antgraphlearning/lists"}