{"id":29248823,"url":"https://github.com/hkuds/anygraph","last_synced_at":"2025-07-04T00:09:14.085Z","repository":{"id":253881256,"uuid":"841247301","full_name":"HKUDS/AnyGraph","owner":"HKUDS","description":"\"AnyGraph: Graph Foundation Model in the Wild\"","archived":false,"fork":false,"pushed_at":"2024-09-11T00:46:14.000Z","size":27588,"stargazers_count":90,"open_issues_count":0,"forks_count":5,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-09-11T05:16:32.617Z","etag":null,"topics":["graph-fundation-models","graph-learning","graph-model","graph-neural-networks","large-models"],"latest_commit_sha":null,"homepage":"https://arxiv.org/pdf/2408.10700","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HKUDS.png","metadata":{"files":{"readme":"README.md","changelog":"History/pretrain_link1.his","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-12T02:46:15.000Z","updated_at":"2024-09-11T00:46:18.000Z","dependencies_parsed_at":"2024-09-11T04:27:57.540Z","dependency_job_id":null,"html_url":"https://github.com/HKUDS/AnyGraph","commit_stats":null,"previous_names":["hkuds/anygraph"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HKUDS/AnyGraph","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FAnyGraph","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FAnyGraph/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FAnyGraph/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FAnyGraph/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HKUDS","download_url":"https://codeload.github.com/HKUDS/AnyGraph/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FAnyGraph/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263421915,"owners_count":23464052,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph-fundation-models","graph-learning","graph-model","graph-neural-networks","large-models"],"created_at":"2025-07-04T00:09:12.457Z","updated_at":"2025-07-04T00:09:13.967Z","avatar_url":"https://github.com/HKUDS.png","language":"Python","readme":"\u003ch1 align='center'\u003eAnyGraph: Graph Foundation Model in the Wild\u003c/h1\u003e\n\n\u003cdiv align='center'\u003e\n\u003ca href='https://arxiv.org/pdf/2408.10700'\u003e\u003cimg src='https://img.shields.io/badge/Paper-PDF-green'\u003e\u003c/a\u003e\n\u003c!-- \u003ca href=''\u003e\u003cimg src='https://img.shields.io/badge/公众号-blue' /\u003e\u003c/a\u003e --\u003e\n\u003c!-- \u003ca href=''\u003e\u003cimg src='https://img.shields.io/badge/CSDN-orange' /\u003e\u003c/a\u003e --\u003e\n\u003cimg src=\"https://badges.pufler.dev/visits/hkuds/anygraph?style=flat-square\u0026logo=github\"\u003e\n\u003cimg src='https://img.shields.io/github/stars/hkuds/anygraph?color=green\u0026style=social' /\u003e\n\n\u003ca href='https://akaxlh.github.io/'\u003eLianghao Xia\u003c/a\u003e and \u003ca href='https://sites.google.com/view/chaoh/group-join-us'\u003eChao Huang\u003c/a\u003e\n\n**Introducing AnyGraph, a graph foundation model designed for zero-shot predictions across domains.**\n\n\u003cimg src='imgs/article cover.png' /\u003e\n\n\u003c/div\u003e\n\n**Objectives of AnyGraph:**\n\n- **Structure Heterogeneity**: Addressing distribution shift in graph structural information.\n- **Feature Heterogeneity**: Handling diverse feature representation spaces across graph datasets.\n- **Fast Adaptation**: Efficiently adapting the model to new graph domains.\n- **Scaling Law Emergence**: Performance scales with the amount of data and model parameters.\n\n\u003cbr\u003e\n\n**Key Features of AnyGraph:**\n\n- **Graph Mixture-of-Experts (MoE)**: Effectively addresses cross-domain heterogeneity using an array of expert models.\n- **Lightweight Graph Expert Routing Mechanism**: Enables swift adaptation to new datasets and domains.\n- **Adaptive and Efficient Graph Experts**: Custom-designed to handle graphs with a wide range of structural patterns and feature spaces.\n- **Extensively Trained and Tested**: Exhibits strong generalizability over 38 diverse graph datasets, showcasing scaling laws and emergent capabilities.\n\n\u003cimg src='imgs/framework_final.jpeg' /\u003e\n\n## News\n- [x] 2024/09/18 Fix minor bugs on loading and saving path, class names, etc.\n- [x] 2024/09/11 Datasets are available now!\n- [x] 2024/09/11 Pre-trained models are available on huggingface now!\n- [x] 2024/08/20 Paper is released.\n- [x] 2024/08/20 Code is released.\n\n## Environment Setup\nDownload the data files at \u003ca href='https://huggingface.co/datasets/hkuds/AnyGraph_datasets'\u003ethis link\u003c/a\u003e. And fill in your own directories for data storage at function `get_data_files(self)` of class `DataHandler` in the file `data_handler.py`.\n\nDownload the pre-trained AnyGraph models at \u003ca href='https://huggingface.co/hkuds/AnyGraph/'\u003ehugging face\u003c/a\u003e or \u003ca href='https://hkuhk-my.sharepoint.com/:u:/g/personal/lhaoxia_hku_hk/Efmm5TJm0B5EnmYzTqg8GWEB1loKzeIR5tcr3hPIOJDXXA?e=2wMgZC'\u003eone drive\u003c/a\u003e, and put it into `Models/`.\n\n**Packages**: Our experiments were conducted with the following package versions:\n* python==3.10.13\n* torch==1.13.0\n* numpy==1.23.4\n* scipy==1.9.3\n\n**Device Requirements**: The training and testing of AnyGraph requires only one GPU with 24G memory (e.g. 3090, 4090). Using larger input graphs may require devices with larger memory.\n\n## Code Structure\nHere is a brief overview of the code structures. The explanations for each directory are enclosed in quotes (##...##).\n```\n./\n│   ├── .gitignore\n│   ├── README.md\n│   ├── data_handler.py ## load and process data\n│   ├── model.py ## implementation for the model\n│   ├── params.py ## hyperparameters\n│   └── main.py ## main file for pretraining and link prediction\n│   ├── History/ ## training and testing logs\n│   │   ├── pretrain_link1.his\n│   │   └── pretrain_link2.his\n│   ├── Models/ ## pre-trained models\n│   │   └── README.md\n│   ├── Utils/ ## utility function\n│   │   └── TimeLogger.py\n│   ├── imgs/ ## images used in readme\n│   │   ├── ablation.png\n│   │   ├── article cover.png\n│   │   ├── datasets.png\n│   │   ├── framework_final.jpeg\n│   │   ├── framework_final.pdf\n│   │   ├── framework_final.png\n│   │   ├── overall_performance2.png\n│   │   ├── routing.png\n│   │   ├── scaling_law.png\n│   │   ├── training_time.png\n│   │   ├── tuning_steps.png\n│   │   └── overall_performance1.png\n│   ├── node_classification/ ## test code for node classification\n│   │   ├── data_handler.py\n│   │   ├── model.py\n│   │   ├── params.py\n│   │   └── main.py\n│   │   ├── Utils/\n│   │   │   └── TimeLogger.py\n```\n\n## Usage\nTo reproduce the test performance reported in the paper, run the following command lines:\n```\n# Test on Link2 and Link1 data, respectively\npython main.py --load pretrain_link1 --epoch 0 --dataset link2 \npython main.py --load pretrain_link2 --epoch 0 --dataset link1\n\n# Test on the Ecommerce datasets in the Link2 and Link1 group, respectively.\n# Testing on Academic and Others datasets are conducted similarily.\npython main.py --load pretrain_link1 --epoch 0 --dataset ecommerce_in_link2\npython main.py --load pretrain_link2 --epoch 0 --dataset ecommerce_in_link1\n\n# Test the performance for node classification datasets\ncd ./node_classification\npython main.py --load pretrain_link2 --epoch 0 --dataset node\n```\n\nTo re-train the two models by yourself, run:\n```\npython main.py --dataset link2+link1 --save pretrain_link2\npython main.py --dataset link1+link2 --save pretrain_link1\n```\n\n## Datasets\n\n\u003cimg src='imgs/datasets.png' /\u003e\n\nThe statistics for the experimental datasets are presented in the table above. We categorize them into distinct groups as below. Note that Link1 and Link2 include datasets from different sources, and the datasets do not share the same feature spaces. This separation ensures a robust evaluation of true zero-shot performance in graph prediction tasks.\n\n| Group | Included Datasets |\n| ----- | ----- |\n| Link1 | Products-tech, Yelp2018, Yelp-textfeat, Products-home, Steam-text, Amazon-text, Amazon-book, Citation-2019, Citation-20Century, Pubmed-link, Citeseer, OGB-PPA, P2P-Gnutella06, Soc-Epinions1, Email-Enron |\n| Link2 | Photo, Goodreads, Fitness, Movielens-1M, Movielens10M, Gowalla, Arxiv, Arxiv-t, Cora, CS, OGB-Collab, Proteins-0, Proteins- 1, Proteins-2, Proteins-3, OGB-DDI, Web-Stanford, RoadNet-PA |\n| Ecommerce | Products-tech, Yelp2018, Yelp-textfeat, Products-home, Steam-text, Amazon-text, Amazon-book, Photo, Goodreads, Fitness, Movielens-1M, Movielens10M, Gowalla |\n| Academic | Citation-2019, Citation-20Century, Pubmed-link, Citeseer, OGB-PPA, Arxiv, Arxiv-t, Cora, CS, OGB-Collab |\n| Others | P2P-Gnutella06, Soc-Epinions1, Email-Enron, Proteins-0, Proteins- 1, Proteins-2, Proteins-3, OGB-DDI, Web-Stanford, RoadNet-PA |\n| Node | Cora, Arxiv, Pubmed, Home, Tech |\n\n\n## Experiments\n\n### Model Pre-Training Curves\nWe present the training logs with respect to epochs below. Each figure contains two curves, each corresponding to two instances of repeated pre-training.\n\n- pretrain_link1\n\n\u003cimg src='imgs/link1_loss_curve.png' width=32%/\u003e\u0026nbsp;\n\u003cimg src='imgs/link1_fullshot_ndcg_curve.png' width=32%/\u003e\u0026nbsp;\n\u003cimg src='imgs/link1_zeroshot_ndcg_curve.png' width=32%/\u003e\n\n- pretrain_link2\n\n\u003cimg src='imgs/link2_loss_curve.png' width=32%/\u003e\u0026nbsp;\n\u003cimg src='imgs/link2_fullshot_ndcg_curve.png' width=32%/\u003e\u0026nbsp;\n\u003cimg src='imgs/link2_zeroshot_ndcg_curve.png' width=32%/\u003e\n\n### Overall Performance Comparison\n\n- Comparing to few-shot end2end models and pre-training and fine-tuning methods.\n![](imgs/overall_performance1.png)\n\n- Comparing to zero-shot graph foundation models.\n\u003cimg src='imgs/overall_performance2.png' width=60%/\u003e\n\n### Scaling Law of AnyGraph\n\nWe explore the scaling law of AnyGraph by evaluating 1) model performance v.s. the number of model parameters, and 2) model performance v.s. the number of training samples. \n\nBelow shows the evaluation results on \n- all datasets across domains (a)\n- academic datasets (b)\n- ecommerce datasets (c)\n- other datasets (d) \n\nIn each subfigure, we show \n- zero-shot performance on unseen datasets w.r.t. the amount of model parameters (left)\n- full-shot performance on training datasets w.r.t. the amount of model parameters (middle)\n- zero-shot performance w.r.t. the amount of training data (right)\n\n![](imgs/scaling_law.png)\n\nThe outcome outlines the following key observations: (see Sec. 4.3 for details)\n- Generalizability of AnyGraph Follows the Scaling Law.\n- Emergent Abilities of AnyGraph.\n- Insufficient Training Data May Bring Bias.\n\n### Ablation Study\n\nThe ablation study investigates the impact of the following modules:\n- The overall MoE architecture\n- Frequency regularization in the expert routing mechanism\n- Graph augmentation in the learning process\n- The utilization of (heterogeneous) node features from different datasets\n  \n\u003cimg src='imgs/ablation.png' width=60% /\u003e\n\n\n### Expert Routing Mechanism\nWe visualize the competence scores between datasets and experts, given by the routing algorithm of AnyGraph. \n\nThe resulting scores below demonstrates the underlying relatedness between different datasets, thus demonstrating the intuitive effectiveness of the routing mechanism. (see Sec. 4.5 for details)\n\n\u003cimg src='imgs/routing.png' width=60% /\u003e\n\n\n### Fast Adaptation of AnyGraph\n\nWe study the fast adaptation abilities of AnyGraph from two aspects:\n- When fine-tuned on unseen datasets, AnyGraph achieves better performance with less training steps. (Fig. 6 below)\n- The training time of AnyGraph is comparative to that of other methods. (Table 3 below)\n\n\u003cimg src='imgs/tuning_steps.png' width=60% /\u003e\n\n\u003cimg src='imgs/training_time.png' width=60% /\u003e\n\n## Citation\n\nIf you find our work useful, please consider citing our paper:\n```\n@article{xia2024anygraph,\n  title={AnyGraph: Graph Foundation Model in the Wild},\n  author={Xia, Lianghao and Huang, Chao},\n  journal={arXiv preprint arXiv:2408.10700},\n  year={2024}\n}\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhkuds%2Fanygraph","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhkuds%2Fanygraph","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhkuds%2Fanygraph/lists"}