{"id":24358821,"url":"https://github.com/bupt-gamma/space4hgnn","last_synced_at":"2025-04-10T05:51:55.825Z","repository":{"id":103997843,"uuid":"429318208","full_name":"BUPT-GAMMA/Space4HGNN","owner":"BUPT-GAMMA","description":"Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network","archived":false,"fork":false,"pushed_at":"2022-04-25T03:26:07.000Z","size":1392,"stargazers_count":28,"open_issues_count":1,"forks_count":5,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-24T07:06:03.978Z","etag":null,"topics":["dgl","heterogeneous-graph-neural-network","openhgnn","sigir2022"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BUPT-GAMMA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-18T06:17:10.000Z","updated_at":"2024-10-30T13:53:37.000Z","dependencies_parsed_at":"2023-07-10T15:34:51.180Z","dependency_job_id":null,"html_url":"https://github.com/BUPT-GAMMA/Space4HGNN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUPT-GAMMA%2FSpace4HGNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUPT-GAMMA%2FSpace4HGNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUPT-GAMMA%2FSpace4HGNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUPT-GAMMA%2FSpace4HGNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BUPT-GAMMA","download_url":"https://codeload.github.com/BUPT-GAMMA/Space4HGNN/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248166928,"owners_count":21058480,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dgl","heterogeneous-graph-neural-network","openhgnn","sigir2022"],"created_at":"2025-01-18T20:23:08.419Z","updated_at":"2025-04-10T05:51:55.808Z","avatar_url":"https://github.com/BUPT-GAMMA.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network\n\nPaper: [Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network](https://arxiv.org/abs/2202.09177)\n\nFollowing [GraphGym](https://github.com/snap-stanford/GraphGym), we release a platform Space4HGNN for designing and evaluating Heterogeneous Graph Neural Networks (HGNN). It is implemented with PyTorch and DGL, using the OpenHGNN package.\n\nWe have deployed the code into [OpenHGNN](https://github.com/BUPT-GAMMA/OpenHGNN). Here we will introduce the *space4hgnn* part in OpenHGNN and how to run it.\n\n## The file tree of Space4HGNN in OpenHGNN\n\n```tree\n.\n├── README.md\n├── openhgnn\n│   ├── __init__.py\n│   ├── dataset\n│   │   ├── LinkPredictionDataset.py\n│   │   ├── NodeClassificationDataset.py\n│   │   ├── README.md\n│   │   ├── __init__.py\n│   │   ├── academic_graph.py\n│   │   ├── base_dataset.py\n│   │   ├── hgb_dataset.py\n│   │   └── utils.py\n│   ├── layers\n│   │   ├── GeneralGNNLayer.py\n│   │   ├── GeneralHGNNLayer.py\n│   │   ├── HeteroGraphConv.py\n│   │   ├── HeteroLinear.py\n│   │   ├── MetapathConv.py\n│   │   ├── SkipConnection.py\n│   │   └── __init__.py\n│   ├── models\n│   │   ├── __init__.py\n│   │   ├── base_model.py\n│   │   ├── general_HGNN.py\n│   │   └── homo_GNN.py\n│   ├── tasks\n│   │   ├── README.md\n│   │   ├── __init__.py\n│   │   ├── base_task.py\n│   │   ├── link_prediction.py\n│   │   ├── node_classification.py\n│   ├── trainerflow\n│   │   ├── README.md\n│   │   ├── link_prediction.py\n│   │   ├── node_classification.py\n│   └── utils\n│       ├── __init__.py\n│       ├── activation.py\n│       ├── evaluator.py\n│       └── utils.py\n├── requirements.txt\n├── setup.py\n├── space4hgnn\n│   ├── README.md\n│   ├── __init__.py\n│   ├── figure\n│   │   ├── distribution.py\n│   │   └── rank.py\n│   ├── generate_yaml.py\n│   ├── parallel.sh\n│   ├── prediction\n│   │   └── excel\n│   │       └── gather_all_Csv.py\n│   └── utils.py\n└── space4hgnn.py\n```\n\n## How to run\n\n### 1 Install\n\nThe installation process is same with OpenHGNN [Get Started](https://github.com/BUPT-GAMMA/OpenHGNN#get-started).\n\n### 2 Run a single experiment\n\n#### 2.1 Generate designs randomly\n\nHere we will generate a random design combination for each dataset and save it in a `.yaml` file. The candidate designs are listed in [`./space4hgnn/generate_yaml.py`](./generate_yaml.py).\n\n```bash\npython ./space4hgnn/generate_yaml.py --gnn_type gcnconv --times 1 --key has_bn --configfile test\n```\n\n``--aggr -a``, specify the gnn type, [and gcnconv, gatconv, sageconv, ginconv are optional].\n\n``--times -t``, the ID of yaml file to control different random sampling.\n\n``--key -k``, specify  a design dimension.\n\n``--configfile -c``, specify a directory name to store configure yaml file.\n\n**Note:**  ``.yaml`` file will be saved in the yaml_file_path which is controlled by four arguments.\n\n```python\nyaml_file_path = './space4hgnn/config/{}/{}/{}_{}.yaml'.format(configfile, key, gnn_type, times)\n# Here yaml_file_path = './space4hgnn/config/test/has_bn/gcnconv_1.yaml' with the above example code\n```\n\n#### 2.2 **Launch an experiment**\n\n```bash\npython space4hgnn.py -m general_HGNN -u metapath -t node_classification -d HGBn-ACM -g 0 -r 5 -a gcnconv -s 1 -k has_bn -v True -c test -p HGB\n```\n\n``--model -m ``  name of models\n\n``--subgraph_extraction -u`` subgraph extraction methods\n\n``--task -t`` name of task\n\n``--dataset -t`` name of dataset\n\n``--gpu -g`` controls which gpu you will use. If you do not have gpu, set -g -1.\n\n``--repeat -r`` times to repeat, default 5\n\n``--gnn_type -a `` gun type. \n\n``--times -t`` same with generating random designs\n\n``--key -k`` a design dimension\n\n``--value -v`` the value of ``key`` design dimension\n\n``--configfile -c  `` load the yaml file which is in the directory configfile\n\n``--predictfile -p`` The file path to store predict files.\n\ne.g.: \n\nWe implement three model families in Space4HGNN, Homogenization model family, Relation model family, Meta-path model family.\n\nFor **Homogenization model family**, we can omit the parameter ``--subgraph_extraction``,\n\n```bash\npython space4hgnn.py -m homo_GNN -t node_classification -d HGBn-ACM -g 0 -r 5 -a gcnconv -s 1 -k has_bn -v True -c test -p HGB\n```\n\nFor **Relation model family**, ``--model`` is general_HGNN and ``--subgraph_extraction`` is relation,\n\n```bash\npython space4hgnn.py -m general_HGNN -u relation -t node_classification -d HGBn-ACM -g 0 -r 5 -a gcnconv -s 1 -k has_bn -v True -c test -p HGB\n```\n\nFor **Meta-path model family**, ``--model`` is general_HGNN and ``--subgraph_extraction`` is meta-path\n\n```bash\npython space4hgnn.py -m general_HGNN -u metapath -t node_classification -d HGBn-ACM -g 0 -r 5 -a gcnconv -s 1 -k has_bn -v True -c test -p HGB\n```\n\n**Note: ** \n\nSimilar with  generating yaml file, experiment will load the design configuration from ``yaml_file_path``. And it will save the results into a `.csv` file in `prediction_file_path`.\n\n```python\nyaml_file_path = './space4hgnn/config/{}/{}/{}_{}.yaml'.format(configfile, key, gnn_type, times)\n# Here yaml_file_path = './space4hgnn/config/test/has_bn/gcnconv_1.yaml'\nprediction_file_path = './space4hgnn/prediction/excel/{}/{}_{}/{}_{}_{}_{}.csv'.format(predictfile, key, value, model_family, gnn_type, times, dataset)\n# Here prediction_file_path = './space4hgnn/prediction/test/has_bn_True/metapath_gcnconv_1_HGBn-ACM.yaml'\n```\n\n### 3 Run a batch of experiments\n\nAn example:\n\n```bash\n./space4hgnn/parallel.sh 0 5 has_bn True node_classification test_paral test_paral\n```\n\nIt will generate  configuration files for the batch of experiments and launch a batch of experiments.\n\nThe following is the arguments descriptions:\n\n1. The first argument controls which gpu to use. Here is 0.\n2. Repeat times. Here is 5\n3. Design dimension. Here is BN.\n4. Choice of design dimension. Here set BN `` True``.\n5. Task name. Here is nodeclassification\n6. Configfile is the path to save configuration files. \n7. Predictfile is the path to save prediction files.\n\n**Note**: \n\nIf you encounter the error ``bash: ./space4hgnn/parallel.sh: Permission denied``, you can try  with cmd ``chmod +x ./space4hgnn/parallel.sh``.\n\n### 3 Analyze the results\n\n#### 3.1 Gather all results\n\nTo gather all experiments results, we should run the following command to gather all results into one ``.csv`` file.\n\n```bash\npython ./space4hgnn/prediction/excel/gather_all_Csv.py -p ./space4hgnn/prediction/excel/HGB\n```\n\n#### 3.2 Analyze with figures\n\nWe offer ``./figure/result.csv`` recording the experimental results.\n\n##### 3.2.1 Ranking analysis\n\nWe analyze the results with average ranking following [GraphGym](https://github.com/snap-stanford/GraphGym#3-analyze-the-results), the corresponding code is in [`figure/rank.py`](./figure/rank.py).\n\n![space4hgnn_rank](./figure/space4hgnn_rank.png)\n\n##### 3.2.2 Distribution estimates\n\nWe analyze the results with distribution estimates following [NDS](https://github.com/facebookresearch/nds), and the corresponding code is in [`figure/distribution.py`](./figure/distribution.py).\n\n![space4hgnn_distribution](./figure/space4hgnn_distribution.png)\n\n## Cite\n\nPlease kindly cite our paper if you use this code:\n\n```\n@inproceedings{zhao2022space4hgnn,\n  title={Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network},\n  author={Zhao, Tianyu and Yang, Cheng and Li, Yibo and Gan, Quan and Wang, Zhenyi and Liang, Fengqi and Zhao, Huan and Shao, Yingxia and Wang, Xiao and Shi, Chuan},\n  booktitle={SIGIR},\n  year={2022}\n}\n```\n\n## Acknowledgement\n\nThe code is built on  [GraphGym](https://github.com/snap-stanford/GraphGym), a method defining design space for graph neural network.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbupt-gamma%2Fspace4hgnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbupt-gamma%2Fspace4hgnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbupt-gamma%2Fspace4hgnn/lists"}