{"id":20216253,"url":"https://github.com/thudm/hgb","last_synced_at":"2025-04-06T06:09:14.355Z","repository":{"id":50352026,"uuid":"309365805","full_name":"THUDM/HGB","owner":"THUDM","description":"Revisiting, benchmarking, and refining Heterogeneous Graph Neural Networks.","archived":false,"fork":false,"pushed_at":"2023-07-28T04:15:02.000Z","size":63313,"stargazers_count":317,"open_issues_count":38,"forks_count":50,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-30T04:09:58.032Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUDM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-11-02T12:39:57.000Z","updated_at":"2025-03-19T03:05:49.000Z","dependencies_parsed_at":"2023-02-12T10:47:08.601Z","dependency_job_id":"339d7fc5-6000-43d4-8e82-93089504b674","html_url":"https://github.com/THUDM/HGB","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FHGB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FHGB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FHGB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FHGB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUDM","download_url":"https://codeload.github.com/THUDM/HGB/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247441052,"owners_count":20939239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T06:27:05.873Z","updated_at":"2025-04-06T06:09:14.335Z","avatar_url":"https://github.com/THUDM.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Heterogeneous Graph Benchmark\n\nRevisiting, benchmarking, and refining Heterogeneous Graph Neural Networks.\n\n**2023.3.2 update**: We make benchmark data including test set pulic. You can download data as follows:\n\n* Node Classification: https://drive.google.com/drive/folders/10-pf2ADCjq_kpJKFHHLHxr_czNNCJ3aX?usp=sharing\n* Link Prediction: https://drive.google.com/drive/folders/1RNOPAQ_jrHlNWMhfNT4wqclnW_O1PqYL?usp=sharing\n\nTherefore, you can get your metric scores locally. Actually, when you run Simple-HGN in [NC-benchmark](NC/benchmark/methods/baseline/) and [LP-benchmark](LP/benchmark/methods/baseline/), data will be downloaded automatically.\n\n***Warning***: As we have opened test data, you should try not to overfit or leak data during training. For example, the order of test data is not random permuted. If you use BatchNorm, you will get a biased norm value.\n\n## Roadmap\n\nWe organize our repo by task, and one sub-folder per task. Currently, we have four tasks, i.e., node classification (NC), link prediction (LP), knowledge-aware recommendation (Recom) and text classification (TC).\n\n### Revisiting\n\nThis part refers to Section 3 and Table 1 in our paper.\n\n* [HAN](./NC/HAN)\n* [GTN](./NC/GTN)\n* [RSHN](./NC/RSHN)\n* [HetGNN](./NC/HetGNN)\n* [MAGNN-nc](./NC/MAGNN) and [MAGNN-lp](./LP/benchmark/methods/MAGNN_ini)\n\n### Benchmarking and Refining\n\nThis part refers to Section 4,5,6 in our paper.\n\n* [Node classification benchmark](./NC/benchmark), [NC-baseline](./NC/benchmark/methods/baseline)\n* [Link prediction benchmark](./LP/benchmark), [LP-baseline](./LP/benchmark/methods/baseline)\n* [Knowledge-aware recommendation benchmark](./Recom), [Recom-baseline](./Recom/baseline)\n\n**We only make half of test labels public to prevent data leakage issues.** The public half data is to help you debug locally. If you want to obtain test scores, you need to **rename all the xxx_full to xxx in data folder** and submit your prediction to our [website](https://www.biendata.xyz/hgb/).\n\nFor node classification and link prediction tasks, you can submit online. But for recommendation task, since the prediction files are too large to submit, you have to test offline by yourself.\n\n**If you want to show your method on our official leaderboard on HGB website, you can make an issue on this repo. Once your code or paper is verified, your method will be displayed on the official leaderboard.**\n\n### FAQ\n\n1. **Where is the code for all HGNNs in paper for HGB?**\n\nFollow this roadmap in this repo:\n\n```\nNC or LP\n|--benchmark\n    |--methods\n        |--HGNNs\n```\n\n2. **How to take part in HGB?**\n\nSee [Get Started](https://www.biendata.xyz/hgb/#/about).\n\n3. **Why can't I obtain test score locally?**\n\nTo prevent data leakage issues, we randomly replace the test set labels. Therefore, you can only get test scores after you submit your prediction to biendata competitions.\n\n4. **What is the format of training data and submission files?**\n\nYou can read the instructions in biendata competitions in \"Data\" and \"Evaluation\" panel. What should be noticed is that, your prediction files should be on the top level of the zipped file.\n\nFor example, you should submit a zip like this:\n\n```\nsubmit.zip\n|--ACM_1.txt\n|--ACM_2.txt\n|--...\n```\n\nInstead of\n\n```\nsubmit.zip\n|--submit/\n    |--ACM_1.txt\n    |--ACM_2.txt\n    |--...\n```\n\nIt is recommended that zip a file using ```zip``` command rather than right click. Because subfolder may be automatically built for some operating systems when using right click.\n\n### More\n\n**This repo is actively under development.** Therefore, there are some extra experiments in this repo beyond our paper, such as graph-based text classification. For more information, see our [website](https://www.biendata.xyz/hgb/). Welcome contribute new tasks, datasets, methods to HGB!\n\nMoreover, we also have an implementation of Simple-HGN in [cogdl](https://github.com/THUDM/cogdl/tree/master/examples/simple_hgn).\n\n\n## Citation\n\n* **Title:** Are we really making much progress? Revisiting, benchmarking and refining the Heterogeneous Graph Neural Networks.\n* **Authors:** Qingsong Lv\\*, Ming Ding\\*, Qiang Liu, Yuxiang Chen, Wenzheng Feng, Siming He, Chang Zhou, Jianguo Jiang, Yuxiao Dong, Jie Tang.\n* **In proceedings:** KDD 2021.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthudm%2Fhgb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthudm%2Fhgb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthudm%2Fhgb/lists"}