{"id":20410045,"url":"https://github.com/thumnlab/nas-bench-graph","last_synced_at":"2026-02-25T23:06:53.100Z","repository":{"id":63446909,"uuid":"501641288","full_name":"THUMNLab/NAS-Bench-Graph","owner":"THUMNLab","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-29T09:27:30.000Z","size":5767,"stargazers_count":24,"open_issues_count":2,"forks_count":2,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-09-17T10:43:29.928Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUMNLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-09T12:21:19.000Z","updated_at":"2025-04-22T00:59:42.000Z","dependencies_parsed_at":"2023-01-31T08:31:23.756Z","dependency_job_id":null,"html_url":"https://github.com/THUMNLab/NAS-Bench-Graph","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/THUMNLab/NAS-Bench-Graph","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FNAS-Bench-Graph","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FNAS-Bench-Graph/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FNAS-Bench-Graph/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FNAS-Bench-Graph/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUMNLab","download_url":"https://codeload.github.com/THUMNLab/NAS-Bench-Graph/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUMNLab%2FNAS-Bench-Graph/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29844845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-25T22:37:40.667Z","status":"ssl_error","status_checked_at":"2026-02-25T22:37:25.960Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T05:44:43.420Z","updated_at":"2026-02-25T23:06:53.084Z","avatar_url":"https://github.com/THUMNLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NAS-Bench-Graph\n\nThis repository provides the official codes and all evaluated architectures for NAS-Bench-Graph, a tailored benchmark for graph neural architecture search.\n\nYijian Qin, Ziwei Zhang, Xin Wang, Zeyang Zhang, Wenwu Zhu, [NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search](https://openreview.net/pdf?id=bBff294gqLp) (NeurIPS 2022)\n\n## Install from PyPI\nYou can directly install our benchmark by `pip install nas_bench_graph`\n\n## Usage \nFirst, read the benchmark of a certain dataset by specifying the name. The nine supported datasets are: cora, citeseer, pubmed, cs, physics, photo, computers, arxiv, and proteins. For example, for the Cora dataset:\n```\nfrom nas_bench_graph import lightread\nbench = lightread('cora')\n```\nThe data is stored as a `dict` in Python.\n\nThen, an architecture needs to be specified by its macro space and operations.\nWe consider the macro space as a directed acyclic graph (DAG) and constrain the DAG to have only one input node for each intermediate node. Therefore, the macro space can be specificed by a list of integers, indicating the input node index for each computing node (0 for the raw input, 1 for the first computing node, etc.). Then, the operations can be specified by a list of strings with the same length. For example, we provide the code to specify the architecture in the following figure:\n![arch](https://user-images.githubusercontent.com/17705534/173767528-eda1bc64-f4d8-4da1-a0e9-8470f55ccc6a.png)\n\n```\nfrom nas_bench_graph import Arch\narch = Arch([0, 1, 2, 1], ['gcn', 'gin', 'fc', 'cheb'])\n# 0 means the inital computing node is connected to the input node\n# 1 means the next computing node is connected to the first computing node\n# 2 means the next computing node is connected to the second computing node \n# 1 means there is another computing node connected to the first computing node\n```\n\nNotice that we assume all leaf nodes (i.e., nodes without descendants) are connected to the output, so there is no need to specific the output node. \n\nBesides, the list can be specified in any order, e.g., the following code can specific the same architecture:\n```\narch = Arch([0, 1, 1, 2], ['gcn', 'cheb', 'gin', 'fc'])\n```\n\nThe benchmark data can be obtained by a look-up table. In this repository, we only provide the validation and test performance, the latency, and the number of parameters as follows:\n\n```\ninfo = bench[arch.valid_hash()]\ninfo['valid_perf']   # validation performance\ninfo['perf']         # test performance\ninfo['latency']      # latency\ninfo['para']         # number of parameters\n```\n\nFor the complete benchmark, please downloadfrom https://figshare.com/articles/dataset/NAS-bench-Graph/20070371, which contains the training/validation/testing performance at each epoch. Since we run each dataset with three random seeds, each dataset has 3 files, e.g.,\n\n```\nfrom nas_bench_graph import read\nbench = read('cora0.bench')   # cora1.bench and cora2.bench \n```\n\nThe full metric for any epoch can be obtained as follows.\n```\ninfo = bench[arch.valid_hash()]\nepoch = 50\ninfo['dur'][epoch][0]   # training performance\ninfo['dur'][epoch][1]   # validation performance\ninfo['dur'][epoch][2]   # testing performance\ninfo['dur'][epoch][3]   # training loss\ninfo['dur'][epoch][4]   # validation loss\ninfo['dur'][epoch][5]   # testing loss\ninfo['dur'][epoch][6]   # best performance\n```\n\n## Example usage of NNI and AutoGL\nNAS-Bench-Graph can be used together with other libraries such AutoGL and NNI.\n\nFor the usage of [AutoGL](https://github.com/THUMNLab/AutoGL), please refer to the [agnn branch](https://github.com/THUMNLab/AutoGL/tree/agnn).\n\nYou can also refer to `runnni.py` to use the benchmark together with [NNI](https://github.com/microsoft/nni/).\n\n\n## Citation\nIf you find that NAS-Bench-Graph helps your research, please consider citing it:\n\n```\n@inproceedings{qin2022nas,\n  title     = {NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search},\n  author    = {Qin, Yijian and Zhang, Ziwei and Wang, Xin and Zhang, Zeyang and Zhu, Wenwu},\n  booktitle = {Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track}\n  year      = {2022}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthumnlab%2Fnas-bench-graph","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthumnlab%2Fnas-bench-graph","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthumnlab%2Fnas-bench-graph/lists"}