{"id":24358809,"url":"https://github.com/bupt-gamma/cpf","last_synced_at":"2025-04-10T05:51:55.289Z","repository":{"id":103997567,"uuid":"330101580","full_name":"BUPT-GAMMA/CPF","owner":"BUPT-GAMMA","description":"The official code of WWW2021 paper: Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework","archived":false,"fork":false,"pushed_at":"2021-07-09T09:38:14.000Z","size":23404,"stargazers_count":74,"open_issues_count":1,"forks_count":20,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-24T07:05:53.495Z","etag":null,"topics":["graph-neural-networks","knowledge-distillation"],"latest_commit_sha":null,"homepage":"https://arxiv.org/pdf/2103.02885.pdf","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BUPT-GAMMA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-01-16T06:41:03.000Z","updated_at":"2025-01-11T17:30:12.000Z","dependencies_parsed_at":"2023-07-10T15:32:24.395Z","dependency_job_id":null,"html_url":"https://github.com/BUPT-GAMMA/CPF","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUPT-GAMMA%2FCPF","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUPT-GAMMA%2FCPF/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUPT-GAMMA%2FCPF/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUPT-GAMMA%2FCPF/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BUPT-GAMMA","download_url":"https://codeload.github.com/BUPT-GAMMA/CPF/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248166928,"owners_count":21058480,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph-neural-networks","knowledge-distillation"],"created_at":"2025-01-18T20:23:02.346Z","updated_at":"2025-04-10T05:51:55.269Z","avatar_url":"https://github.com/BUPT-GAMMA.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CPF\n\nThe official code of WWW2021 paper: Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-fixed-10-node-per)](https://paperswithcode.com/sota/node-classification-on-cora-fixed-10-node-per)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-fixed-5-node-per)](https://paperswithcode.com/sota/node-classification-on-cora-fixed-5-node-per)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-3)](https://paperswithcode.com/sota/node-classification-on-cora-3)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-1)](https://paperswithcode.com/sota/node-classification-on-cora-1)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-05)](https://paperswithcode.com/sota/node-classification-on-cora-05)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-amz-computers)](https://paperswithcode.com/sota/node-classification-on-amz-computers?p=extract-the-knowledge-of-graph-neural)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-amz-photo)](https://paperswithcode.com/sota/node-classification-on-amz-photo?p=extract-the-knowledge-of-graph-neural)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-with-public-split)](https://paperswithcode.com/sota/node-classification-on-cora-with-public-split?p=extract-the-knowledge-of-graph-neural)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-citeseer-with-public)](https://paperswithcode.com/sota/node-classification-on-citeseer-with-public?p=extract-the-knowledge-of-graph-neural)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-pubmed-with-public)](https://paperswithcode.com/sota/node-classification-on-pubmed-with-public?p=extract-the-knowledge-of-graph-neural)\n\n## Getting Started\n\n### Requirements\n\n- Python version \u003e= 3.6\n- PyTorch version \u003e= 1.7.1\n- DGL\n- Optuna (optional)\n\n## Usage\n\n### Quick start\n\n1. use `python train_dgl.py --dataset=XXX --teacher=XXX`  to run teacher model.\n2. use `python spawn_worker.py --dataset=XXX --teacher=XXX ` to run student model, we provide our hyper-parameters setting as reported in our paper, and an AutoML version for hyper-parameters search. (Our code supports Optuna to search best hyper-parameters for knowledge distillation. You can use `--automl`  to run Optuna code.)\n\n## Add your own datasets\n\nYou can add your own datasets to folder `data`, the formats should accord to DGL requirements.\n\n## Add your own models\n\nYou can add your own teacher or student model by adding them into folder `models`, and following the format of model run.\n\n## Results\n\nThere are some results on GCN teacher model, with different datasets and student varients. More results can be seen in our paper.\n\n| Datasets    | GCN (Teacher) | CPF-ind (Student) | CPF-tra (Student) | improvement |\n| ----------- | ------------- | ----------------- | ----------------- | ----------- |\n| Cora        | 0.8244        | **0.8576**        | 0.8567            | 4.0%        |\n| Citeseer    | 0.7110        | 0.7619            | **0.7652**        | 7.6%        |\n| Pubmed      | 0.7804        | 0.8080            | **0.8104**        | 3.8%        |\n| A-Computers | 0.8318        | **0.8443**        | **0.8443**        | 1.5%        |\n| A-Photo     | 0.9072        | **0.9317**        | 0.9248            | 2.7%        |\n\n## Benchmark Rankings\n\nThere are results use several models run on different benchmark datasets. Our experiments settings are available in the following form and the `pwc.conf.yaml` file. For simple usage, please try AutoML for hyper-parameters search.\n\nNote: \n\n- Remember to change the load data function to load_citation when running public split benchmarks. \n- Use original load data function when running AMZ datasets, remember to slice the test sets to corresponding size.\n\n|                          Benchmark                           | Model         | Acc    | layer | emb_dim | feat_drop | attn_drop | lr   | wd   |\n| :----------------------------------------------------------: | ------------- | ------ | ----- | ------- | --------- | --------- | ---- | ---- |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-fixed-10-node-per)](https://paperswithcode.com/sota/node-classification-on-cora-fixed-10-node-per) | CPF-tra-GCNII | 84.1%  | 6     | 16      | 0.2       | 0.5       | 5e-3 | 1e-2 |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-fixed-5-node-per)](https://paperswithcode.com/sota/node-classification-on-cora-fixed-5-node-per) | CPF-tra-APPNP | 80.26% | 8     | 32      | 0.2       | 0.2       | 5e-3 | 5e-4 |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-3)](https://paperswithcode.com/sota/node-classification-on-cora-3) | CPF-tra-GCNII | 84.18% | 9     | 8       | 0.5       | 0.8       | 5e-3 | 1e-2 |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-1)](https://paperswithcode.com/sota/node-classification-on-cora-1) | CPF-ind-APPNP | 80.24% | 8     | 16      | 0.8       | 0.2       | 5e-3 | 1e-2 |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-05)](https://paperswithcode.com/sota/node-classification-on-cora-05) | CPF-ind-APPNP | 77.3%  | 7     | 32      | 0.8       | 0.2       | 1e-3 | 1e-3 |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-amz-computers)](https://paperswithcode.com/sota/node-classification-on-amz-computers?p=extract-the-knowledge-of-graph-neural) | CPF-ind-GAT   | 85.5%  | 8     | 16      | 0.2       | 0.5       | 1e-3 | 1e-2 |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-amz-photo)](https://paperswithcode.com/sota/node-classification-on-amz-photo?p=extract-the-knowledge-of-graph-neural) | CPF-ind-GAT   | 94.1%  | 9     | 32      | 0.5       | 0.5       | 1e-2 | 1e-2 |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-cora-with-public-split)](https://paperswithcode.com/sota/node-classification-on-cora-with-public-split?p=extract-the-knowledge-of-graph-neural) | CPF-ind-APPNP | 85.3%  | 10    | 64      | 0.8       | 0.8       | 5e-3 | 5e-4 |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-citeseer-with-public)](https://paperswithcode.com/sota/node-classification-on-citeseer-with-public?p=extract-the-knowledge-of-graph-neural) | CPF-ind-APPNP | 74.6%  | 6     | 64      | 0.5       | 0.5       | 5e-3 | 1e-2 |\n| [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/extract-the-knowledge-of-graph-neural/node-classification-on-pubmed-with-public)](https://paperswithcode.com/sota/node-classification-on-pubmed-with-public?p=extract-the-knowledge-of-graph-neural) | CPF-tra-GCNII | 83.2%  | 8     | 16      | 0.8       | 0.8       | 1e-2 | 5e-4 |\n\n## Cite\n\nPlease cite our paper if you use this code in your own work:\n\n```\n@inproceedings{yang2021extract,\n  title={Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework},\n  author={Cheng Yang and Jiawei Liu and Chuan Shi},\n  booktitle={Proceedings of The Web Conference 2021 (WWW ’21)},\n  publisher={ACM},\n  year={2021}\n}\n```\n\n## Contact Us\n\nPlease open an issue or contact Liu_Jiawei@bupt.edu.cn with any questions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbupt-gamma%2Fcpf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbupt-gamma%2Fcpf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbupt-gamma%2Fcpf/lists"}