{"id":20515451,"url":"https://github.com/snap-research/graphless-neural-networks","last_synced_at":"2025-04-06T04:12:54.819Z","repository":{"id":37489641,"uuid":"422002464","full_name":"snap-research/graphless-neural-networks","owner":"snap-research","description":"[ICLR 2022] Code for Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation (GLNN)","archived":false,"fork":false,"pushed_at":"2024-10-22T20:26:49.000Z","size":700,"stargazers_count":88,"open_issues_count":0,"forks_count":20,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-30T03:07:04.326Z","etag":null,"topics":["deep-learning","distillation","efficient-inference","gnn","graph-algorithm","graph-neural-networks","knowledge-distillation","pytorch","scalability"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snap-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-27T23:25:11.000Z","updated_at":"2024-12-30T22:26:26.000Z","dependencies_parsed_at":"2024-12-17T20:05:15.313Z","dependency_job_id":"6c140203-3647-4eaa-8ab3-f78441b32a05","html_url":"https://github.com/snap-research/graphless-neural-networks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-research%2Fgraphless-neural-networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-research%2Fgraphless-neural-networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-research%2Fgraphless-neural-networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-research%2Fgraphless-neural-networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snap-research","download_url":"https://codeload.github.com/snap-research/graphless-neural-networks/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247430874,"owners_count":20937874,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","distillation","efficient-inference","gnn","graph-algorithm","graph-neural-networks","knowledge-distillation","pytorch","scalability"],"created_at":"2024-11-15T21:21:49.468Z","updated_at":"2025-04-06T04:12:54.752Z","avatar_url":"https://github.com/snap-research.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- #region --\u003e\n# Graph-less Neural Networks (GLNN)\n\nCode for [Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation](https://arxiv.org/pdf/2110.08727.pdf) by [Shichang Zhang](https://shichangzh.github.io/), [Yozen Liu](https://research.snap.com/team/yozen-liu/), [Yizhou Sun](http://web.cs.ucla.edu/~yzsun/), and [Neil Shah](http://nshah.net/).\n\n\n## Overview\n### Distillation framework\n\u003cp align=\"center\"\u003e\n  \u003cbr /\u003e\n  \u003cimg src=\"imgs/glnn.png\" width=\"800\"\u003e\n  \u003cbr /\u003e\n\u003c/p\u003e\n\n\n### Accuracy vs. inference time on the `ogbn-products` dataset\n\n\u003cp align=\"center\"\u003e\n  \u003cbr /\u003e\n  \u003cimg src=\"imgs/trade_off.png\" width=\"800\"\u003e\n  \u003cbr /\u003e\n\u003c/p\u003e\n\n\n## Getting Started\n\n### Setup Environment\n\nWe use conda for environment setup. You can use\n\n`bash ./prepare_env.sh`\n\nwhich will create a conda environment named `glnn` and install relevant requirements (from `requirements.txt`).   For simplicity, we use CPU-based `torch` and `dgl` versions in this guide, as specified in requirements.  To run experiments with CUDA, please install `torch` and `dgl` with proper CUDA support, remove them from `requirements.txt`, and properly set the `--device` argument in the scripts. See https://pytorch.org/ and https://www.dgl.ai/pages/start.html for more installation details. \n\nBe sure to activate the environment with\n\n`conda activate glnn`\n\nbefore running experiments as described below.\n\n\n\n### Preparing datasets\nTo run experiments for dataset used in the paper, please download from the following links and put them under `data/` (see below for instructions on organizing the datasets).\n\n- *CPF data* (`cora`, `citeseer`, `pubmed`, `a-computer`, and `a-photo`): Download the '.npz' files from [here](https://github.com/BUPT-GAMMA/CPF/tree/master/data/npz). Rename `amazon_electronics_computers.npz` and `amazon_electronics_photo.npz` to `a-computer.npz` and `a-photo.npz` respectively.\n\n- *OGB data* (`ogbn-arxiv` and `ogbn-products`): Datasets will be automatically downloaded when running the `load_data` function in `dataloader.py`. More details [here](https://ogb.stanford.edu/).\n\n- *BGNN data* (`house_class` and `vk_class`): Follow the instructions [here](https://github.com/dmlc/dgl/tree/473d5e0a4c4e4735f1c9dc9d783e0374328cca9a/examples/pytorch/bgnn) and download dataset pre-processed in DGL format from [here](https://www.dropbox.com/s/verx1evkykzli88/datasets.zip).\n\n- *NonHom data* (`penn94` and `pokec`): Follow the instructions [here](https://github.com/CUAI/Non-Homophily-Benchmarks) to download the `penn94` dataset and its splits. The `pokec` dataset will be automatically downloaded when running the `load_data` function in `dataloader.py`.\n\n- Your favourite datasets: download and add to the `load_data` function in `dataloader.py`.\n\n\n### Usage\n\nTo quickly train a teacher model you can run `train_teacher.py` by specifying the experiment setting, i.e. transductive (`tran`) or inductive (`ind`), teacher model, e.g. `GCN`, and dataset, e.g. `cora`, as per the example below.\n\n```\npython train_teacher.py --exp_setting tran --teacher GCN --dataset cora\n```\n\nTo quickly train a student model with a pretrained teacher you can run `train_student.py` by specifying the experiment setting, teacher model, student model, and dataset like the example below. Make sure you train the teacher using the `train_teacher.py` first and have its result stored in the correct path specified by `--out_t_path`.\n\n```\npython train_student.py --exp_setting ind --teacher SAGE --student MLP --dataset citeseer --out_t_path outputs\n```\n\nFor more examples, and to reproduce results in the paper, please refer to scripts in `experiments/` as below.\n\n```\nbash experiments/sage_cpf.sh\n```\n\nTo extend GLNN to your own model, you may do one of the following.\n- Add your favourite model architectures to the `Model` class in `model.py`. Then follow the examples above.\n- Train teacher model and store its output (log-probabilities). Then train the student by `train_student.py` with the correct `--out_t_path`.\n\n\n## Results\n\nGraphSAGE vs. MLP vs. GLNN under the production setting described in the paper (transductive and inductive combined). Delta_MLP (Delta_GNN) represents difference between the GLNN and the MLP (GNN). Results show classification accuracy (higher is better); Delta_GNN \u003e 0 indicates GLNN outperforms GNN. We observe that GLNNs always improve from MLPs by large margins and achieve competitive results as GNN on 6/7 datasets. Please see Table 3 in the paper for more details.  \n\n| Datasets   | GNN(SAGE)      | MLP          | GLNN           | Delta_MLP       | Delta_GNN         |\n|------------|----------------|--------------|----------------|-----------------|-------------------|\n| Cora       | **79.29**      | 58.98        | 78.28          | 19.30 (32.72\\%) | -1.01 (-1.28\\%)   |\n| Citseer    | 68.38          | 59.81        | **69.27**      | 9.46 (15.82\\%)  | 0.89 (1.30\\%)     |\n| Pubmed     | **74.88**      | 66.80        | 74.71          | 7.91 (11.83\\%)  | -0.17 (-0.22\\%)   |\n| A-computer | 82.14          | 67.38        | **82.29**      | 14.90 (22.12\\%) | 0.15 (0.19\\%)     |\n| A-photo    | 91.08          | 79.25        | **92.38**      | 13.13 (16.57\\%) | 1.30 (1.42\\%)     |\n| Arxiv      | **70.73**      | 55.30        | 65.09          | 9.79 (17.70\\%)  | -5.64 (-7.97\\%)   |\n| Products   | **76.60**      | 63.72        | 75.77          | 12.05 (18.91\\%) | -0.83 (-1.09\\%)   |\n\n\u003c!-- \n| Datasets   | Eval | SAGE           | MLP/MLP+     | GLNN/GLNN+     | $\\Delta_{MLP}$  | $\\Delta_{GNN}$    |\n|------------|------|----------------|--------------|----------------|-----------------|-------------------|\n| Cora       | prod | **79.29**      | 58.98        | 78.28          | 19.30 (32.72\\%) | -1.01 (-1.28\\%)   |\n|            | ind  | 81.33 ± 2.19   | 59.09 ± 2.96 | 73.82 ± 1.93   | 14.73 (24.93\\%) | -7.51 (-9.23\\%)   |\n|            | tran | 78.78 ± 1.92   | 58.95 ± 1.66 | 79.39 ± 1.64   | 20.44 (34.66\\%) | 0.61 (0.77\\%)     |\n| Citseer    | prod | 68.38          | 59.81        | **69.27**      | 9.46 (15.82\\%)  | 0.89 (1.30\\%)     |\n|            | ind  | 69.75 ± 3.59   | 60.06 ± 5.00 | 69.25 ± 2.25   | 9.19 (15.30\\%)  | -0.5 (-0.7\\%)     |\n|            | tran | 68.04 ± 3.34   | 59.75 ± 2.48 | 69.28 ± 3.12   | 9.63 (15.93\\%)  | 1.24 (1.82\\%)     |\n| Pubmed     | prod | **74.88**      | 66.80        | 74.71          | 7.91 (11.83\\%)  | -0.17 (-0.22\\%)   |\n|            | ind  | 75.26 ± 2.57   | 66.85 ± 2.96 | 74.30 ± 2.61   | 7.45 (11.83\\%)  | -0.96 (-1.27\\%)   |\n|            | tran | 74.78 ± 2.22   | 66.79 ± 2.90 | 74.81 ± 2.39   | 8.02 (12.01\\%)  | 0.03 (0.04\\%)     |\n| A-computer | prod | 82.14          | 67.38        | **82.29**      | 14.90 (22.12\\%) | 0.15 (0.19\\%)     |\n|            | ind  | 82.08 ± 1.79   | 67.84 ± 1.78 | 80.92 ± 1.36   | 13.08 (19.28\\%) | -1.16 (-1.41\\%)   |\n|            | tran | 82.15 ± 1.55   | 67.27 ± 1.36 | 82.63 ± 1.40   | 15.36 (22.79\\%) | 0.48 (0.58\\%)     |\n| A-photo    | prod | 91.08          | 79.25        | **92.38**      | 13.13 (16.57\\%) | 1.30 (1.42\\%)     |\n|            | ind  | 91.50 ± 0.79   | 79.44 ± 1.72 | 91.18 ± 0.81   | 11.74 (14.78\\%) | -0.32 (-0.35\\%)   |\n|            | tran | 90.80 ± 0.77   | 79.20 ± 1.64 | 92.68 ± 0.56   | 13.48 (17.01\\%) | 1.70 (1.87\\%)     |\n| Arxiv      | prod | **70.73**      | 55.30        | 65.09          | 9.79 (17.70\\%)  | -5.64 (-7.97\\%)   |\n|            | ind  | 70.64 ± 0.67   | 55.40 ± 0.56 | 60.48 ± 0.46   | 4.3 (7.76\\%)    | -10.94 (-15.49\\%) |\n|            | tran | 70.75 ± 0.27   | 55.28 ± 0.49 | 71.46 ± 0.33   | 11.16 (20.18\\%) | -4.31 (-6.09\\%)   |\n| Products   | prod | **76.60**      | 63.72        | 75.77          | 12.05 (18.91\\%) | -0.83 (-1.09\\%)   |\n|            | ind  | 76.89 ± 0.53   | 63.70 ± 0.66 | 75.16 ± 0.34   | 11.44 (17.96\\%) | -1.73 (-2.25\\%)   |\n|            | tran | 76.53 ±0.55    | 63.73 ± 0.69 | 75.92 ± 0.61   | 12.20 (19.15\\%) | -0.61 (-0.79\\%)   |\n --\u003e\n\n## Citation\n\nIf you find our work useful, please cite the following:\n\n```BibTeX\n@inproceedings{zhang2021graphless,\n      title={Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation}, \n      author={Shichang Zhang and Yozen Liu and Yizhou Sun and Neil Shah},\n      booktitle={International Conference on Learning Representations}\n      year={2022},\n      url={https://arxiv.org/abs/2110.08727}\n}\n```\n\n## Contact Us\n\nPlease open an issue or contact `shichang@cs.ucla.edu` if you have any questions.\n\n\u003c!-- #endregion --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnap-research%2Fgraphless-neural-networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnap-research%2Fgraphless-neural-networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnap-research%2Fgraphless-neural-networks/lists"}