{"id":18852803,"url":"https://github.com/tech-srl/bottleneck","last_synced_at":"2025-06-18T22:35:14.538Z","repository":{"id":41507357,"uuid":"292922511","full_name":"tech-srl/bottleneck","owner":"tech-srl","description":"Code for the paper: \"On the Bottleneck of Graph Neural Networks and Its Practical Implications\"","archived":false,"fork":false,"pushed_at":"2022-04-25T01:22:47.000Z","size":864,"stargazers_count":94,"open_issues_count":0,"forks_count":21,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-13T15:09:05.282Z","etag":null,"topics":["bottleneck","gnns","graph","networks","neural","of","on","over-squashing","oversquashing","the"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tech-srl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-09-04T18:44:19.000Z","updated_at":"2024-12-27T22:32:15.000Z","dependencies_parsed_at":"2022-08-26T08:00:21.703Z","dependency_job_id":null,"html_url":"https://github.com/tech-srl/bottleneck","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tech-srl%2Fbottleneck","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tech-srl%2Fbottleneck/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tech-srl%2Fbottleneck/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tech-srl%2Fbottleneck/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tech-srl","download_url":"https://codeload.github.com/tech-srl/bottleneck/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248860191,"owners_count":21173381,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bottleneck","gnns","graph","networks","neural","of","on","over-squashing","oversquashing","the"],"created_at":"2024-11-08T03:41:36.943Z","updated_at":"2025-04-14T10:22:32.072Z","avatar_url":"https://github.com/tech-srl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# On the Bottleneck of Graph Neural Networks and its Practical Implications\n\nThis is the official implementation of the paper: [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://openreview.net/pdf?id=i80OPhOCVH2) (ICLR'2021),\nwhich introduces the **over-squashing** problem of GNNs.\n\nBy [Uri Alon](http://urialon.cswp.cs.technion.ac.il/) and [Eran Yahav](http://www.cs.technion.ac.il/~yahave/).\nSee also the [[video]](https://youtu.be/vrLsEwzZTCQ), [[poster]](https://urialon.cswp.cs.technion.ac.il/wp-content/uploads/sites/83/2021/03/bottleneck_poster.pdf) and [[slides]](https://urialon.cswp.cs.technion.ac.il/wp-content/uploads/sites/83/2020/07/bottleneck_slides.pdf). \n\nthis repository is divided into three sub-projects:\n\n1. The subdirectory `tf-gnn-samples` is a clone of \n[https://github.com/microsoft/tf-gnn-samples](https://github.com/microsoft/tf-gnn-samples) by Brockschmidt (ICML'2020). \nThis project can be used to reproduce the QM9 and VarMisuse experiments of Section 4.2 and 4.2 in the paper.\nThis sub-project depends on TensorFlow 1.13.\nThe instructions for our clone are the same as their original code, except that reproducing our experiments \n(the QM9 dataset and VarMisuse) can be done by running the\nscript `tf-gnn-samples/run_qm9_benchs_fa.py` or `tf-gnn-samples/run_varmisuse_benchs_fa.py` instead of their original scripts.\nFor additional dependencies and instructions, see their original README: \n[https://github.com/microsoft/tf-gnn-samples/blob/master/README.md](https://github.com/microsoft/tf-gnn-samples/blob/master/README.md).\nThe main modification that we performed is using a Fully-Adjacent layer as the last \nGNN layer and we describe in our paper. \n2. The subdirectory `gnn-comparison` is a clone of [https://github.com/diningphil/gnn-comparison](https://github.com/diningphil/gnn-comparison)\nby Errica et al. (ICLR'2020). \nThis project can be used to reproduce the biological experiments (Section 4.3, the ENZYMES and NCI1 datasets).\nThis sub-project depends on PyTorch 1.4 and Pytorch-Geometric.\nFor additional dependencies and instructions, see their original README: \n[https://github.com/diningphil/gnn-comparison/blob/master/README.md](https://github.com/diningphil/gnn-comparison/blob/master/README.md).\nThe instructions for our clone are the same, except that we added an additional flag to every `config_*.yml` file, called `last_layer_fa`, \nwhich is set to `True` by default, and reproduces our experiments.\nThe main modification that we performed is using a Fully-Adjacent layer as the last \nGNN layer.\n3. The main directory (in which this file resides) can be used to reproduce the experiments of \nSection 4.1 in the paper, for the \"Tree-NeighborsMatch\" problem. The rest of this README file includes the \ninstructions for this main directory. \nThis repository can be used to reproduce the experiments of \n\nThis project was designed to be useful in experimenting with new GNN architectures and new solutions for the over-squashing problem. \n\nFeel free to open an issue with any questions.\n\n\n# The Tree-NeighborsMatch problem\n![alt text](images/fig5.png \"Figure 5 from the paper\")\n\n## Requirements\n\n### Dependencies\nThis project is based on PyTorch 1.4.0 and the [PyTorch Geometric](https://pytorch-geometric.readthedocs.io/) library.\n* First, install PyTorch from the official website: [https://pytorch.org/](https://pytorch.org/).\n* Then install PyTorch Geometric: [https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html](https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html)\n* Eventually, run the following to verify that all dependencies are satisfied:\n```setup\npip install -r requirements.txt\n```\n\nThe `requirements.txt` file lists the additional requirements.\n However, PyTorch Geometric might requires manual installation, and we thus recommend to use the \n`requirements.txt` file only afterward.\n\n\nVerify that importing the dependencies goes without errors:\n```\npython -c 'import torch; import torch_geometric'\n```\n\n\n\n### Hardware\nTraining on large trees (depth=8) might require ~60GB of RAM and about 10GB of GPU memory.\nGPU memory can be compromised by using a smaller batch size and using the `--accum_grad` flag.\n\nFor example, instead of running:\n```\npython main.py --batch_size 1024 --type GGNN\n```\n\nThe following uses gradient accumulation, and takes less GPU memory:\n```\npython main.py --batch_size 512 --accum_grad 2 --type GGNN\n```\n\n## Reproducing Experiments\n\nTo run a single experiment from the paper, run:\n\n```\npython main.py --help\n```\nAnd see the available flags.\nFor example, to train a GGNN with depth=4, run:\n```\npython main.py --task DICTIONARY --eval_every 1000 --depth 4 --num_layers 5 --batch_size 1024 --type GGNN\n```  \n\nTo train a GNN across all depths, run one of the following:\n```\npython run-gcn-2-8.py\npython run-gat-2-8.py\npython run-ggnn-2-8.py\npython run-gin-2-8.py\n```\n\n## Results\n\nThe results of running the above scripts are (Section 4.1 in the paper):\n\n\n![alt text](images/fig3.png \"Figure 3 from the paper\")\n\n\nr:   | 2   \t| 3   \t| 4    \t| 5    \t| 6    \t| 7    \t| 8    \t|\n------\t|-----\t|-----\t|------\t|------\t|------\t|------\t|------\t|\n **GGNN** \t| 1.0 \t| 1.0 \t| 1.0  \t| 0.60 \t| 0.38 \t| 0.21 \t| 0.16 \t|\n **GAT**  \t| 1.0 \t| 1.0 \t| 1.0  \t| 0.41 \t| 0.21 \t| 0.15 \t| 0.11 \t|\n **GIN**  \t| 1.0 \t| 1.0 \t| 0.77 \t| 0.29 \t| 0.20 \t|      \t|      \t|\n **GCN**  \t| 1.0 \t| 1.0 \t| 0.70 \t| 0.19 \t| 0.14 \t| 0.09 \t| 0.08 \t|\n\n## Experiment with other GNN types\nTo experiment with other GNN types:\n* Add the new GNN type to the `GNN_TYPE` enum [here](common.py#L34), for example: `MY_NEW_TYPE = auto()`\n* Add another `elif self is GNN_TYPE.MY_NEW_TYPE:` to instantiate the new GNN type object [here](common.py#L47)\n* Use the new type as a flag for the `main.py` file:\n```\npython main.py --type MY_NEW_TYPE ...\n```\n\n## Citation\nIf you want to cite this work, please use this bibtex entry:\n```\n@inproceedings{\n    alon2021on,\n    title={On the Bottleneck of Graph Neural Networks and its Practical Implications},\n    author={Uri Alon and Eran Yahav},\n    booktitle={International Conference on Learning Representations},\n    year={2021},\n    url={https://openreview.net/forum?id=i80OPhOCVH2}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftech-srl%2Fbottleneck","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftech-srl%2Fbottleneck","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftech-srl%2Fbottleneck/lists"}