{"id":13653178,"url":"https://github.com/seongjunyun/Graph_Transformer_Networks","last_synced_at":"2025-04-23T06:31:26.395Z","repository":{"id":36394137,"uuid":"207291691","full_name":"seongjunyun/Graph_Transformer_Networks","owner":"seongjunyun","description":"Graph Transformer Networks (Authors' PyTorch implementation for the NeurIPS 19 paper)","archived":false,"fork":false,"pushed_at":"2023-02-15T16:19:09.000Z","size":19290,"stargazers_count":974,"open_issues_count":17,"forks_count":178,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-11-10T04:36:23.480Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seongjunyun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-09-09T11:18:13.000Z","updated_at":"2024-11-08T09:14:42.000Z","dependencies_parsed_at":"2024-01-12T20:04:14.201Z","dependency_job_id":null,"html_url":"https://github.com/seongjunyun/Graph_Transformer_Networks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seongjunyun%2FGraph_Transformer_Networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seongjunyun%2FGraph_Transformer_Networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seongjunyun%2FGraph_Transformer_Networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seongjunyun%2FGraph_Transformer_Networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seongjunyun","download_url":"https://codeload.github.com/seongjunyun/Graph_Transformer_Networks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250384945,"owners_count":21421824,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T02:01:06.865Z","updated_at":"2025-04-23T06:31:21.383Z","avatar_url":"https://github.com/seongjunyun.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"# Graph Transformer Networks\nThis repository is the implementation of [Graph Transformer Networks(GTN)](https://arxiv.org/abs/1911.06455) and [Fast Graph Transformer Networks with Non-local Operations (FastGTN)](https://pdf.sciencedirectassets.com/271125/1-s2.0-S0893608022X00075/1-s2.0-S0893608022002003/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHkaCXVzLWVhc3QtMSJHMEUCIQC5JKs%2BBKb0MPqBvG9De58QPzs52B2jcbCKrlS8Ahtx8wIgXRKM1pwytn%2Fg3I%2BpRawictl9bHkbukMC1av%2FjfvDGakqzAQIIhAFGgwwNTkwMDM1NDY4NjUiDCgbkDuexMDYYCd7eyqpBPka68pkQYZf%2FozGqEf8msefhrCK%2Bt3vWHggdPa9o1Nfahc6uf6RvB1iUr2qGE%2BmDtqkZgHllcBwjK9N5oGEEEz%2FXR%2Bo5GNUcWU8OyyAxpunLswUrabAPi5%2FKI6EKT8k0f%2BfxajwSizgKlEwYcOa5SPCi62e87GCI1hzQlTvjhub54aA6JkzMGYBrAODaTH6fxTW2bL8kgXFQCjMwsEy81yyh%2F6xWZ1b9ybHrdobu3ivHDiN1n7oXoW1o7E9ZPg2Tm4%2B9iFLeBF49QZlsVTxMb%2BnhSDJYoEmnyEM3OJRCuAXex%2F1Xhu0GzsvhgR9Ahaofbx9b2XNK8926l4eFW7sO9Q2Bu4VqJ4jqhYI74CYJA5t1BE49jMaNCZs%2Bl163Mnl4GuzTDZiTtg0GOnnDf5HZ2n4DSP0sTGYK5QWSQRbMBwI2s5eB1mb0gzIimHglsd1TfGhav%2BtD3X6149li6LUQQy9gxrEQLWhJY%2Frggl6lbJ2yb5yanW42sf2iVdcmX3WpevKqyuGRDo4TWN59D6h2T1xC7f5NQ8uRW4wTFDRZ%2FUZjX3gyVwE2qquYR%2BbMVwvD4R6fbi9AONo%2BU68fEZNcYJQ5igRWAWtZk6cGQno8XPZbnYUYfAO1Q9WWagJq%2FJC2eDPVJYb330BrV3rbmpasvmWUnkJwUYVNhAeSGp9AiS%2BWCNR3Wo4qMDoPhULj31UJ2967m9m1HCkJ%2FOWwOlT4zDuEZmDBGefvysw69aZmQY6qQEm7If21VPh69neRb1%2BJGDySZcFw53B7jrt%2BI1ERmLBsDV9%2B3cPMBiFwRltFW1aT%2FDFRBdxNordu2sB4UD7FzMbm1a6KOlbrntZQntaiNS9S4gM0DoMccmTZgRkHlmkUr%2FjxGyVntJ6EL7pTvFbxNPeic6o5v8UZs%2B%2BpBcs2cXJ%2BCrtHucNoLkf6RlwPqr3PfQNERNEuMN2hhkRsByEhBbeQsu22ViDbc9C\u0026X-Amz-Algorithm=AWS4-HMAC-SHA256\u0026X-Amz-Date=20220918T015554Z\u0026X-Amz-SignedHeaders=host\u0026X-Amz-Expires=300\u0026X-Amz-Credential=ASIAQ3PHCVTY6A6RHQPB%2F20220918%2Fus-east-1%2Fs3%2Faws4_request\u0026X-Amz-Signature=0f0752d788b3d905a14a4f0b5bf56efa71a37c93fea77d5671ec170b55e28027\u0026hash=8fdbb1b3d93f684d5528e2ae717306bb9eaae0380ae6fb68b81361df1f1a54f0\u0026host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61\u0026pii=S0893608022002003\u0026tid=spdf-9cbe5006-8e6c-4bc8-a955-6f50a80987f5\u0026sid=26eb78364bb5a0466f4bf51-3ba867604046gxrqa\u0026type=client\u0026ua=4d52035053565707575c\u0026rr=74c6764b6874c08c).\n\n\u003e Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, Hyunwoo J. Kim, Graph Transformer Networks, In Advances in Neural Information Processing Systems (NeurIPS 2019).\n\n\u003e Seongjun Yun, Minbyul Jeong, Sungdong Yoo, Seunghun Lee, Sean S, Yi, Raehyun Kim, Jaewoo Kang, Hyunwoo J. Kim, Graph Transformer Networks: Learning meta-path graphs to improve\nGNNs, Neural Networks 2022.\n\n![](https://github.com/seongjunyun/Graph_Transformer_Networks/blob/master/GTN.png)\n\n### Updates\n* \\[**Sep 19, 2022**\\] We released the source code of our [FastGTN with non-local operations]((https://pdf.sciencedirectassets.com/271125/1-s2.0-S0893608022X00075/1-s2.0-S0893608022002003/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHkaCXVzLWVhc3QtMSJHMEUCIQC5JKs%2BBKb0MPqBvG9De58QPzs52B2jcbCKrlS8Ahtx8wIgXRKM1pwytn%2Fg3I%2BpRawictl9bHkbukMC1av%2FjfvDGakqzAQIIhAFGgwwNTkwMDM1NDY4NjUiDCgbkDuexMDYYCd7eyqpBPka68pkQYZf%2FozGqEf8msefhrCK%2Bt3vWHggdPa9o1Nfahc6uf6RvB1iUr2qGE%2BmDtqkZgHllcBwjK9N5oGEEEz%2FXR%2Bo5GNUcWU8OyyAxpunLswUrabAPi5%2FKI6EKT8k0f%2BfxajwSizgKlEwYcOa5SPCi62e87GCI1hzQlTvjhub54aA6JkzMGYBrAODaTH6fxTW2bL8kgXFQCjMwsEy81yyh%2F6xWZ1b9ybHrdobu3ivHDiN1n7oXoW1o7E9ZPg2Tm4%2B9iFLeBF49QZlsVTxMb%2BnhSDJYoEmnyEM3OJRCuAXex%2F1Xhu0GzsvhgR9Ahaofbx9b2XNK8926l4eFW7sO9Q2Bu4VqJ4jqhYI74CYJA5t1BE49jMaNCZs%2Bl163Mnl4GuzTDZiTtg0GOnnDf5HZ2n4DSP0sTGYK5QWSQRbMBwI2s5eB1mb0gzIimHglsd1TfGhav%2BtD3X6149li6LUQQy9gxrEQLWhJY%2Frggl6lbJ2yb5yanW42sf2iVdcmX3WpevKqyuGRDo4TWN59D6h2T1xC7f5NQ8uRW4wTFDRZ%2FUZjX3gyVwE2qquYR%2BbMVwvD4R6fbi9AONo%2BU68fEZNcYJQ5igRWAWtZk6cGQno8XPZbnYUYfAO1Q9WWagJq%2FJC2eDPVJYb330BrV3rbmpasvmWUnkJwUYVNhAeSGp9AiS%2BWCNR3Wo4qMDoPhULj31UJ2967m9m1HCkJ%2FOWwOlT4zDuEZmDBGefvysw69aZmQY6qQEm7If21VPh69neRb1%2BJGDySZcFw53B7jrt%2BI1ERmLBsDV9%2B3cPMBiFwRltFW1aT%2FDFRBdxNordu2sB4UD7FzMbm1a6KOlbrntZQntaiNS9S4gM0DoMccmTZgRkHlmkUr%2FjxGyVntJ6EL7pTvFbxNPeic6o5v8UZs%2B%2BpBcs2cXJ%2BCrtHucNoLkf6RlwPqr3PfQNERNEuMN2hhkRsByEhBbeQsu22ViDbc9C\u0026X-Amz-Algorithm=AWS4-HMAC-SHA256\u0026X-Amz-Date=20220918T015554Z\u0026X-Amz-SignedHeaders=host\u0026X-Amz-Expires=300\u0026X-Amz-Credential=ASIAQ3PHCVTY6A6RHQPB%2F20220918%2Fus-east-1%2Fs3%2Faws4_request\u0026X-Amz-Signature=0f0752d788b3d905a14a4f0b5bf56efa71a37c93fea77d5671ec170b55e28027\u0026hash=8fdbb1b3d93f684d5528e2ae717306bb9eaae0380ae6fb68b81361df1f1a54f0\u0026host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61\u0026pii=S0893608022002003\u0026tid=spdf-9cbe5006-8e6c-4bc8-a955-6f50a80987f5\u0026sid=26eb78364bb5a0466f4bf51-3ba867604046gxrqa\u0026type=client\u0026ua=4d52035053565707575c\u0026rr=74c6764b6874c08c)), which improves GTN's scalability (Fast) and performance (non-local operations). \n* \\[**Sep 19, 2022**\\] We updated the source code of our GTNs to address the issue where the latest version of torch_geometric removed the backward() of the multiplication of sparse matrices (spspmm). To be specific, we implemented the multiplication of sparse matrices using [pytorch.sparse.mm](https://pytorch.org/docs/stable/generated/torch.sparse.mm.html) that includes backward() operation.\n\n## Installation\n\nInstall [pytorch](https://pytorch.org/get-started/locally/)\n\nInstall [torch_geometric](https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html)\n\nTo run the previous version of GTN (in prev_GTN folder),\n``` \n$ pip install torch-sparse-old\n```\n** The latest version of torch_geometric removed the backward() of the multiplication of sparse matrices (spspmm), so to solve the problem, we uploaded the old version of torch-sparse with backward() on pip under the name torch-sparse-old.\n\n## Data Preprocessing\nWe used datasets from [Heterogeneous Graph Attention Networks](https://github.com/Jhy1993/HAN) (Xiao Wang et al.) and uploaded the preprocessing code of acm data as an example.\n\n## Running the code\n*** To check the best performance of GTN in DBLP and ACM datasets, we recommend running the GTN in [OpenHGNN](https://github.com/BUPT-GAMMA/OpenHGNN/tree/main/openhgnn/output/GTN) implemented with the DGL library. Since the newly used torch.sparsemm requires more memory than the existing torch_sparse.spspmm, it was not possible to run the best case with num_layer \u003e 1 in DBLP and ACM datasets. \n``` \n$ mkdir data\n$ cd data\n```\nDownload datasets (DBLP, ACM, IMDB) from this [link](https://drive.google.com/file/d/1Nx74tgz_-BDlqaFO75eQG6IkndzI92j4/view?usp=sharing) and extract data.zip into data folder.\n```\n$ cd ..\n```\n\n- DBLP\n\t\n\t- GTN\n\t```\n\t$ python main.py --dataset DBLP --model GTN --num_layers 1 --epoch 50 --lr 0.02 --num_channels 2\n\t```\n\t- FastGTN\n\t1) w/ non-local operations ( \u003e24 GB)\n\t```\n\t$ python main.py --dataset DBLP --model FastGTN --num_layers 4 --epoch 100 --lr 0.02 --channel_agg mean --num_channels 2 --non_local_weight 0 --K 3   --non_local\n  ```\n\t 2) w/o non-local operations\n\t```\n\t$ python main.py --dataset DBLP --model FastGTN --num_layers 4 --epoch 100 --lr 0.02 --channel_agg mean --num_channels 2\n  ```\n\n- ACM\n\t\n\t- GTN\n\t```\n\t$ python main_gpu.py --dataset ACM --model GTN --num_layers 1 --epoch 100 --lr 0.02 --num_channels 2\n\t```\n\t- FastGTN\n\t1) w/ non-local operations \n\t```\n\t$ python main_gpu.py --dataset ACM --model FastGTN --num_layers 3 --epoch 200 --lr 0.05 --channel_agg mean --num_channels 2 --non_local_weight -1 --K 1 --non_local\n  ```\n\t 2) w/o non-local operations\n\t```\n\t$ python main_gpu.py --dataset ACM --model FastGTN --num_layers 3 --epoch 200 --lr 0.05 --channel_agg mean --num_channels 2\n  ```\n\n- IMDB\n\t\n\t- GTN\n\t```\n\t$ python main.py --dataset IMDB --model GTN --num_layers 2 --epoch 50 --lr 0.02 --num_channels 2\n\t```\n\t- FastGTN\n\t1. w/ non-local operations \n\t```\n\t$ python main.py --dataset IMDB --model FastGTN --num_layers 3 --epoch 50 --lr 0.02 --channel_agg mean --num_channels 2 --non_local_weight -2 --K 2 --non_local\n  ```\n\t 2) w/o non-local operations\n\t```\n\t$ python main.py --dataset IMDB --model FastGTN --num_layers 3 --epoch 50 --lr 0.02 --channel_agg mean --num_channels 2\n  ```\n\n\n## Citation\nIf this work is useful for your research, please cite our [GTN](https://arxiv.org/abs/1911.06455) and [FastGTN](https://reader.elsevier.com/reader/sd/pii/S0893608022002003?token=71585B1BEE922F5060A60F850BC1EA8C67B4077ECC43793878B38754A499AC67450DACAB0FAEA5EC4607CD106CC58974\u0026originRegion=us-east-1\u0026originCreation=20220918020619):\n```\n@inproceedings{yun2019GTN,\n  title={Graph Transformer Networks},\n  author={Yun, Seongjun and Jeong, Minbyul and Kim, Raehyun and Kang, Jaewoo and Kim, Hyunwoo J},\n  booktitle={Advances in Neural Information Processing Systems},\n  pages={11960--11970},\n  year={2019}\n}\n```\n```\n@article{yun2022FastGTN,\ntitle = {Graph Transformer Networks: Learning meta-path graphs to improve GNNs},\njournal = {Neural Networks},\nvolume = {153},\npages = {104-119},\nyear = {2022},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseongjunyun%2FGraph_Transformer_Networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseongjunyun%2FGraph_Transformer_Networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseongjunyun%2FGraph_Transformer_Networks/lists"}