{"id":13570719,"url":"https://github.com/LirongWu/awesome-graph-self-supervised-learning","last_synced_at":"2025-04-04T07:31:52.203Z","repository":{"id":38364436,"uuid":"188432386","full_name":"LirongWu/awesome-graph-self-supervised-learning","owner":"LirongWu","description":"Code for TKDE paper \"Self-supervised learning on graphs: Contrastive, generative, or predictive\"","archived":false,"fork":false,"pushed_at":"2024-03-10T13:28:46.000Z","size":438,"stargazers_count":1291,"open_issues_count":0,"forks_count":160,"subscribers_count":16,"default_branch":"main","last_synced_at":"2024-05-22T15:05:23.208Z","etag":null,"topics":["data-augmentation","deep-learning","graph-neural-networks","machine-learning","pre-training","pretext-task","representation-learning","self-supervised-learning","transfer-learning","unsupervised-learning"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LirongWu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-05-24T14:06:38.000Z","updated_at":"2024-05-20T06:42:48.000Z","dependencies_parsed_at":"2024-03-08T15:03:14.745Z","dependency_job_id":"1b0250bd-df03-42e2-9854-193243b122fb","html_url":"https://github.com/LirongWu/awesome-graph-self-supervised-learning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LirongWu%2Fawesome-graph-self-supervised-learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LirongWu%2Fawesome-graph-self-supervised-learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LirongWu%2Fawesome-graph-self-supervised-learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LirongWu%2Fawesome-graph-self-supervised-learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LirongWu","download_url":"https://codeload.github.com/LirongWu/awesome-graph-self-supervised-learning/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246952274,"owners_count":20859812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-augmentation","deep-learning","graph-neural-networks","machine-learning","pre-training","pretext-task","representation-learning","self-supervised-learning","transfer-learning","unsupervised-learning"],"created_at":"2024-08-01T14:00:54.486Z","updated_at":"2025-04-04T07:31:52.166Z","avatar_url":"https://github.com/LirongWu.png","language":null,"funding_links":[],"categories":["Machine Learning Algorithms","Related Awesome","Table of Contents","Graph Learning","图监督_半监督_对比学习","Other Lists","Others"],"sub_categories":["Sparsity and Popularity Biases","Pre-Print Status","网络服务_其他","TeX Lists"],"readme":"# Awesome Graph Self-Supervised Learning\n\n![PRs Welcome](https://img.shields.io/badge/PRs-Welcome-green)[![Awesome](https://awesome.re/badge.svg)](https://awesome.re)![GitHub stars](https://img.shields.io/github/stars/LirongWu/awesome-graph-self-supervised-learning?color=yellow)  ![GitHub forks](https://img.shields.io/github/forks/LirongWu/awesome-graph-self-supervised-learning?color=blue\u0026label=Fork)\n\nA curated list for awesome self-supervised graph representation learning resources. Inspired by [awesome-deep-vision](https://github.com/kjw0612/awesome-deep-vision), [awesome-adversarial-machine-learning](https://github.com/yenchenlin/awesome-adversarial-machine-learning), [awesome-deep-learning-papers](https://github.com/terryum/awesome-deep-learning-papers), [awesome-architecture-search](https://github.com/markdtw/awesome-architecture-search), and [awesome-self-supervised-learning](https://github.com/jason718/awesome-self-supervised-learning).\n\n#### Why Self-Supervised?\n\u003e Self-Supervised Learning has become an exciting direction in AI community. \n\n  - Jitendra Malik: \"Supervision is the opium of the AI researcher\"\n  - Alyosha Efros: \"The AI revolution will not be supervised\"\n  - Yann LeCun: \"self-supervised learning is the cake, supervised learning is the icing on the cake, reinforcement learning is the cherry on the cake\"\n\n## Table of Contents\n\n- [Overview](#Overview)\n- [Training Strategy](#Training-Strategy)\n- [Contrastive Learning](#Contrastive-Learning)\n  - [Same-Scale Contrasting](#Global-Global-Contrasting)\n    - [Global-Global Contrasting](#Global-Global-Contrasting)\n    - [Context-Context Contrasting](#Context-Context-Contrasting)\n    - [Local-Local Contrasting](#Local-Local-Contrasting)\n  - [Corss-Scale Contrasting](#Local-Global-Contrasting)\n    - [Local-Global Contrasting](#Local-Global-Contrasting)\n    - [Local-Context Contrasting](#Local-Context-Contrasting)\n    - [Context-Global Contrasting](#Context-Global-Contrasting)\n- [Generative Learning](#Generative-Learning)\n  - [Graph Autoencoding](#Graph-Autoencoding)\n  - [Graph Autoregression](#Graph-Autoregression)\n- [Predictive Learning](#Predictive-Learning)\n  - [Node Property Prediction](#Node-Property-Prediction)\n  - [Context-based Prediction](#Context-based-Prediction)\n  - [Self-Training](#Self-Training)\n  - [Domain Knowledge-based Prediction](#Domain-Knowledge-based-Prediction)\n- [A Summary of Methodology Details](#A-Summary-of-Methodology-Details)\n- [A Summary of Implementation Details](#A-Summary-of-Implementation-Details)\n- [A Summary of Common Graph Datasets](#A-Summary-of-Common-Graph-Datasets)\n- [A Summary of Open-source Codes](#A-Summary-of-Open-source-Codes)\n\n## Overview\n\nWe extend the concept of self-supervised learning, which first emerged in the fields of computer vision and natural language processing, to present a timely and comprehensive review of the existing SSL techniques for graph data. Specifically, we divide existing graph SSL methods into three categories: contrastive, generative, and predictive as shown below.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src='./figs/categories.PNG' width=\"500\"\u003e\n\u003c/p\u003e\n\n- Contrastive Learning: it contrasts the views generated by different data augmentation methods. The information about the differences and sameness between data-data pairs (inter-data) is used as self-supervision signals.\n- Generative Learning: it focuses on the (intra-data) information embedded in the data, generally based on prtext tasks such as reconstruction, which exploit the attributes and structure of the data itself as self-supervision signals.\n- Predictive Learning: it generally self-generates labels from graph data through some simple statistical analysis, or expert knowledge, and designs prediction-based pretext tasks based on the self-generated labels to handle the data-label relationship.\n\n## Training Strategy\n\nConsidering the relationship among bottleneck encoders, self-supervised pretext tasks, and downstream tasks, the training strategies can be divided into three categories: Pre-training and Fine-tuning (P\\\u0026F), Joint Learning (JL), and Unsupervised Representation Learning (URL), with their detailed workflow shown below.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src='./figs/training strategy.PNG' width=\"500\"\u003e\n\u003c/p\u003e\n\n- Pre-train\\\u0026Fine-tune (P\u0026F): it first pre-trains the encoder with unlabeled nodes by the self-supervised pretext tasks. The pre-trained encoder’s parameters are then used as the initialization of the encoder used in supervised fine-tuning for downstream tasks.\n- Joint Learning (JL): an auxiliary pretext task with self-supervision is included to help learn the supervised downstream task. The encoder is trained through both the pretext task and the downstream task simultaneously.\n- Unsupervised Representation Learning (URL): it first pre-trains the encoder with unlabeled nodes by the self-supervised pretext tasks. The pre-trained encoder’s parameters are then frozen and used in the supervised downstream task with additional labels.\n\n## Contrastive Learning\n\nA general framework for contrastive learning is shown below. The two contrasting components may be local, contextual, or global, corresponding to node-level (marked in red), subgraph-level (marked in green), or graph-level (marked in yellow) information in the graph. The contrastive learning can thus contrast two views (at the *same* or *different* scales), which leads to two categories of algorithm: (1) same-scale contrasting, including *Local-Local (L-L)* contrasting, *Context-Context (C-C)* contrasting, and *Global-Global (G-G)* contrasting; and (2) cross-scale contrasting, including *Local-Context (L-C)* contrasting, *Local-Global (L-G)* contrasting, and *Context-Global (C-G)* contrasting.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src='./figs/contrasting_pretext.PNG' width=\"800\"\u003e\n\u003c/p\u003e\n\n\n\n#### Global-Global Contrasting\n\n- GraphCL: Graph Contrastive Learning with Augmentations.\n  - Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen. *NIPS 2020*. [[pdf]](https://proceedings.neurips.cc/paper/2020/file/3fe230348e9a12c13120749e3f9fa4cd-Paper.pdf) [[code]](https://github.com/Shen-Lab/GraphCL)\n- IGSD: Iterative Graph Self-Distillation.\n  - H. Zhang, S. Lin, W. Liu, P. Zhou, J. Tang, X. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2010.12609.pdf)\n- DACL: Towards Domain-Agnostic Contrastive Learning.\n  - V. Verma, M.-T. Luong, K. Kawaguchi, H. Pham, andQ. V. Le. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2011.04419.pdf)\n- LCC: Label Contrastive Coding Based Graph Neural Network for Graph Classification.\n  - Y. Ren, J. Bai, and J. Zhang. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2101.05486.pdf) [[code]](https://github.com/YuxiangRen/Label-Contrastive-Coding-based-Graph-Neural-Network-for-Graph-Classification-)\n- CCGL: Contrastive Cascade Graph Learning. \n  - X. Xu, F. Zhou, K. Zhang, and S. Liu. *TKDE 2022*. [[pdf]](https://arxiv.org/pdf/2107.12576) [[code]](https://github.com/Xovee/ccgl)\n- CSSL: Contrastive Self-Supervised Learning for Graph Classification.\n  - J. Zeng and P. Xie. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2009.05923.pdf)\n\n#### Context-Context Contrasting\n\n- GCC: Graph Contrastive Coding for Graph Neural Network Pre-training.\n  - J. Qiu, Q. Chen, Y. Dong, J. Zhang, H. Yang, M. Ding, K. Wang, and J. Tang. *KDD 2020*. [[pdf]](https://arxiv.org/pdf/2006.09963.pdf) [[code]](https://github.com/THUDM/GCC)\n\n#### Local-Local Contrasting\n\n- CDNMF: Contrastive Deep Nonnegative Matrix Factorization for Community Detection.\n  - Y. Li, J. Chen, C. Chen, L. Yang, Z. Zheng. *ICASSP 2024*. [[pdf]](https://arxiv.org/abs/2311.02357) [[code]](https://github.com/6lyc/CDNMF)\n- GRACE: Deep Graph Contrastive Representation Learning.\n  - Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.04131.pdf) [[code]](https://github.com/CRIPAC-DIG/GRACE)\n- GCA: Graph Contrastive Learning with Adaptive Augmentation.\n  - Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2010.14945.pdf) [[code]](https://github.com/CRIPAC-DIG/GCA)\n- GROC: Towards Robust Graph Contrastive Learning.\n  - N. Jovanovi´c, Z. Meng, L. Faber, and R. Wattenhofer. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2102.13085.pdf)\n- SEPT: Socially-Aware Self-Supervised Tri-Training for Recommendation.\n  -  J. Yu, H. Yin, M. Gao, X. Xia, X. Zhang, and N. Q. V.Hung. *Arxiv 2021*. [[pdf]](https://arxiv.org/abs/2106.03569) [[code]](https://github.com/Coder-Yu/QRec)\n- STDGI: Spatio-Temporal Deep Graph Infomax.\n  - F. L. Opolka, A. Solomon, C. Cangea, P. Veliˇckovi´c, P. Li` o, and R. D. Hjelm. *Arxiv 2019*. [[pdf]](https://arxiv.org/pdf/1904.06316.pdf)\n- GMI: Graph Representation Learning via Graphical Mutual Information Maximization.\n  - L. Yu, S. Pei, C. Zhang, L. Ding, J. Zhou, L. Li, and X. Zhang. *WWW 2020*. [[pdf]](https://arxiv.org/pdf/2002.01169.pdf) [[code]](https://github.com/zpeng27/GMI)\n- KS2L: Self-Supervised Smoothing Graph Neural Networks.\n  - L. Yu, S. Pei, C. Zhang, L. Ding, J. Zhou, L. Li, and X. Zhang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2009.00934.pdf)\n- CG3: Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning.\n  - S. Wan, S. Pan, J. Yang, and C. Gong. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2009.07111.pdf)\n- BGRL: Bootstrapped Representation Learning on Graphs.\n  - S. Thakoor, C. Tallec, M. G. Azar, R. Munos, P. Veliˇckovi´c, and M. Valko. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2102.06514.pdf)[[code]](https://github.com/nerdslab/bgrl)\n- SelfGNN: Self-supervised Graph Neural Networks without Explicit Negative Sampling.\n  - Z. T. Kefato and S. Girdzijauskas. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2103.14958.pdf) [[code]](https://github.com/zekarias-tilahun/SelfGNN)\n- HeCo: Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning.\n  - X. Wang, N. Liu, H. Han, and C. Shi. *Arxiv 2021*. [[pdf]](https://arxiv.org/abs/2105.09111) [[code]](https://github.com/liun-online/HeCo)\n- PT-DGNN: Pre-training on Dynamic Graph Neural Networks.\n  - J. Zhang, K. Chen, and Y. Wang. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2102.12380.pdf) [[code]](https://github.com/Mobzhang/PT-DGNN)\n- COAD: Coad: Contrastive Pretraining with Adversarial Fine-tuning for Zero-shot Expert Linking.\n  - B. Chen, J. Zhang, X. Zhang, X. Tang, L. Cai, H. Chen, C. Li, P. Zhang, and J. Tang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2012.11336.pdf) [[code]](https://github.com/allanchen95/Expert-Linking)\n- Contrast-Reg: Improving Graph Representation Learning by Contrastive Regularization.\n  - K. Ma, H. Yang, H. Yang, T. Jin, P. Chen, Y. Chen, B. F. Kamhoua, and J. Cheng. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2101.11525.pdf)\n- C-SWM: Contrastive Learning of Structured World Models.\n  - T. Kipf, E. van der Pol, and M. Welling. *Arxiv 2019. [[pdf]](https://arxiv.org/pdf/1911.12247.pdf) [[code]](https://github.com/tkipf/c-swm)\n\n#### Local-Global Contrasting\n\n- DGI: Deep Graph Infomax.\n  - P. Velickovic, W. Fedus, W. L. Hamilton, P. Li` o, Y. Bengio, and R. D. Hjelm. *ICLR 2019*. [[pdf]](https://arxiv.org/pdf/1809.10341.pdf) [[code]](https://github.com/PetarV-/DGI)\n- HDMI: Hdmi: High-order Deep Multiplex Infomax.\n  - B. Jing, C. Park, and H. Tong. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2102.07810.pdf)\n- DMGI: Unsupervised Attributed Multiplex Network Embedding.\n  - C. Park, D. Kim, J. Han, and H. Yu. *AAAI 2020*. [[pdf]](https://ojs.aaai.org/index.php/AAAI/article/view/5985/5841) [[code]](https://github.com/pcy1302/DMGI)\n- MVGRL: Contrastive Multi-View Representation Learning on Graphs.\n  - K. Hassani and A. H. K. Ahmadi. *ICML 2020*. [[pdf]](http://proceedings.mlr.press/v119/hassani20a/hassani20a.pdf) [[code]](https://github.com/kavehhassani/mvgrl)\n- HDGI: Heterogeneous Deep Graph Infomax.\n  - Y. Ren, B. Liu, C. Huang, P. Dai, L. Bo, and J. Zhang. *Arxiv 2019*. [[pdf]](https://arxiv.org/pdf/1911.08538.pdf) [[code]](https://github.com/YuxiangRen/Heterogeneous-Deep-Graph-Infomax)\n\n#### Local-Context Contrasting\n\n- CDNMF: Contrastive Deep Nonnegative Matrix Factorization for Community Detection.\n  - Y. Li, J. Chen, C. Chen, L. Yang, Z. Zheng. *ICASSP 2024*. [[pdf]](https://arxiv.org/abs/2311.02357) [[code]](https://github.com/6lyc/CDNMF)\n- Subg-Con: Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning.\n  - Y. Jiao, Y. Xiong, J. Zhang, Y. Zhang, T. Zhang, and Y. Zhu. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2009.10273.pdf) [[code]](https://github.com/yzjiao/Subg-Con)\n- Cotext Prediction: Strategies for Pre-training Graph Neural Networks.\n  - W. Hu, B. Liu, J. Gomes, M. Zitnik, P. Liang, V. S. Pande, and J. Leskovec. *ICLR 2020*. [[pdf]](https://arxiv.org/pdf/1905.12265.pdf) [[code]](http://snap.stanford.edu/gnn-pretrain)\n- GIC: Leveraging Cluster-level Node Information for Unsupervised Graph Representation Learning.\n  - C. Mavromatis and G. Karypis. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2009.06946.pdf) [[code]](https://github.com/cmavro/Graph-InfoClust-GIC)\n- GraphLoG: Self-Supervised Graph-level Representation Learning with Local and Global Structure.\n  - M. Xu, H. Wang, B. Ni, H. Guo, and J. Tang. *OpenReview 2021*. [[pdf]](https://openreview.net/forum?id=DAaaaqPv9-q) [[code]](https://openreview.net/forum?id=DAaaaqPv9-q)\n- MHCN: Self-Supervised Multi-channel Hypergraph Convolutional Network for Social Recommendation.\n  - J. Yu, H. Yin, J. Li, Q. Wang, N. Q. V. Hung, and X. Zhang. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2101.06448.pdf) [[code]](https://github.com/Coder-Yu/RecQ)\n- EGI: Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization.\n  - Q. Zhu, Y. Xu, H.Wang, C. Zhang, J. Han, and C. Yang. *Arxiv 2020*. [[pdf]](https://arxiv.org/abs/2009.05204) [[code]](https://openreview.net/forum?id=J_pvI6ap5Mn)\n\n#### Context-Global Contrasting\n\n- MICRO-Graph: Motif-Driven Contrastive Learning of Graph Representations.\n  - S. Zhang, Z. Hu, A. Subramonian, and Y. Sun. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2012.12533.pdf) [[code]](https://drive.google.com/file/d/1b751rpnV-SDmUJvKZZI-AvpfEa9eHxo9/)\n- InfoGraph: Unsupervised and Semi-Supervised Graph-level Representation Learning via Mutual Information Maximization.\n  - F. Sun, J. Hoffmann, V. Verma, and J. Tang. *ICLR 2020*. [[pdf]](https://arxiv.org/pdf/1908.01000.pdf) [[code]](https://github.com/fanyun-sun/InfoGraph)\n- SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism.\n  - Q. Sun, H. Peng, J. Li, J. Wu, Y. Ning, P. S. Yu, and L. He. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2101.08170.pdf) [[code]](https://github.com/RingBDStack/SUGAR)\n- BiGI: Bipartite Graph Embedding via Mutual Information Maximization.\n  - J. Cao, X. Lin, S. Guo, L. Liu, T. Liu, and B. Wang. *WSDM 2021*. [[pdf]](https://arxiv.org/abs/1505.05192) [[code]](https://github.com/clhchtcjj/BiNE)\n- HTC: Graph Representation Learning by Ensemble Aggregating Subgraphs via Mutual Information Maximization.\n  - C. Wang and Z. Liu. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2103.13125.pdf)\n- DITNet: Drug Target Prediction using Graph Representation Learning via Substructures Contrast.\n  - S. Cheng, L. Zhang, B. Jin, Q. Zhang, and X. Lu. *Preprints 2021*. [[pdf]](https://www.preprints.org/manuscript/202103.0337/v1) [[code]](https://github.com/FangpingWan/NeoDTI)\n\n## Generative Learning\n\n####  Graph Autoencoding\n- CDNMF: Contrastive Deep Nonnegative Matrix Factorization for Community Detection.\n  - Y. Li, J. Chen, C. Chen, L. Yang, Z. Zheng. *ICASSP 2024*. [[pdf]](https://arxiv.org/abs/2311.02357) [[code]](https://github.com/6lyc/CDNMF)\n- GraphMAE: Self-supervised Masked Graph Autoencoders\n  - Z. Hou, X. Liu, Y. Cen, Y. Dong, H. Yang, C. Wang, and J. Tang. *KDD 2022* [[pdf]](https://arxiv.org/pdf/2205.10803.pdf) [[code]](https://github.com/THUDM/GraphMAE)\n- Graph Completion: When Does Self-Supervision Help Graph Convolutional Networks?\n  - Y. You, T. Chen, Z. Wang, and Y. Shen. *PMLR 2020*. [[pdf]](http://proceedings.mlr.press/v119/you20a/you20a.pdf) [[code]](https://github.com/Shen-Lab/SS-GCNs)\n- Node Attribute Masking: Self-Supervised Learning on Graphs: Deep Insights and New Direction.\n  - W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.10141.pdf) [[code]](https://github.com/ChandlerBang/SelfTask-GNN)\n- Edge Attribute Masking: Strategies for Pre-training Graph Neural Networks.\n  - W. Hu, B. Liu, J. Gomes, M. Zitnik, P. Liang, V. S. Pande, and J. Leskovec. *ICLR 2020*. [[pdf]](https://arxiv.org/pdf/1905.12265.pdf) [[code]](http://snap.stanford.edu/gnn-pretrain)\n- Node Attribute and Embedding Denoising: Graph-based Neural Network Models with Multiple Self-Supervised Auxiliary Tasks.\n  - F. Manessi and A. Rozza. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2011.07267.pdf)\n- Adjacency Matrix Reconstruction: Self-Supervised Training of Graph Convolutional Networks.\n  - Q. Zhu, B. Du, and P. Yan. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.02380.pdf)\n- Graph Bert: Only Attention is Needed for Learning Graph Representations.\n  - J. Zhang, H. Zhang, C. Xia, and L. Sun. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2001.05140.pdf) [[code]](https://github.com/anonymous-sourcecode/Graph-Bert)\n- Pretrain-Recsys: Pretraining Graph Neural Networks for Cold-start Users and Items Representation.\n  - B. Hao, J. Zhang, H. Yin, C. Li, and H. Chen. *WSDM 2021*. [[pdf]](https://dl.acm.org/doi/abs/10.1145/3437963.3441738) [[code]](https://github.com/jerryhao66/Pretrain-Recsys)\n- SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks.\n  - B. Fatemi, L. E. Asri, and S. M. Kazemi. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2102.05034.pdf) [[code]](https://github.com/BorealisAI/SLAPS-GNN)\n- G-BERT: Pre-Training of Graph Augmented Transformers for Medication Recommendation.\n  - J. Shang, T. Ma, C. Xiao, and J. Sun. *Arxiv 2019*. [[pdf]](https://arxiv.org/pdf/1906.00346.pdf) [[code]](https://github.com/jshang123/G-Bert)\n\n####  Graph Autoregression\n\n- GPT-GNN: Generative Pre-training of Graph Neural Networks.\n  - Z. Hu, Y. Dong, K. Wang, K. Chang, and Y. Sun. *KDD 2020*. [[pdf]](https://dl.acm.org/doi/pdf/10.1145/3394486.3403237) [[code]](https://github.com/acbull/GPT-GNN)\n\n## Predictive Learning\n\nA comparison of the predictive learning is shown below. The predictive method generally self-generates labels from graph data and then designs prediction-based pretext tasks based on the self-generated labels. Categorized  by  how  the  labels  areobtained,  we  summarize  predictive  learning  methods  forgraph data into four categories: \n\n- Node Property Prediction: it pre-calculates the node properties, such as node degree and used them as self-supervised labels. \n- Context-based Prediction: the local or global contextual information in the graph, such as  the shortest path length between nodes can be extracted as labels to help with self-supervised learning. \n- Self-Training: it applies algorithms such as unsupervised clustering to obtain pseudo-labels and then updates the pseudo-label set of the previous stage based on the prediction results or losses. \n- Domain Knowledge-based Prediction: the domain knowledge, such as expert knowledge or specialized tools, can be used in advance to obtain informative labels.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src='./figs/predictive.PNG' width=\"1000\"\u003e\n\u003c/p\u003e\n\n####  Node Property Prediction\n\n- Node Property Prediction: Self-Supervised Learning on Graphs: Deep Insights and New Direction.\n  - W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.10141.pdf) [[code]](https://github.com/ChandlerBang/SelfTask-GNN)\n\n####  Context-based Prediction\n\n- S2GRL: Self-Supervised Graph Representation Learning via Global Context Prediction.\n  - Z. Peng, Y. Dong, M. Luo, X.-M. Wu, and Q. Zheng. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2003.01604.pdf)\n- PairwiseDistance: Self-Supervised Learning on Graphs: Deep Insights and New Direction.\n  - W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.10141.pdf) [[code]](https://github.com/ChandlerBang/SelfTask-GNN)\n- PairwiseAttsim: Self-Supervised Learning on Graphs: Deep Insights and New Direction.\n  - W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.10141.pdf) [[code]](https://github.com/ChandlerBang/SelfTask-GNN)\n- Distance2Cluster: Self-Supervised Learning on Graphs: Deep Insights and New Direction.\n  - W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.10141.pdf) [[code]](https://github.com/ChandlerBang/SelfTask-GNN)\n- EdgeMask: Self-Supervised Learning on Graphs: Deep Insights and New Direction.\n  - W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.10141.pdf) [[code]](https://github.com/ChandlerBang/SelfTask-GNN)\n- TopoTER: Unsupervised Learning of Topology Transformation Equivariant Representations.\n  - X. Gao, W. Hu, and G.-J. Qi. *OpenReview 2021*. [[pdf]](https://openreview.net/forum?id=9az9VKjOx00)\n- Centrality Score Ranking: Pretraining Graph Neural Networks for Generic Structural Feature Extraction.\n  - Z. Hu, C. Fan, T. Chen, K.-W. Chang, and Y. Sun. *Arxiv 2019*.  [[pdf]](https://arxiv.org/pdf/1905.13728.pdf)\n- Meta-path prediction: Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs.\n  - D. Hwang, J. Park, S. Kwon, K. Kim, J. Ha, and H. J. Kim. *NIPS 2020*. [[pdf]](https://arxiv.org/pdf/2007.08294.pdf) [[code]](https://github.com/mlvlab/SELAR)\n- SLiCE: Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networks.\n  - P. Wang, K. Agarwal, C. Ham, S. Choudhury, and C. K. Reddy. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2007.11192.pdf) [[code]](https://github.com/pnnl/SLICE)\n- Distance2Labeled: Self-Supervised Learning on Graphs: Deep Insights and New Direction.\n  - W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.10141.pdf) [[code]](https://github.com/ChandlerBang/SelfTask-GNN)\n- Distance2Labeled: Self-Supervised Learning on Graphs: Deep Insights and New Direction.\n  - W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2006.10141.pdf) [[code]](https://github.com/ChandlerBang/SelfTask-GNN)\n- HTM: Hop-count based Self-Supervised Anomaly Detection on Attributed Networks.\n  - T. Huang, Y. Pei, V. Menkovski, and M. Pechenizkiy. *Arxiv 2021*. [[pdf]](https://arxiv.org/pdf/2104.07917.pdf)\n\n####  Self-Training\n\n- Multi-stage Self-training: Deeper insights into Graph Convolutional Networks for Semi-Supervised Learning.\n  - Q. Li, Z. Han, and X. Wu. *AAAI 2018*. [[pdf]](https://ojs.aaai.org/index.php/AAAI/article/view/11604) [[code]](https://github.com/Davidham3/deeper_insights_into_GCNs)\n- Node Clustering and Partitioning: When Does Self-Supervision Help Graph Convolutional Networks.\n  - Y. You, T. Chen, Z. Wang, and Y. Shen. *PMLR 2020*. [[pdf]](http://proceedings.mlr.press/v119/you20a/you20a.pdf) [[code]](https://github.com/Shen-Lab/SS-GCNs)\n- CAGAN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation Learning.\n  - Y. Zhu, Y. Xu, F. Yu, S. Wu, and L. Wang. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2009.01674.pdf)\n- M3S: Multi-stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes.\n  - K. Sun, Z. Lin, and Z. Zhu. *AAAI 2020*. [[pdf]](https://deepai.org/publication/multi-stage-self-supervised-learning-for-graph-convolutional-networks) [[code]](https://github.com/datake/M3S)\n- Cluster Preserving: Pretraining Graph Neural Networks for Generic Structural Feature Extraction.\n  - Z. Hu, C. Fan, T. Chen, K.-W. Chang, and Y. Sun. *Arxiv 2019*. [[pdf]](https://arxiv.org/pdf/1905.13728.pdf)\n- SEF: Self-Supervised Edge Features for Improved Graph Neural Network Training.\n  - A. Sehanobish, N. G. Ravindra, and D. van Dijk. *Arxiv 2020*. [[pdf]](https://arxiv.org/pdf/2007.04777.pdf)[[code]](https://github.com/nealgravindra/self-supervsed_edge_feats)\n\n####  Domain Knowledge-based Prediction\n\n- Contextual Molecular Property Prediction: Self-Supervised Graph Transformer on Large-Scale Molecular Data.\n  - Y. Rong, Y. Bian, T. Xu, W. Xie, Y. Wei, W. Huang, and J. Huang. *NIPS 2020*. [[pdf]](https://drug.ai.tencent.com/publications/GROVER.pdf) [[code]](https://github.com/tencent-ailab/grover)\n- Graph-level Motif Prediction: Self-Supervised Graph Transformer on Large-scale Molecular Data.\n  - Y. Rong, Y. Bian, T. Xu, W. Xie, Y. Wei, W. Huang, and J. Huang. *NIPS 2020*. [[pdf]](https://drug.ai.tencent.com/publications/GROVER.pdf) [[code]](https://github.com/tencent-ailab/grover)\n- DrRepair: Graph-based, Self-Supervised Program Repair from Diagnostic Feedback.\n  - M. Yasunaga and P. Liang. *PMLR 2020*. [[pdf]](http://proceedings.mlr.press/v119/yasunaga20a/yasunaga20a.pdf) [[code]](https://github.com/michiyasunaga/DrRepair)\n\n\nA summary of all the surveyed works is presented below.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src='./figs/overview.PNG' width=\"1200\"\u003e\n\u003c/p\u003e\n\n\n\n## A Summary of Methodology Details\nAbout Graph Property, Pretext Task, Data Augmentation, Objective Function, Training Strategy, and Year of publication.\n\n| Methods                                      |  Graph Property  |         Pretext-Task          |                      Data Augmentation                       |                  Objective Function                   | Training Strategy | Year |\n| :------------------------------------------- | :--------------: | :---------------------------: | :----------------------------------------------------------: | :---------------------------------------------------: | :---------------: | :--: |\n| CDNMF                             |    Attributed    |         Contrastive/L-C + Generative/AE        |                      None                       |                          InfoNCE + AE                          |      URL      | 2024 |\n| Graph Completion                             |    Attributed    |         Generative/AE         |                      Attribute Masking                       |                          MAE                          |      P\\\u0026F/JL      | 2020 |\n| Node Attribute Masking                       |    Attributed    |         Generative/AE         |                      Attribute Masking                       |                          MAE                          |      P\\\u0026F/JL      | 2020 |\n| Edge Attribute Masking                       |    Attributed    |         Generative/AE         |                      Attribute Masking                       |                          MAE                          |       P\\\u0026F        | 2019 |\n| Node Attribute and\u003cbr/\u003eEmbedding Denoising   |    Attributed    |         Generative/AE         |                      Attribute Masking                       |                          MAE                          |        JL         | 2020 |\n| Adjacency Matrix\u003cbr/\u003eReconstruction          |    Attributed    |         Generative/AE         |           Attribute Masking\u003cbr/\u003eEdge Perturbation            |                        MAE                         |        JL         | 2020 |\n| Graph Bert                                   |    Attributed    |         Generative/AE         |           Attribute Masking\u003cbr/\u003eEdge Perturbation            |                          MAE                          |       P\\\u0026F        | 2020 |\n| Pretrain-Recsys                              |    Attributed    |         Generative/AE         |                      Edge Perturbation                       |                          MAE                          |       P\\\u0026F        | 2021 |\n| GPT-GNN                                      |  Heterogeneous   |         Generative/AR         |           Attribute Masking\u003cbr/\u003eEdge Perturbation            |                      MAE/InfoNCE                      |       P\\\u0026F        | 2020 |\n| GraphCL                                      |    Attributed    |        Contrastive/G-G        | Attribute Masking\u003cbr/\u003eEdge Perturbation\u003cbr/\u003eRandom Walk Sampling |                        InfoNCE                        |        URL        | 2020 |\n| IGSD                                         |    Attributed    |        Contrastive/G-G        |             Edge Perturbation\u003cbr/\u003eEdge Doffisopm             |                        InfoNCE                        |      JL/URL       | 2020 |\n| DACL                                         |    Attributed    |        Contrastive/G-G        |                            Mixup                             |                        InfoNCE                        |        URL        | 2020 |\n| LCC                                          |    Attributed    |        Contrastive/G-G        |                             None                             |                        InfoNCE                        |        JL         | 2021 |\n| CCGL | Attributed | Contrastive/G-G | Information Re-Diffusion | InfoNCE | P\\\u0026F | 2021 |\n| CSSL                                         |    Attributed    |        Contrastive/G-G        |   NodeInsertion\u003cbr/\u003eEdge Perturbation\u003cbr/\u003eUniform Sampling   |                        InfoNCE                        |    P\\\u0026F/JL/URL    | 2020 |\n| GCC                                          |   Unattributed   |        Contrastive/C-C        |                Random\u0026nbsp;Walk\u0026nbsp;Sampling                |                        InfoNCE                        |     P\\\u0026F/URL      | 2020 |\n| GRACE                                        |    Attributed    |        Contrastive/L-L        |           Attribute Masking\u003cbr/\u003eEdge Perturbation            |                        InfoNCE                        |        URL        | 2020 |\n| GCA                                          |    Attributed    |        Contrastive/L-L        |                       Attention-based                        |                        InfoNCE                        |        URL        | 2020 |\n| GROC                                         |    Attributed    |        Contrastive/L-L        |                        Gradient-based                        |                        InfoNCE                        |        URL        | 2021 |\n| SEPT                                         |    Attributed    |        Contrastive/L-L        |                        Edge Perturbation                        |                        InfoNCE                        |        JL        | 2021 |\n| STDGI                                        | Spatial-Temporal |        Contrastive/L-L        |                     Attribute Shuffling                      |                     JS Estimator                      |        URL        | 2019 |\n| GMI                                          |    Attributed    |        Contrastive/L-L        |                             None                             |                     SP Estimator                      |        URL        | 2020 |\n| KS2L                                         |    Attributed    |        Contrastive/L-L        |                             None                             |                        InfoNCE                        |        URL        | 2020 |\n| CG3                                          |    Attributed    |        Contrastive/L-L        |                             None                             |                        InfoNCE                        |        JL         | 2020 |\n| BGRL                                         |    Attributed    |        Contrastive/L-L        |           Attribute Masking\u003cbr/\u003eEdge Perturbation            |                     Inner Product                     |        URL        | 2021 |\n| SelfGNN                                      |    Attributed    |        Contrastive/L-L        |             Attribute Masking\u003cbr/\u003eEdge Diffusion             |                          MSE                          |        URL        | 2021 |\n| HeCo                                      |    Heterogeneous    |        Contrastive/L-L        |             None             |                          InfoNCE                          |        URL        | 2021 |\n| PT-DGNN                                      |     Dynamic      |        Contrastive/L-L        |           Attribute Masking\u003cbr/\u003eEdge Perturbation            |                       InforNCE                        |       P\\\u0026F        | 2021 |\n| COAD                                         |    Attributed    |        Contrastive/L-L        |                             None                             |                     Triplet Margin Loss                      |       P\\\u0026F        | 2020 |\n| Contrst-Reg                                  |    Attributed    |        Contrastive/L-L        |                     Attribute Shuffling                      |                        InfoNCE                        |        JL         | 2021 |\n| DGI                                          |    Attributed    |        Contrastive/L-G        |                          Arbitrary                           |                     JS Estimator                      |        URL        | 2019 |\n| HDMI                                         |    Attributed    |        Contrastive/L-G        |                     Attribute Shuffling                      |                     JS Estimator                      |        URL        | 2021 |\n| DMGI                                         |  Heterogeneous   |        Contrastive/L-G        |                     Attribute Shuffling                      |                   JS Estimator/MAE                    |        URL        | 2020 |\n| MVGRL                                        |    Attributed    |        Contrastive/L-G        | Attribute Masking\u003cbr/\u003eEdge Perturbation\u003cbr/\u003eEdge Diffusion\u003cbr/\u003eRandom Walk Sampling | DV Estimator\u003cbr/\u003eJS Estimator\u003cbr/\u003eNT-Xent\u003cbr/\u003eInfoNCE |        URL        | 2020 |\n| HDGI                                         |  Heterogeneous   |        Contrastive/L-G        |                     Attribute Shuffling                      |                     JS Estimator                      |        URL        | 2019 |\n| Subg-Con                                     |    Attributed    |        Contrastive/L-C        |                     Importance Sampling                      |                    Triplet Margin Loss                     |        URL        | 2020 |\n| Cotext Prediction                            |    Attributed    |        Contrastive/L-C        |                      Ego-nets Sampling                       |                          Cross Entropy                           |       P\\\u0026F        | 2019 |\n| GIC                                          |    Attributed    |        Contrastive/L-C        |                          Arbitrary                           |                     JS Estimator                      |        URL        | 2020 |\n| GraphLoG                                     |    Attributed    |        Contrastive/L-C        |                      Attribute Masking                       |                        InfoNCE                        |        URL        | 2021 |\n| MHCN                                         |  Heterogeneous   |        Contrastive/L-C        |                     Attribute Shuffling                      |                        InfoNCE                        |        JL         | 2021 |\n| EGI                                          |    Attributed    |        Contrastive/L-C        |                      Ego-nets Sampling                       |                     SP Estimator                      |       P\\\u0026F        | 2020 |\n| MICRO-Graph                                  |    Attributed    |        Contrastive/C-G        |                      Knowledge Sampling                      |                        InfoNCE                        |        URL        | 2020 |\n| InfoGraph                                    |    Attributed    |        Contrastive/C-G        |                             None                             |                     SP Estimator                      |        URL        | 2019 |\n| SUGAR                                        |    Attributed    |        Contrastive/C-G        |                         BFS Sampling                         |                     JS Estimator                      |        JL         | 2021 |\n| BiGI                                         |  Heterogeneous   |        Contrastive/C-G        |           Edge Perturbation\u003cbr/\u003eEgo-nets Sampling            |                     JS Estimator                      |        JL         | 2021 |\n| HTC                                          |    Attributed    |        Contrastive/C-G        |                     Attribute Shuffling                      |             SP Estimator\u003cbr/\u003eDV Estimator             |        URL        | 2021 |\n| Node\u0026nbsp;Property\u0026nbsp;Prediction           |    Attributed    | Predictive/Node\u0026nbsp;Property |                             None                             |                          MAE                          |      P\\\u0026F/JL      | 2020 |\n| S2GRL                                        |    Attributed    |   Predictive/Context-based    |                             None                             |                          Cross Entropy                           |        URL        | 2020 |\n| PairwiseDistance                             |    Attributed    |   Predictive/Context-based    |                             None                             |                          Cross Entropy                           |      P\\\u0026F/JL      | 2020 |\n| PairwiseAttrSim                              |    Attributed    |   Predictive/Context-based    |                             None                             |                          MAE                          |      P\\\u0026F/JL      | 2020 |\n| Distance2Cluster                             |    Attributed    |   Predictive/Context-based    |                             None                             |                          MAE                          |      P\\\u0026F/JL      | 2020 |\n| EdgeMask                                     |    Attributed    |   Predictive/Context-based    |                             None                             |                          Cross Entropy                           |      P\\\u0026F/JL      | 2020 |\n| TopoTER                                      |    Attributed    |   Predictive/Context-based    |                      Edge Perturbation                       |                          Cross Entropy                           |        URL        | 2021 |\n| Centrality Score Ranking                     |    Attributed    |   Predictive/Context-based    |                             None                             |                          Cross Entropy                           |       P\\\u0026F        | 2019 |\n| Meta-path prediction                         |  Heterogeneous   |   Predictive/Context-based    |                             None                             |                          Cross Entropy                           |        JL         | 2020 |\n| SLiCE                                        |  Heterogeneous   |   Predictive/Context-based    |                             None                             |                          Cross Entropy                           |       P\\\u0026F        | 2020 |\n| Distance2Labeled                             |    Attributed    |   Predictive/Context-based    |                             None                             |                          MAE                          |      P\\\u0026F/JL      | 2020 |\n| ContextLabel                                 |    Attributed    |   Predictive/Context-based    |                             None                             |                          MAE                          |      P\\\u0026F/JL      | 2020 |\n| HCM                                          |    Attributed    |   Predictive/Context-based    |                      Edge Perturbation                       |                Bayesian\u0026nbsp;Inference                |        URL        | 2021 |\n| Contextual Molecular\u003cbr/\u003eProperty Prediction |    Attributed    |    Predictive/Domain-based    |                             None                             |                          Cross Entropy                           |       P\\\u0026F        | 2020 |\n| Graph-level Motif Prediction                 |    Attributed    |    Predictive/Domain-based    |                             None                             |                          Cross Entropy                           |       P\\\u0026F        | 2020 |\n| Multi-stage Self-training                    |    Attributed    |   Predictive/Self-training    |                             None                             |                         None                          |        JL         | 2018 |\n| Node Clustering                              |    Attributed    |   Predictive/Self-training    |                             None                             |                      Clustering                       |      P\\\u0026F/JL      | 2020 |\n| Graph Partitioning                           |    Attributed    |   Predictive/Self-training    |                             None                             |                     Graph Partitioning                      |      P\\\u0026F/JL      | 2020 |\n| CAGAN                                        |    Attributed    |   Predictive/Self-training    |                             None                             |                      Clustering                       |        URL        | 2020 |\n| M3S                                          |    Attributed    |   Predictive/Self-training    |                             None                             |                      Clustering                       |        JL         | 2020 |\n| Cluster Preserving                           |    Attributed    |   Predictive/Self-training    |                             None                             |                     Cross Entropy                     |       P\\\u0026F        | 2019 |\n\n\n\n\n\n## A Summary of Implementation Details\n\nAbout Task Level, Evaluation Metric, and Evaluation Datasets.\n\n| Methods                                      |   Task Level    | Evaluation Metric                                            |                           Dataset                            |\n| :------------------------------------------- | :-------------: | :----------------------------------------------------------- | :----------------------------------------------------------: |\n| CDNMF                                        |      Node       | Node Clustering (Acc, NMI)                                   |                    Cora, Citeseer, Pubmed                    |\n| Graph Completion                             |      Node       | Node Classification (Acc)                                    |                    Cora, Citeseer, Pubmed                    |\n| Node Attribute Masking                       |      Node       | Node Classification (Acc)                                    |                Cora, Citeseer, Pubmed, Reddit                |\n| Edge Attribute Masking                       |      Graph      | Graph Classification (ROC-AUC)                               | MUTAG, PTC, PPI, BBBP, Tox21, ToxCast, ClinTox, MUV, HIV, SIDER, BACE |\n| Node Attribute and\u003cbr/\u003eEmbedding Denoising   |      Node       | Node Classification (Acc)                                    |                    Cora, Citeseer, Pubmed                    |\n| Adjacency Matrix\u003cbr/\u003eReconstruction          |      Node       | Node Classification (Acc)                                    |                    Cora, Citeseer, Pubmed                    |\n| Graph Bert                                   |      Node       | Node Classification (Acc)\u003cbr/\u003eNode Clustering (NMI)          |                    Cora, Citeseer, Pubmed                    |\n| Pretrain-Recsys                              |    Node/Link    | -                                                            |                   ML-1M, MOOCs and Last-FM                   |\n| GPT-GNN                                      |    Node/Link    | Node Classification (F1-score)\u003cbr/\u003eLink Prediction (ROC-AUC) |                     OAG, Amazon, Reddit                      |\n| GraphCL                                      |      Graph      | Graph Classification  (Acc, ROC-AUC)                         | NCI1, PROTEINS, D\\\u0026D, COLLAB, RDT-B, RDT-M5K, GITHUB, MNIST, CIFAR10, MUTAG, IMDB-B, BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, BACE, PPI |\n| IGSD                                         |      Graph      | Graph Classification (Acc)                                   |      MUTAG, PTC\\_MR, NCI1, IMDB-B, QM9, COLLAB, IMDB-M       |\n| DACL                                         |      Graph      | Graph Classification (Acc)                                   |        MUTAG, PTC\\_MR, IMDB-B, IMDB-M, RDT-B, RDT-M5K        |\n| LCC                                          |      Graph      | Graph Classification (Acc)                                   |   IMDB-B, IMDB-M, COLLAB, MUTAG, PROTEINS, PTC, NCI1, D\\\u0026D   |\n| CCGL | Graph | Cascade Graph Prediction (MSLE) | Weibo, Twitter, ACM, APS, DBLP |\n| CSSL                                         |      Graph      | Graph Classification (Acc)                                   |          PROTEINS, D\\\u0026D, NCI1, NCI109, Mutagenicity          |\n| GCC                                          |   Node/Graph    | Node Classification (Acc)\u003cbr/\u003eGraph Classification (Acc)     | US-Airport, H-index, COLLAB, IMDB-B, IMDB-M, RDT-B, RDT-M5K  |\n| GRACE                                        |      Node       | Node Classification (Acc, Micro-F1)                          |          Cora, Citeseer, Pubmed, DBLP, Reddit, PPI           |\n| GCA                                          |      Node       | Node Classification (Acc)                                    | Wiki-CS, Amazon-Computers, Amazon-Photo, Coauthor-CS, Coauthor-Physics |\n| GROC                                         |      Node       | Node Classification (Acc)                                    |        Cora, Citeseer, Pubmed, Amazon-Photo, Wiki-CS         |\n| SEPT                                         |      Node/Link       | -                                    |        Last-FM, Douban, Yelp       |\n| STDGI                                        |      Node       | Node\u0026nbsp;Regression\u0026nbsp;(MAE,\u0026nbsp;RMSE,\u0026nbsp;MAPE)        |                           METR-LA                            |\n| GMI                                          |    Node/Link    | Node Classification (Acc, Micro-F1)\u003cbr/\u003eLink Prediction (ROC-AUC) |   Cora, Citeseer, PubMed, Reddit, PPI, BlogCatalog, Flickr   |\n| KS2L                                         |    Node/Link    | Node Classification (Acc)\u003cbr/\u003eLink Prediction (ROC-AUC)      | Cora, Citeseer, Pubmed, Amazon-Computers, Amazon-Photo, Coauthor-CS |\n| CG3                                          |      Node       | Node Classification (Acc)                                    | Cora, Citeseer, Pubmed, Amazon-Computers, Amazon-Photo, Coauthor-CS |\n| BGRL                                         |      Node       | Node Classification (Acc, Micro-F1)                          | Wiki-CS, Amazon-Computers, Amazon-Photo, PPI, Coauthor-CS, Coauthor-Physics, ogbn-arxiv |\n| SelfGNN                                      |      Node       | Node Classification (Acc)                                    | Cora, Citeseer, Pubmed, Amazon-Computers, Amazon-Photo, Coauthor-CS, Coauthor-Physics |\n| HeCo                                         |      Node       | Node Classification\u003cbr/\u003e(ROC-AUC, Micro-F1, Macro-F1)\u003cbr/\u003eNode Clustering (NMI, ARI)                                    |        ACM, DBLP, Freebase, AMiner         |\n| PT-DGNN                                      |      Link       | Link Prediction (ROC-AUC)                                    |               HepPh, Math Overflow, Super User               |\n| COAD                                         |    Node/Link    | Node Clustering\u003cbr/\u003e(Precision, Recall, F1-score)\u003cbr/\u003eLink Prediction (HitRatio@K, MRR) |                    AMiner, News, LinkedIn                    |\n| Contrast-Reg                                 |    Node/Link    | Node Classification (Acc)\u003cbr/\u003eNode Clustering\u003cbr/\u003e(NMI, Acc, Macro-F1)\u003cbr/\u003eLink Prediction (ROC-AUC) | Cora, Citeseer, Pubmed, Reddit, ogbn-arxiv,  Wikipedia, ogbn-products, Amazo-Computers, Amazo-Photo |\n| DGI                                          |      Node       | Node Classification (Acc, Micro-F1)                          |             Cora, Citeseer, Pubmed, Reddit, PPI              |\n| HDMI                                         |      Node       | Node Classification\u003cbr/\u003e(Micro-F1, Macro-F1)\u003cbr/\u003eNode Clustering (NMI) |                   ACM, IMDB, DBLP, Amazon                    |\n| DMGI                                         |      Node       | Node Clustering (NMI)\u003cbr/\u003eNode Classification (Acc)          |                   ACM, IMDB, DBLP, Amazon                    |\n| MVGRL                                        |   Node/Graph    | Node Classification (Acc)\u003cbr/\u003eNode Clustering (NMI, ARI)\u003cbr/\u003eGraph Classification (Acc) | Cora, Citeseer, Pubmed, MUTAG, PTC\\_MR, IMDB-B, IMDB-M, RDT-B |\n| HDGI                                         |      Node       | Node Classification\u003cbr/\u003e(Micro-F1, Macro-F1)\u003cbr/\u003eNode Clustering (NMI, ARI) |                       ACM, DBLP, IMDB                        |\n| Subg-Con                                     |      Node       | Node Classification (Acc, Micro-F1)                          |         Cora, Citeseer, Pubmed, PPI, Flickr, Reddit          |\n| Cotext Prediction                            |      Graph      | Graph Classification (ROC-AUC)                               | MUTAG, PTC, PPI, BBBP, Tox21, ToxCast, ClinTox, MUV, HIV, SIDER, BACE |\n| GIC                                          |    Node/Link    | Node Classification (Acc)\u003cbr/\u003eNode Clustering (Acc, NMI, ARI)\u003cbr/\u003eLink Prediction (ROC-AUC, ROC-AP) | Cora, Citeseer, Pubmed, Amazon-Computers, Amazon-Photo, Coauthor-CS, Coauthor-Physics |\n| GraphLoG                                     |      Graph      | Graph Classification (ROC-AUC)                               |     BBBP, Tox21, ToxCast, ClinTox, MUV, HIV, SIDER, BACE     |\n| MHCN                                         |    Node/Link    | -                                                            |                    Last-FM, Douban, Yelp                     |\n| EGI                                          |    Node/Link    | Node Classification (Acc)\u003cbr/\u003eLink Prediction (ROC-AUC, MRR) |                        YAGO, Airport                         |\n| MICRO-Graph                                  |      Graph      | Graph Classification (ROC-AUC)                               |       BBBP, Tox21, ToxCast, ClinTox, HIV, SIDER, BACE        |\n| InfoGraph                                    |      Graph      | Graph Classification (Acc)                                   |     MUTAG, PTC\\_MR, RDT-B, RDT-M5K, IMDB-B, QM9, IMDB-M      |\n| SUGAR                                        |      Graph      | Graph Classification (Acc)                                   |           MUTAG, PTC, PROTEINS, D\\\u0026D, NCI1, NCI109           |\n| BiGI                                         |      Link       | Link Prediction (AUC-ROC, AUC-PR)                            |               DBLP, ML-100K, ML-1M, Wikipedia                |\n| HTC                                          |      Graph      | Graph Classification (Acc)                                   |     MUTAG, PTC\\_MR, IMDB-B, IMDB-M, RDT-B, QM9, RDT-M5K      |\n| Node\u0026nbsp;Property\u0026nbsp;Prediction           |      Node       | Node Classification (Acc)                                    |                Cora, Citeseer, Pubmed, Reddit                |\n| S2GRL                                        |    Node/Link    | Node Classification (Acc, Micro-F1)\u003cbr/\u003eNode Clustering (NMI)\u003cbr/\u003eLink Prediction (ROC-AUC) |   Cora, Citeseer, Pubmed, PPI, Flickr, BlogCatalog, Reddit   |\n| PairwiseDistance                             |      Node       | Node Classification (Acc)                                    |                Cora, Citeseer, Pubmed, Reddit                |\n| PairwiseAttrSim                              |      Node       | Node Classification (Acc)                                    |                Cora, Citeseer, Pubmed, Reddit                |\n| Distance2Cluster                             |      Node       | Node Classification (Acc)                                    |                Cora, Citeseer, Pubmed, Reddit                |\n| EdgeMask                                     |      Node       | Node Classification (Acc)                                    |                Cora, Citeseer, Pubmed, Reddit                |\n| TopoTER                                      |   Node/Graph    | Node Classification (Acc)\u003cbr/\u003eGraph Classification (Acc)     | Cora, Citeseer, Pubmed, MUTAG, PTC-MR, RDT-B, RDT-M5K, IMDB-B, IMDB-M |\n| Centrality Score Ranking                     | Node/Link/Graph | Node Classification (Micro-F1)\u003cbr/\u003eLink Prediction (Micro-F1)\u003cbr/\u003eGraph Classification (Micro-F1) |         Cora, Pubmed, ML-100K, ML-1M, IMDB-M, IMDB-B         |\n| Meta-path prediction                         |    Node/Link    | Node Classification (F1-score)\u003cbr/\u003eLink Prediction (ROC-AUC) |              ACM, IMDB, Last-FM, Book-Crossing               |\n| SLiCE                                        |      Link       | Link Prediction (ROC-AUC, Micro-F1)                          |         Amazon, DBLP, Freebase, Twitter, Healthcare          |\n| Distance2Labeled                             |      Node       | Node Classification (Acc)                                    |                Cora, Citeseer, Pubmed, Reddit                |\n| ContextLabel                                 |      Node       | Node Classification (Acc)                                    |                Cora, Citeseer, Pubmed, Reddit                |\n| HCM                                          |      Node       | Node Classification (ROC-AUC)                                |           ACM, Amazon, Enron, BlogCatalog, Flickr            |\n| Contextual Molecular\u003cbr/\u003eProperty Prediction |      Graph      | Graph Classification (Acc)\u003cbr/\u003eGraph Regression (MAE)        | BBBP,\u0026nbsp;SIDER,\u0026nbsp;ClinTox,\u0026nbsp;BACE,\u0026nbsp;Tox21,\u0026nbsp;ToxCast,\u0026nbsp;ESOL,\u0026nbsp;FreeSolv,\u0026nbsp;Lipo,\u0026nbsp;QM7,\u0026nbsp;QM8 |\n| Graph-level Motif Prediction                 |      Graph      | Graph Classification (Acc)\u003cbr/\u003eGraph Regression (MAE)        | BBBP, SIDER, ClinTox, BACE, Tox21, ToxCast, ESOL, FreeSolv, Lipo, QM7, QM8 |\n| Multi-stage Self-training                    |      Node       | Node Classification (Acc)                                    |                    Cora, Citeseer, Pubmed                    |\n| Node Clustering                              |      Node       | Node Classification (Acc)                                    |                    Cora, Citeseer, Pubmed                    |\n| Graph Partitioning                           |      Node       | Node Classification (Acc)                                    |                    Cora, Citeseer, Pubmed                    |\n| CAGAN                                        |      Node       | Node Classfication\u003cbr/\u003e(Micro-F1, Macro-F1)\u003cbr/\u003eNode Clustering\u003cbr/\u003e(Micro-F1, Macro-F1, NMI) |                    Cora, Citeseer, Pubmed                    |\n| M3S                                          |      Node       | Node Classification (Acc)                                    |                    Cora, Citeseer, Pubmed                    |\n| Cluster Preserving                           | Node/Link/Graph | Node Classification (Micro-F1)\u003cbr/\u003eLink Prediction (Micro-F1)\u003cbr/\u003eGraph Classification (Micro-F1) |         Cora, Pubmed, ML-100K, ML-1M, IMDB-M, IMDB-B         |\n\n## A Summary of Common Graph Datasets\n\nAbout category, graph number, node number per graph, edge number per graph, dimensionality of node attributes, class number, and citation papers.\n\n|     Dataset      |     Category      | \\#Graph | \\#Node (Avg.) | \\#Edge (Avg.) | \\#Feature | \\#Class |\n| :--------------: | :---------------: | :-----: | :-----------: | :-----------: | :-------: | :-----: |\n|       Cora       | Citation Network  |    1    |     2708      |     5429      |   1433    |    7    |\n|     Citeseer     | Citation Network  |    1    |     3327      |     4732      |   3703    |    6    |\n|      Pubmed      | Citation Network  |    1    |     19717     |     44338     |    500    |    3    |\n|     Wiki-CS      | Citation Network  |    1    |     11701     |    216123     |    300    |   10    |\n|   Coauthor-CS    | Citation Network  |    1    |     18333     |     81894     |   6805    |   15    |\n| Coauthor-Physics | Citation Network  |    1    |     34493     |    247962     |   8415    |    5    |\n|    DBLP (v12)    | Citation Network  |    1    |    4894081    |   45564149    |     -     |    -    |\n|    ogbn-arxiv    | Citation Network  |    1    |    169343     |    1166243    |    128    |   40    |\n|      Reddit      |  Social Network   |    1    |    232965     |   11606919    |    602    |   41    |\n|   BlogCatalog    |  Social Network   |    1    |     5196      |    171743     |   8189    |    6    |\n|      Flickr      |  Social Network   |    1    |     7575      |    239738     |   12047   |    9    |\n|      COLLAB      |  Social Networks  |  5000   |     74.49     |    2457.78    |     -     |    2    |\n|      RDT-B       |  Social Networks  |  2000   |    429.63     |    497.75     |     -     |    2    |\n|     RDT-M5K      |  Social Networks  |  4999   |    508.52     |    594.87     |     -     |    5    |\n|      IMDB-B      |  Social Networks  |  1000   |     19.77     |     96.53     |     -     |    2    |\n|      IMDB-M      |  Social Networks  |  1500   |     13.00     |     65.94     |     -     |    3    |\n|     ML-100K      |  Social Networks  |    1    |     2625      |    100000     |     -     |    5    |\n|      ML-1M       |  Social Networks  |    1    |     9940      |    1000209    |     -     |    5    |\n|       PPI        | Protein Networks  |   24    |     56944     |    818716     |    50     |   121   |\n|       D\\\u0026D       | Protein Networks  |  1178   |    284.32     |    715.65     |    82     |    2    |\n|     PROTEINS     | Protein Networks  |  1113   |     39.06     |     72.81     |     4     |    2    |\n|       NCI1       |  Molecule Graphs  |  4110   |     29.87     |     32.30     |    37     |    2    |\n|      MUTAG       |  Molecule Graphs  |   188   |     17.93     |     19.79     |     7     |    2    |\n|  QM9 (QM7, QM8)  |  Molecule Graphs  | 133885  |       -       |       -       |     -     |    -    |\n|       BBBP       |  Molecule Graphs  |  2039   |     24.05     |     25.94     |     -     |    2    |\n|      Tox21       |  Molecule Graphs  |  7831   |     18.51     |     25.94     |     -     |   12    |\n|     ToxCast      |  Molecule Graphs  |  8575   |     18.78     |     19.26     |     -     |   167   |\n|     ClinTox      |  Molecule Graphs  |  1478   |     26.13     |     27.86     |     -     |    2    |\n|       MUV        |  Molecule Graphs  |  93087  |     24.23     |     26.28     |     -     |   17    |\n|       HIV        |  Molecule Graphs  |  41127  |     25.53     |     27.48     |     -     |    2    |\n|      SIDER       |  Molecule Graphs  |  1427   |     33.64     |     35.36     |     -     |   27    |\n|       BACE       |  Molecule Graphs  |  1513   |     34.12     |     36.89     |     -     |    2    |\n|     PTC      |  Molecule Graphs  |   344   |     14.29     |     14.69     |     19     |    2    |\n|      NCI109      |  Molecule Graphs  |  4127   |     29.68     |     32.13     |     -     |    2    |\n|   Mutagenicity   |  Molecule Graphs  |  4337   |     30.32     |     30.77     |     -     |    2    |\n|      MNIST       |  Others (Image)   |    -    |     70000     |       -       |    784    |   10    |\n|     CIFAR10      |  Others (Image)   |    -    |     60000     |       -       |   1024    |   10    |\n|     METR-LA      | Others (Traffic)  |    1    |      207      |     1515      |     2     |    -    |\n| Amazon-Computers | Others (Purchase) |    1    |     13752     |    245861     |    767    |   10    |\n|   Amazon-Photo   | Others (Purchase) |    1    |     7650      |    119081     |    745    |    8    |\n|  ogbn-products   | Others (Purchase) |    1    |    2449029    |   61859140    |    100    |   47    |\n\n## A Summary of Open-source Codes\n\n| Methods                                  | Github                                                       |\n| :--------------------------------------- | :----------------------------------------------------------- |\n| CDNMF                                    | https://github.com/6lyc/CDNMF                                |\n| Graph Completion                         | https://github.com/Shen-Lab/SS-GCNs                          |\n| Node Attribute Masking                   | https://github.com/ChandlerBang/SelfTask-GNN                 |\n| Edge Attribute Masking                   | http://snap.stanford.edu/gnn-pretrain                        |\n| Attribute and Embedding Denoising        | N.A.                                                         |\n| Adjacency Matrix Reconstruction          | N.A.                                                         |\n| Graph Bert                               | https://github.com/anonymous-sourcecode/Graph-Bert           |\n| Pretrain-Recsys                          | https://github.com/jerryhao66/Pretrain-Recsys                |\n| SLAPS                                    | https://github.com/BorealisAI/SLAPS-GNN                      |\n| G-BERT                                   | https://github.com/jshang123/G-Bert                          |\n| GPT-GNN                                  | https://github.com/acbull/GPT-GNN                            |\n| GraphCL                                  | https://github.com/Shen-Lab/GraphCL                          |\n| IGSD                                     | N.A.                                                         |\n| DACL                                     | N.A.                                                         |\n| LCC                                      | https://github.com/YuxiangRen                                |\n| CCGL                                     | https://github.com/Xovee/ccgl                                |\n| CSSL                                     | N.A.                                                         |\n| GCC                                      | https://github.com/THUDM/GCC                                 |\n| GRACE                                    | https://github.com/CRIPAC-DIG/GRACE                          |\n| GCA                                      | https://github.com/CRIPAC-DIG/GCA                            |\n| GROC                                     | N.A.                                                         |\n| SEPT                                     | https://github.com/Coder-Yu/QRec                             |\n| STDGI                                    | N.A.                                                         |\n| GMI                                      | https://github.com/zpeng27/GMI                               |\n| KS2L                                     | N.A.                                                         |\n| CG3                                      | N.A.                                                         |\n| BGRL                                     | N.A.                                                         |\n| SelfGNN                                  | https://github.com/zekarias-tilahun/SelfGNN                  |\n| HeCo                                     | https://github.com/liun-online/HeCo                          |\n| PT-DGNN                                  | https://github.com/Mobzhang/PT-DGNN                          |\n| COAD                                     | https://github.com/allanchen95/Expert-Linking                |\n| Contrast-Reg                             | N.A.                                                         |\n| C-SWM                                    | https://github.com/tkipf/c-swm                               |\n| DGI                                      | https://github.com/PetarV-/DGI                               |\n| HDMI                                     | N.A.                                                         |\n| DMGI                                     | https://github.com/pcy1302/DMGI                              |\n| MVGRL                                    | https://github.com/kavehhassani/mvgrl                        |\n| HDGI                                     | https://github.com/YuxiangRen/Heterogeneous-Deep-Graph-Infomax |\n| Subg-Con                                 | https://github.com/yzjiao/Subg-Con                           |\n| Cotext Prediction                        | http://snap.stanford.edu/gnn-pretrain                        |\n| GIC                                      | https://github.com/cmavro/Graph-InfoClust-GIC                |\n| GraphLoG                                 | https://openreview.net/forum?id=DAaaaqPv9-q                  |\n| MHCN                                     | https://github.com/Coder-Yu/RecQ                             |\n| EGI                                      | https://openreview.net/forum?id=J_pvI6ap5Mn                  |\n| MICRO-Graph                              | https://drive.google.com/file/d/1b751rpnV-SDmUJvKZZI-AvpfEa9eHxo9/ |\n| InfoGraph                                | https://github.com/fanyun-sun/InfoGraph                      |\n| SUGAR                                    | https://github.com/RingBDStack/SUGAR                         |\n| BiGI                                     | https://github.com/clhchtcjj/BiNE                            |\n| HTC                                      | N.A.                                                         |\n| DITNET                                   | https://github.com/FangpingWan/NeoDTI                        |\n| Node Property Prediction                 | https://github.com/ChandlerBang/SelfTask-GNN                 |\n| S2GRL                                    | N.A.                                                         |\n| PairwiseDistance                         | https://github.com/ChandlerBang/SelfTask-GNN                 |\n| PairwiseAttrSim                          | https://github.com/ChandlerBang/SelfTask-GNN                 |\n| Distance2Cluster                         | https://github.com/ChandlerBang/SelfTask-GNN                 |\n| EdgeMask                                 | https://github.com/ChandlerBang/SelfTask-GNN                 |\n| TopoTER                                  | N.A.                                                         |\n| Centrality Score Ranking                 | N.A.                                                         |\n| Meta-path prediction                     | https://github.com/mlvlab/SELAR                              |\n| SLiCE                                    | https://github.com/pnnl/SLICE                                |\n| Distance2Labeled                         | https://github.com/ChandlerBang/SelfTask-GNN                 |\n| ContextLabel                             | https://github.com/ChandlerBang/SelfTask-GNN                 |\n| HCM                                      | N.A.                                                         |\n| Contextual Molecular Property Prediction | https://github.com/tencent-ailab/grover                      |\n| Graph-level Motif Prediction             | https://github.com/tencent-ailab/grover                      |\n| DrRepair                                 | https://github.com/michiyasunaga/DrRepair                    |\n| Multi-stage Self-training                | https://github.com/Davidham3/deeper_insights_into_GCNs       |\n| Node Clustering                          | https://github.com/Shen-Lab/SS-GCNs                          |\n| Graph Partitioning                       | https://github.com/Shen-Lab/SS-GCNs                          |\n| CAGAN                                    | N.A.                                                         |\n| M3S                                      | https://github.com/datake/M3S                                |\n| Cluster Preserving                       | N.A.                                                         |\n| SEF                                      | https://github.com/nealgravindra/self-supervsed_edge_feats   |\n\n\n## Contribute\n\nIf you would like to help contribute this list, please feel free to contact me or add [pull request](https://github.com/LirongWu/awesome-graph-self-supervised-learning/pulls) with the following Markdown format:\n\n```markdown\n- Paper Name. \n  - Author List. *Conference Year*. [[pdf]](link) [[code]](link)\n```\n\nThis is a Github Summary of our [Survey](https://arxiv.org/abs/2105.07342). If you find this file useful in your research, please consider citing:\n\n```\n@article{wu2021self,\n  title={Self-supervised Learning on Graphs: Contrastive, Generative, or Predictive},\n  author={Wu, Lirong and Lin, Haitao and Tan, Cheng and Gao, Zhangyang and Li, Stan Z},\n  journal={IEEE Transactions on Knowledge and Data Engineering},\n  year={2021},\n  publisher={IEEE}\n}\n```\n\n## Feedback\nIf you have any issue about this work, please feel free to contact me by email: \n* Lirong Wu: wulirong@westlake.edu.cn\n* Haitao Lin: linhaitao@westlake.edu.cn\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLirongWu%2Fawesome-graph-self-supervised-learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FLirongWu%2Fawesome-graph-self-supervised-learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLirongWu%2Fawesome-graph-self-supervised-learning/lists"}