{"id":19979274,"url":"https://github.com/edisonleeeee/maskgae","last_synced_at":"2025-04-09T11:11:59.420Z","repository":{"id":55052552,"uuid":"494317543","full_name":"EdisonLeeeee/MaskGAE","owner":"EdisonLeeeee","description":"[KDD 2023] What’s Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders ","archived":false,"fork":false,"pushed_at":"2024-10-29T06:45:34.000Z","size":3223,"stargazers_count":84,"open_issues_count":1,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-02T09:08:15.558Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2205.10053","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EdisonLeeeee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-20T04:18:00.000Z","updated_at":"2025-03-16T09:48:06.000Z","dependencies_parsed_at":"2023-02-17T20:30:47.594Z","dependency_job_id":"eb9836d1-012e-453a-be23-96e456594e2d","html_url":"https://github.com/EdisonLeeeee/MaskGAE","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EdisonLeeeee%2FMaskGAE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EdisonLeeeee%2FMaskGAE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EdisonLeeeee%2FMaskGAE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EdisonLeeeee%2FMaskGAE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EdisonLeeeee","download_url":"https://codeload.github.com/EdisonLeeeee/MaskGAE/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248027411,"owners_count":21035594,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T03:37:23.038Z","updated_at":"2025-04-09T11:11:59.394Z","avatar_url":"https://github.com/EdisonLeeeee.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MaskGAE\n\n\n\u003e [**What’s Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders**](https://arxiv.org/abs/2205.10053) (KDD 2023)\\\n\u003e [**MaskGAE: Masked Graph Modeling Meets Graph Autoencoders**](https://arxiv.org/abs/2205.10053v1) (arXiv 2022)\n\u003e\n\u003e Jintang Li, Ruofan Wu, Wangbin Sun, Liang Chen, Sheng Tian, Liang Zhu, Changhua Meng, Zibin Zheng, Weiqiang Wang    \n\n**This repository is an official PyTorch implementation of MaskGAE.**\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figs/maskgae.png\"/\u003e\n\u003cp align=\"center\"\u003e\u003cem\u003eFig. 1. MaskGAE framework and masking strategies.\u003c/em\u003e\n\u003c/p\u003e\n\n# Abstract\nThe last years have witnessed the emergence of a promising self-supervised learning strategy, referred to as masked autoencoding. However, there is a lack of theoretical understanding of how masking matters on graph autoencoders (GAEs). In this work, we present masked graph autoencoder (MaskGAE), a self-supervised learning framework for graph-structured data. Different from standard GAEs, MaskGAE adopts masked graph modeling (MGM) as a principled pretext task - masking a portion of edges and attempting to reconstruct the missing part with partially visible, unmasked graph structure. To understand whether MGM can help GAEs learn better representations, we provide both theoretical and empirical evidence to comprehensively justify the benefits of this pretext task. Theoretically, we establish close connections between GAEs and contrastive learning, showing that MGM significantly improves the self-supervised learning scheme of GAEs. Empirically, we conduct extensive experiments on a variety of graph benchmarks, demonstrating the superiority of MaskGAE over several state-of-the-arts on both link prediction and node classification tasks.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"figs/comparison.png\"/\u003e\n\u003cp align=\"center\"\u003e\u003cem\u003eFig. 2. Comparison of masked language modeling (MLM), masked image modeling (MIM) and masked graph modeling (MGM).\u003c/em\u003e\n\u003c/p\u003e\n\n\n# News\n\n+ Oct 15, 2024: 🔥🔥🔥 We released a new Graph Autoencoders Benchmark --- [lrGAE](https://github.com/EdisonLeeeee/lrGAE).\n+ Nov 30, 2023: 🔥🔥🔥 MaskGAE based solution has just won the 2nd place in [ICDM Cup 2023](https://tugraph.antgroup.com/blog?id=15): community detection based on graph pretrained models. Code is open-sourced [here](https://github.com/Echochef/ICDM), congrats!\n+ Oct 27, 2023: 🔥🔥🔥 Check out our poster [here](./poster.pdf)!\n\n# Requirements\nHigher versions should be also available.\n\n+ numpy==1.21.6\n+ torch==1.12.1+cu102\n+ torch-cluster==1.6.0\n+ torch_geometric\u003e=2.4.0\n+ torch-scatter==2.0.9\n+ torch-sparse==0.6.14\n+ scipy==1.7.3\n+ texttable==1.6.2\n+ CUDA 10.2\n+ CUDNN 7.6.0\n\n# Installation\n\n```bash\npip install -r requirements.txt\n```\n\n# Dataset\n| Dataset      | #Nodes     | #Edges     | #Features | #Classes | Density   |\n|--------------|------------|------------|-----------|----------|-----------|\n| Cora         | 2,708      | 10,556     | 1,433     | 7        | 0.144%    |\n| CiteSeer     | 3,327      | 9,104      | 3,703     | 6        | 0.082%    |\n| Pubmed       | 19,717     | 88,648     | 500       | 3        | 0.023%    |\n| Photo        | 7,650      | 238,162    | 745       | 8        | 0.407%    |\n| Computer     | 13,752     | 491,722    | 767       | 10       | 0.260%    |\n| arXiv        | 16,9343    | 2,315,598  | 128       | 40       | 0.008%    |\n| MAG          | 736,389    | 10,792,672 | 128       | 349      | 0.002%    |\n| Collab       | 235,868    | 1,285,465  | 128       | -        | 0.002%    |\n\nAll datasets used throughout experiments are publicly available in [PyTorch Geometric library](https://github.com/pyg-team/pytorch_geometric).\n\n# Reproduction\n## Clone this project\n```bash\ngit clone https://github.com/EdisonLeeeee/MaskGAE.git\ncd MaskGAE\n```\n\n## Link prediction experiments\n+ Cora\n```bash\npython train_linkpred.py --dataset Cora --bn\npython train_linkpred.py --dataset Cora --bn --mask Edge\n```\n+ Citeseer\n```bash\npython train_linkpred.py --dataset Citeseer --bn\npython train_linkpred.py --dataset Citeseer --bn --mask Edge\n```\n+ Pubmed\n```bash\npython train_linkpred.py --dataset Pubmed --bn --encoder_dropout 0.2\npython train_linkpred.py --dataset Pubmed --bn --encoder_dropout 0.2 --mask Edge\n```\n+ Collab\n```bash\npython train_linkpred_ogb.py\npython train_linkpred_ogb.py --mask Edge\n```\n\n## Node classification experiments\n\n+ Cora\n```bash\npython train_nodeclas.py --dataset Cora --bn --l2_normalize --alpha 0.004 --full_data\npython train_nodeclas.py --dataset Cora --bn --l2_normalize --alpha 0.003 --mask Edge --eval_period 10\n```\n+ Citeseer\n```bash\npython train_nodeclas.py --dataset Citeseer --bn --l2_normalize --nodeclas_weight_decay 0.1 --alpha 0.001 --lr 0.02 --full_data\npython train_nodeclas.py --dataset Citeseer --bn --l2_normalize --nodeclas_weight_decay 0.1 --alpha 0.001  --lr 0.02 --mask Edge\n```\n+ Pubmed\n```bash\npython train_nodeclas.py --dataset Pubmed --bn --l2_normalize --alpha 0.001  --encoder_dropout 0.5 --decoder_dropout 0.5 --full_data\npython train_nodeclas.py --dataset Pubmed --bn --l2_normalize --alpha 0.001  --encoder_dropout 0.5 --mask Edge\n```\n+ Photo\n```bash\npython train_nodeclas.py --dataset Photo --bn --nodeclas_weight_decay 5e-3 --decoder_channels 128 --lr 0.005\npython train_nodeclas.py --dataset Photo --bn --nodeclas_weight_decay 5e-3 --decoder_channels 64 --mask Edge\n```\n+ Computers\n```bash\npython train_nodeclas.py --dataset Computers --bn --encoder_dropout 0.5 --alpha 0.002 --encoder_channels 128 --hidden_channels 256 --eval_period 20\npython train_nodeclas.py --dataset Computers --bn --encoder_dropout 0.5 --alpha 0.003 --encoder_channels 128 --hidden_channels 256 --eval_period 10 --mask Edge\n```\n+ arxiv\n```bash\npython train_nodeclas.py --dataset arxiv --bn --decoder_channels 128 --decoder_dropout 0. --decoder_layers 4 \\\n                          --encoder_channels 256 --encoder_dropout 0.2 --encoder_layers 4 \\\n                          --hidden_channels 512 --lr 0.0005 --nodeclas_weight_decay 0 --weight_decay 0.0001 --epochs 100  \\\n                          --eval_period 10\npython train_nodeclas.py --dataset arxiv --bn --decoder_channels 128 --decoder_dropout 0. --decoder_layers 4 \\\n                          --encoder_channels 256 --encoder_dropout 0.2 --encoder_layers 4 \\\n                          --hidden_channels 512 --lr 0.0005 --nodeclas_weight_decay 0 --weight_decay 0.0001 --epochs 100  \\\n                          --eval_period 10 --mask Edge\n```\n+ MAG\n```bash\npython train_nodeclas.py --dataset mag --alpha 0.003 --bn --decoder_channels 128\\\n                         --encoder_channels 256 --encoder_dropout 0.7 --epochs 100 \\\n                         --hidden_channels 128 --nodeclas_weight_decay 1e-5 --weight_decay 5e-5 --eval_period 10                                       \npython train_nodeclas.py --dataset mag --alpha 0.003 --bn --decoder_channels 128\n                         --encoder_channels 256 --encoder_dropout 0.7 --epochs 100 \\\n                         --hidden_channels 128 --nodeclas_weight_decay 1e-5 --weight_decay 5e-5 --eval_period 10 --mask Edge   \n```\n\n\n# Citation\n\n```bibtex\n@inproceedings{maskgae,\n  author       = {Jintang Li and\n                  Ruofan Wu and\n                  Wangbin Sun and\n                  Liang Chen and\n                  Sheng Tian and\n                  Liang Zhu and\n                  Changhua Meng and\n                  Zibin Zheng and\n                  Weiqiang Wang},\n  title        = {What's Behind the Mask: Understanding Masked Graph Modeling for Graph\n                  Autoencoders},\n  booktitle    = {{KDD}},\n  pages        = {1268--1279},\n  publisher    = {{ACM}},\n  year         = {2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedisonleeeee%2Fmaskgae","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fedisonleeeee%2Fmaskgae","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedisonleeeee%2Fmaskgae/lists"}