https://github.com/thudm/graphmae2
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner in WWW'23
https://github.com/thudm/graphmae2
deep-learning graph-neural-networks self-supervised-learning
Last synced: 11 months ago
JSON representation
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner in WWW'23
- Host: GitHub
- URL: https://github.com/thudm/graphmae2
- Owner: THUDM
- License: mit
- Created: 2023-04-05T03:44:53.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2023-07-06T12:59:18.000Z (almost 3 years ago)
- Last Synced: 2025-04-10T15:45:48.113Z (about 1 year ago)
- Topics: deep-learning, graph-neural-networks, self-supervised-learning
- Language: Python
- Homepage:
- Size: 230 KB
- Stars: 156
- Watchers: 10
- Forks: 14
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised
Graph Learner
Implementation for WWW'23 paper: [GraphMAE2: A Decoding-Enhanced Masked Self-Supervised
Graph Learner](https://arxiv.org/abs/2304.04779).
[GraphMAE] The predecessor of this work: [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://arxiv.org/abs/2205.10803) can be found [here](https://github.com/THUDM/GraphMAE).
❗ Update
[2023-04-19] We have made **checkpoints** of pre-trained models on different datasets available - feel free to download them from [Google Drive](https://drive.google.com/drive/folders/1GiuP0PtIZaYlJWIrjvu73ZQCJGr6kGkh).
Dependencies
* Python >= 3.7
* [Pytorch](https://pytorch.org/) >= 1.9.0
* pyyaml == 5.4.1
Quick Start
For quick start, you could run the scripts:
**Node classification**
```bash
sh run_minibatch.sh # for mini batch node classification
# example: sh run_minibatch.sh ogbn-arxiv 0
sh run_fullbatch.sh # for full batch node classification
# example: sh run_fullbatch.sh cora 0
# Or you could run the code manually:
# for mini batch node classification
python main_large.py --dataset ogbn-arxiv --encoder gat --decoder gat --seed 0 --device 0
# for full batch node classification
python main_full_batch.py --dataset cora --encoder gat --decoder gat --seed 0 --device 0
```
Supported datasets:
* mini batch node classification: `ogbn-arxiv`, `ogbn-products`, `mag-scholar-f`, `ogbn-papers100M`
* full batch node classification: `cora`, `citeseer`, `pubmed`
Run the scripts provided or add `--use_cfg` in command to reproduce the reported results.
**For Large scale graphs**
Before starting mini-batch training, you'll need to generate local clusters if you want to use local-clustering for training. By default, the program will load dataset from `./data` and save the generated local clusters to `./lc_ego_graphs`. To generate a local cluster, you should first install [localclustering](https://github.com/kfoynt/LocalGraphClustering) and then run the following command:
```
python ./datasets/localclustering.py --dataset --data_dir
```
And we also provide the pre-generated local clusters which can be downloaded [here](https://cloud.tsinghua.edu.cn/d/64f859f389ca43eda472/) and then put into `lc_ego_graphs` for usage.
Datasets
During the code's execution, the OGB and small-scale datasets (Cora, Citeseer, and PubMed) will be downloaded automatically. For the MAG-SCHOLAR dataset, you can download the raw data from [here](https://figshare.com/articles/dataset/mag_scholar/12696653) or use our processed version, which can be found [here](https://cloud.tsinghua.edu.cn/d/776e73d84d47454c958d/) (the four feature files have to be merged in to a `feature_f.npy`). Once you have the dataset, place it into the `./data/mag_scholar_f` folder for later usage. The folder should contain the following files:
```
- mag_scholar_f
|--- edge_index_f.npy
|--- split_idx_f.pt
|--- feature_f.npy
|--- label_f.npy
```
Soon, we will provide [SAINTSampler](https://arxiv.org/abs/1907.04931) as the baseline.
Experimental Results
Experimental results of node classification on large-scale datasets (Accuracy, %):
| | Ogbn-arxiv | Ogbn-products | Mag-Scholar-F | Ogbn-papers100M |
| ------------------ | ------------ | ------------ | ------------ | -------------- |
| MLP | 55.50±0.23 | 61.06±0.08 | 39.11±0.21 | 47.24±0.31 |
| SGC | 66.92±0.08 | 74.87±0.25 | 54.68±0.23 | 63.29±0.19 |
| Random-Init | 68.14±0.02 | 74.04±0.06 | 56.57±0.03 | 61.55±0.12 |
| CCA-SSG | 68.57±0.02 | 75.27±0.05 | 51.55±0.03 | 55.67±0.15 |
| GRACE | 69.34±0.01 | 79.47±0.59 | 57.39±0.02 | 61.21±0.12 |
| BGRL | 70.51±0.03 | 78.59±0.02 | 57.57±0.01 | 62.18±0.15 |
| GGD | - | 75.70±0.40 | - | 63.50±0.50 |
| GraphMAE | 71.03±0.02 | 78.89±0.01 | 58.75±0.03 | 62.54±0.09 |
| **GraphMAE2** | **71.89±0.03** | **81.59±0.02** | **59.24±0.01** | **64.89±0.04** |
Citing
If you find this work is helpful to your research, please consider citing our paper:
```
@inproceedings{hou2023graphmae2,
title={GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner},
author={Zhenyu Hou, Yufei He, Yukuo Cen, Xiao Liu, Yuxiao Dong, Evgeny Kharlamov, Jie Tang},
booktitle={Proceedings of the ACM Web Conference 2023 (WWW’23)},
year={2023}
}
```