{"id":13830062,"url":"https://github.com/yang-han/P-reg","last_synced_at":"2025-07-09T10:32:31.069Z","repository":{"id":119122227,"uuid":"261092241","full_name":"yang-han/P-reg","owner":"yang-han","description":"Rethinking Graph Regularization for Graph Neural Networks (AAAI2021)","archived":false,"fork":false,"pushed_at":"2021-06-06T07:15:22.000Z","size":544,"stargazers_count":32,"open_issues_count":0,"forks_count":8,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-08-04T10:02:21.348Z","etag":null,"topics":["aaai2021","gcn","gnn","graph-neural-networks","regularization"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2009.02027","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yang-han.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-05-04T05:56:32.000Z","updated_at":"2024-07-30T12:05:21.000Z","dependencies_parsed_at":"2024-01-15T17:39:00.464Z","dependency_job_id":"7c201611-51f4-4b84-be3b-2ba619d89098","html_url":"https://github.com/yang-han/P-reg","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yang-han%2FP-reg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yang-han%2FP-reg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yang-han%2FP-reg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yang-han%2FP-reg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yang-han","download_url":"https://codeload.github.com/yang-han/P-reg/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225533040,"owners_count":17484185,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aaai2021","gcn","gnn","graph-neural-networks","regularization"],"created_at":"2024-08-04T10:00:54.639Z","updated_at":"2024-11-20T10:31:47.179Z","avatar_url":"https://github.com/yang-han.png","language":"Python","funding_links":[],"categories":["[Label Inference Methods](#content)"],"sub_categories":["Graph Regularization Approaches"],"readme":"# [AAAI2021] Rethinking Regularization for Graph Neural Networks\n\nThis is the source code to reproduce the experimental results for *[Rethinking Graph Regularization for Graph Neural Networks](https://arxiv.org/abs/2009.02027)*.\n\nThe code for graph-level experiments is in the `./graph_level/` sub-folder.\n\n## Dependencies\n\n```shell\npython==3.7.6\npytorch==1.5.0\npytorch_geometric==1.4.3\nnumpy==1.18.1\n```\n\n## Code Description\n\n### `main.py`\n\nThe entry file. Load the datasets and models, train and evaluate the model.\n\n### `conv.py`\n\nThe `IConv` is modified from the `torch.geometric.nn.GCNConv`, to implement the propagation of output, i.e., $\\hat{A}Z$ in the paper.\n\nComparing to the original `GCNConv`, `IConv` removed the `bias` matrix, and replaced the `weight` matrix by an untrainable Identity matrix.\n\n### `models.py`\n\n`GCN`, `GAT` and `MLP` are implemented in a standard way and provided in this file.\n\n`PREGGCN`, `PREGGAT` and `PREGMLP` have an additional method `propagation()`, which is to further propagate the output of the vanilla `GCN`,`GAT` and `MLP` models.\n\nA typical Propagation-regularization can be computed as:\n\n```python3\nsoft_cross_entropy(\n    F.softmax(\n        model.propagation(data.x, data.edge_index),\n        dim=1\n    ),\n    F.softmax(\n        model(data.x, data.edge_index),\n        dim=1\n    )\n)\n```\n\n### `phi.py`\n\n`soft_cross_entropy()`, `kl_div()`, `squared_error()` are provided in `phi.py` as different $\\phi$ functions.\n\n### `loss.py`\n\n`LabelSmoothingLoss`, `confidence_penalty` and `laplacian_reg` are provided in `loss.py` as baselines.\n\n### `utils.py`\n\nSome useful functions are implemented in this file.\n\n`generate_split()` is used to generate the random splits for each dataset. And the generated splits we used in our experiments are in the `./splits/` folder.\n\n`Mask` is the structure that random `split` are stored.\n\n`load_dataset(), load_split()` are provided to load the datasets and random splits.\n\n## Reproducing Experimental Results\n\n### Random splits (in Table 1)\n\nPassing `--num_splits 5` to `main.py` means using the first 5 randomly generated splits provided in the `./splits/` folder. Set `--mu 0` to use the vanilla models without P-reg.\n\n```bash\nmodels: ['PREGGCN', 'PREGGAT', 'PREGMLP']\ndatasets: ['cora', 'citeseer', 'pubmed', 'cs', 'physics', 'computers', 'photo']\n```\n\nThe command to train and evaluate a model is:\n\n```bash\npython main.py --dataset $dataset --model $model --mu $mu --num_seeds $num_seeds --num_splits $num_splits\n```\n\nFor example, experiments with GCN+P-reg (mu=0.5) on CORA dataset for 5 splits and 5 seeds for each split:\n\n```bash\npython main.py --dataset cora --model preggcn --mu 0.5 --num_seeds 5 --num_splits 5\n```\n\nFor complete commands to run all experiments, please refer to `random_run.sh`.\n\n### Plantoid standard split (in Table 2)\n\nPassing `--num_splits 1` to `main.py` means using the standard split of the Plaintoid datasets. Set `--mu 0` to use the vanilla models without P-reg.\n\n```bash\nmodels: ['PREGGCN', 'PREGGAT']\ndatasets: ['cora', 'citeseer', 'pubmed']\n```\n\nCommands to reproduce experimental results on CORA, CiteSeer and PubMed datasets:\n\n```bash\n# CORA GAT+P-reg mu=0.45 standard split 10 seeds\npython main.py --num_splits 1 --num_seeds 10 --dataset cora --model preggat --mu 0.45\n# CiteSeer GCN+P-reg mu=0.35 standard split 10 seeds\npython main.py --num_splits 1 --num_seeds 10 --dataset citeseer --model preggcn --mu 0.35\n# PubMed GCN+P-reg mu=0.15 standard split 10 seeds\npython main.py --num_splits 1 --num_seeds 10 --dataset pubmed --model preggcn --mu 0.15\n```\n\n## Tips\n\n1. `--model PREGGCN --mu 0` means to use the vanilla `GCN` model. (Similarly, to use vanilla `GAT` and `MLP`, please set `--mu 0`.)\n2. `--num_splits 1` means to use the standard split that is provided in the Plantoid dataset (CORA, CiteSeer and PubMed), while `--num_splits 5` to `main.py` means using the first 5 randomly generated splits provided in the `./splits/` folder (for all 7 datasets).\n3. In `main.py`, replace the `soft_cross_entropy` with `kl_div` or `squared_error` (provided in `phi.py`) to experiment with different $phi$ functions.\n4. In `main.py`, replace the `nll_loss` to `LabelSmoothingLoss` (provided in `loss.py`) to experiment with Label Smoothing. Add `confidence_penalty` or `laplacian_reg` (provided in `loss.py`) to the original loss item to experiment with Confidence Penalty or Laplacian Regularizer.\n5. In our experiments, for GCN and MLP, we use `hidden_size=64`, while for GAT, we use `hidden_size=16`.\n6. In our experiments, for CORA, CiteSeer and PubMed, we use `weight_decay=5e-4`, while for CS, Physics, Computers and Photo, we use `weight_decay=0`. This is determined by the vanilla model performance.\n7. By default, the training is stopped with validation accuracy no longer increases for 200 epochs (patience=200).\n8. The code of other state-of-the-art methods is either from their corresponding official repository or pytoch-geometric benchmarking code. Details are attached below.\n9. The code for graph-level experiments is in the `./graph_level/` folder.\n\n## Citation\n\n```BibTex\n@inproceedings{yang2021rethinking,\n  author    = {Han Yang and Kaili Ma and James Cheng},\n  title     = {Rethinking Graph Regularization for Graph Neural Networks},\n  booktitle = {Thirty-Fifth {AAAI} Conference on Artificial Intelligence, {AAAI}\n               2021, Virtual Event, February 2-9, 2021},\n  pages     = {4573--4581},\n  year      = {2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyang-han%2FP-reg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyang-han%2FP-reg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyang-han%2FP-reg/lists"}