{"id":18752904,"url":"https://github.com/ashleve/graph_classification","last_synced_at":"2025-04-13T00:31:26.303Z","repository":{"id":48630423,"uuid":"318681163","full_name":"ashleve/graph_classification","owner":"ashleve","description":"Benchmarking GNNs with PyTorch Lightning: Open Graph Benchmarks and image classification from superpixels","archived":false,"fork":false,"pushed_at":"2022-07-18T11:28:40.000Z","size":1121,"stargazers_count":30,"open_issues_count":0,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-26T18:52:33.747Z","etag":null,"topics":["graph-classification","hydra","image-classification","ogbg","open-graph-benchmark","pytorch-lightning","superpixels"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ashleve.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-05T02:07:52.000Z","updated_at":"2025-01-22T16:08:46.000Z","dependencies_parsed_at":"2022-09-05T15:40:55.557Z","dependency_job_id":null,"html_url":"https://github.com/ashleve/graph_classification","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":"ashleve/lightning-hydra-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashleve%2Fgraph_classification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashleve%2Fgraph_classification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashleve%2Fgraph_classification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashleve%2Fgraph_classification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ashleve","download_url":"https://codeload.github.com/ashleve/graph_classification/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248650590,"owners_count":21139670,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph-classification","hydra","image-classification","ogbg","open-graph-benchmark","pytorch-lightning","superpixels"],"created_at":"2024-11-07T17:23:05.277Z","updated_at":"2025-04-13T00:31:25.528Z","avatar_url":"https://github.com/ashleve.png","language":"Jupyter Notebook","readme":"\u003cdiv align=\"center\"\u003e\n\n# Graph Classification Benchmarks\n\n\u003ca href=\"https://pytorch.org/get-started/locally/\"\u003e\u003cimg alt=\"PyTorch\" src=\"https://img.shields.io/badge/PyTorch-ee4c2c?logo=pytorch\u0026logoColor=white\"\u003e\u003c/a\u003e\n\u003ca href=\"https://pytorchlightning.ai/\"\u003e\u003cimg alt=\"Lightning\" src=\"https://img.shields.io/badge/-Lightning-792ee5?logo=pytorchlightning\u0026logoColor=white\"\u003e\u003c/a\u003e\n\u003ca href=\"https://hydra.cc/\"\u003e\u003cimg alt=\"Config: Hydra\" src=\"https://img.shields.io/badge/Config-Hydra-89b8cd\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/ashleve/lightning-hydra-template\"\u003e\u003cimg alt=\"Template\" src=\"https://img.shields.io/badge/-Lightning--Hydra--Template-017F2F?style=flat\u0026logo=github\u0026labelColor=gray\"\u003e\u003c/a\u003e\u003cbr\u003e\n\n\u003c/div\u003e\n\n## Description\n\nThis repository is supposed to be a place for curated, high quality benchmarks of Graph Neural Networks, implemented with PyTorch Lightning and Hydra.\u003cbr\u003e\nOnly datasets big enough to provide good measures are taken into consideration.\u003cbr\u003e\nBuilt with [lightning-hydra-template](https://github.com/ashleve/lightning-hydra-template).\n\n### Datasets\n\n- [Open Graph Benchmarks](https://ogb.stanford.edu/docs/graphprop/) (graph property prediction)\n- Image classification from superpixels (MNIST, FashionMNIST, CIFAR10)\n\n## How to run\n\nInstall dependencies\n\n```yaml\n# clone project\ngit clone https://github.com/ashleve/graph_classification\ncd graph_classification\n\n# [OPTIONAL] create conda environment\nconda create -n myenv python=3.8\nconda activate myenv\n\n# install pytorch and pytorch geometric according to instructions\n# https://pytorch.org/get-started/\n# https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html\n\n# install requirements\npip install -r requirements.txt\n```\n\nTrain model with default configuration\n\n```yaml\n# train on CPU\npython run.py trainer.gpus=0\n\n# train on GPU\npython run.py trainer.gpus=1\n```\n\nTrain model with chosen experiment configuration from [configs/experiment/](configs/experiment/)\n\n```yaml\npython run.py experiment=GAT/gat_ogbg_molpcba\npython run.py experiment=GraphSAGE/graphsage_mnist_sp75\npython run.py experiment=GraphSAGE/graphsage_cifar10_sp100\n```\n\nYou can override any parameter from command line like this\n\n```yaml\npython run.py trainer.max_epochs=20 datamodule.batch_size=64\n```\n\n## Methodology\n\nFor each experiment, we run a series of 10 random hparams runs, and 5 optimization runs, using Optuna bayesian sampler. The hyperparameter search configs are available under [configs/hparams_search](configs/hparams_search).\n\nAfter finding best hyperparameters, each experiment was repeated 5 times with different random seeds. The only exception are the `ogbg-molhiv` experiments, which were repeated 10 times each (because of high varience of results).\n\nThe results were averaged and reported in the table below.\n\n## Results\n\n| Architecture | MNIST-sp75    | FashionMNIST-sp75 | CIFAR10-sp100 | ogbg-molhiv   | ogbg-molcpba  |\n| ------------ | ------------- | ----------------- | ------------- | ------------- | ------------- |\n| GCN          | 0.955 ± 0.014 | 0.835 ± 0.016     | 0.518 ± 0.007 | 0.755 ± 0.019 | 0.231 ± 0.003 |\n| GIN          | 0.966 ± 0.008 | 0.861 ± 0.012     | 0.512 ± 0.020 | 0.757 ± 0.025 | 0.240 ± 0.001 |\n| GAT          | 0.976 ± 0.008 | 0.889 ± 0.003     | 0.617 ± 0.005 | 0.751 ± 0.026 | 0.234 ± 0.003 |\n| GraphSAGE    | 0.981 ± 0.005 | 0.897 ± 0.012     | 0.629 ± 0.012 | 0.761 ± 0.025 | 0.256 ± 0.004 |\n\nThe `+-` denotes standard deviation across all seeds.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashleve%2Fgraph_classification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fashleve%2Fgraph_classification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashleve%2Fgraph_classification/lists"}