{"id":15633267,"url":"https://github.com/rusty1s/pyg_autoscale","last_synced_at":"2025-08-22T05:32:43.128Z","repository":{"id":48068017,"uuid":"335200768","full_name":"rusty1s/pyg_autoscale","owner":"rusty1s","description":"Implementation of \"GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings\" in PyTorch","archived":false,"fork":false,"pushed_at":"2022-08-30T13:41:10.000Z","size":216,"stargazers_count":164,"open_issues_count":18,"forks_count":27,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-08T23:22:43.729Z","etag":null,"topics":["graph-neural-networks","pytorch","pytorch-geometric","scalability"],"latest_commit_sha":null,"homepage":"http://arxiv.org/abs/2106.05609","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rusty1s.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-02T07:12:46.000Z","updated_at":"2025-02-12T14:28:08.000Z","dependencies_parsed_at":"2023-01-16T23:15:10.650Z","dependency_job_id":null,"html_url":"https://github.com/rusty1s/pyg_autoscale","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rusty1s/pyg_autoscale","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusty1s%2Fpyg_autoscale","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusty1s%2Fpyg_autoscale/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusty1s%2Fpyg_autoscale/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusty1s%2Fpyg_autoscale/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rusty1s","download_url":"https://codeload.github.com/rusty1s/pyg_autoscale/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusty1s%2Fpyg_autoscale/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271590501,"owners_count":24786239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-22T02:00:08.480Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph-neural-networks","pytorch","pytorch-geometric","scalability"],"created_at":"2024-10-03T10:47:58.555Z","updated_at":"2025-08-22T05:32:42.680Z","avatar_url":"https://github.com/rusty1s.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003ePyGAS: Auto-Scaling GNNs in PyG\u003c/h1\u003e\n\n\u003cimg width=\"100%\" src=\"https://raw.githubusercontent.com/rusty1s/pyg_autoscale/master/figures/overview.png?token=ABU7ZAXZ7WT3RIOSYHIDIVDAEI3SY\" /\u003e\n\n--------------------------------------------------------------------------------\n\n*PyGAS* is the practical realization of our *\u003cins\u003eG\u003c/ins\u003eNN\u003cins\u003eA\u003c/ins\u003euto\u003cins\u003eS\u003c/ins\u003ecale* (GAS) framework, which scales arbitrary message-passing GNNs to large graphs, as described in our paper:\n\nMatthias Fey, Jan E. Lenssen, Frank Weichert, Jure Leskovec: **[GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings](http://arxiv.org/abs/2106.05609)** *(ICML 2021)*\n\nGAS prunes entire sub-trees of the computation graph by utilizing historical embeddings from prior training iterations, leading to constant GPU memory consumption in respect to input mini-batch size, and maximally expressivity.\n\n*PyGAS* is implemented in [PyTorch](https://pytorch.org/) and utilizes the [PyTorch Geometric](https://github.com/rusty1s/pytorch_geometric) (PyG) library.\nIt provides an easy-to-use interface to convert a common or custom GNN from PyG into its scalable variant:\n\n```python\nfrom torch_geometric.nn import SAGEConv\nfrom torch_geometric_autoscale import ScalableGNN\nfrom torch_geometric_autoscale import metis, permute, SubgraphLoader\n\nclass GNN(ScalableGNN):\n    def __init__(self, num_nodes, in_channels, hidden_channels,\n                 out_channels, num_layers):\n        # * pool_size determines the number of pinned CPU buffers\n        # * buffer_size determines the size of pinned CPU buffers,\n        #   i.e. the maximum number of out-of-mini-batch nodes\n\n        super().__init__(num_nodes, hidden_channels, num_layers,\n                         pool_size=2, buffer_size=5000)\n\n        self.convs = ModuleList()\n        self.convs.append(SAGEConv(in_channels, hidden_channels))\n        for _ in range(num_layers - 2):\n            self.convs.append(SAGEConv(hidden_channels, hidden_channels))\n        self.convs.append(SAGEConv(hidden_channels, out_channels))\n\n    def forward(self, x, adj_t, *args):\n        for conv, history in zip(self.convs[:-1], self.histories):\n            x = conv(x, adj_t).relu_()\n            x = self.push_and_pull(history, x, *args)\n        return self.convs[-1](x, adj_t)\n\nperm, ptr = metis(data.adj_t, num_parts=40, log=True)\ndata = permute(data, perm, log=True)\nloader = SubgraphLoader(data, ptr, batch_size=10, shuffle=True)\n\nmodel = GNN(...)\nfor batch, *args in loader:\n    out = model(batch.x, batch.adj_t, *args)\n```\n\nA detailed description of `ScalableGNN` can be found [in its implementation](https://github.com/rusty1s/pyg_autoscale/blob/master/torch_geometric_autoscale/models/base.py#L13).\n\n## Requirements\n\n* Install [**PyTorch \u003e= 1.7.0**](https://pytorch.org/get-started/locally/)\n* Install [**PyTorch Geometric \u003e= 1.7.0**](https://github.com/rusty1s/pytorch_geometric#installation):\n\n```\npip install torch-scatter -f https://pytorch-geometric.com/whl/torch-${TORCH}+${CUDA}.html\npip install torch-sparse -f https://pytorch-geometric.com/whl/torch-${TORCH}+${CUDA}.html\npip install torch-geometric\n```\n\nwhere `${TORCH}` should be replaced by either `1.7.0` or `1.8.0`, and `${CUDA}` should be replaced by either `cpu`, `cu92`, `cu101`, `cu102`, `cu110` or `cu111`, depending on your PyTorch installation.\n\n## Installation\n\n```\npip install git+https://github.com/rusty1s/pyg_autoscale.git\n```\n\nor\n\n```\npython setup.py install\n```\n\n## Project Structure\n\n* **`torch_geometric_autoscale/`** contains the source code of *PyGAS*\n* **`examples/`** contains examples to demonstrate how to apply GAS in practice\n* **`small_benchmark/`** includes experiments to evaluate GAS performance on *small-scale* graphs\n* **`large_benchmark/`** includes experiments to evaluate GAS performance on *large-scale* graphs\n\nWe use [**Hydra**](https://hydra.cc/) to manage hyperparameter configurations.\n\n## Cite\n\nPlease cite [our paper](http://arxiv.org/abs/2106.05609) if you use this code in your own work:\n\n```\n@inproceedings{Fey/etal/2021,\n  title={{GNNAutoScale}: Scalable and Expressive Graph Neural Networks via Historical Embeddings},\n  author={Fey, M. and Lenssen, J. E. and Weichert, F. and Leskovec, J.},\n  booktitle={International Conference on Machine Learning (ICML)},\n  year={2021},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frusty1s%2Fpyg_autoscale","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frusty1s%2Fpyg_autoscale","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frusty1s%2Fpyg_autoscale/lists"}