{"id":15601069,"url":"https://github.com/lucidrains/graph-transformer-pytorch","last_synced_at":"2025-04-07T06:07:02.725Z","repository":{"id":44846099,"uuid":"378269003","full_name":"lucidrains/graph-transformer-pytorch","owner":"lucidrains","description":"Implementation of Graph Transformer in Pytorch, for potential use in replicating Alphafold2","archived":false,"fork":false,"pushed_at":"2023-06-22T14:41:24.000Z","size":140,"stargazers_count":216,"open_issues_count":1,"forks_count":19,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-31T05:04:43.123Z","etag":null,"topics":["artificial-intelligence","deep-learning","graphs","transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lucidrains.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-18T21:33:47.000Z","updated_at":"2025-03-20T07:16:16.000Z","dependencies_parsed_at":"2024-10-23T01:35:30.018Z","dependency_job_id":null,"html_url":"https://github.com/lucidrains/graph-transformer-pytorch","commit_stats":{"total_commits":12,"total_committers":1,"mean_commits":12.0,"dds":0.0,"last_synced_commit":"8613d2141c84f1cbaa4e2802e3091638c706c9ca"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fgraph-transformer-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fgraph-transformer-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fgraph-transformer-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fgraph-transformer-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lucidrains","download_url":"https://codeload.github.com/lucidrains/graph-transformer-pytorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247601447,"owners_count":20964864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","deep-learning","graphs","transformer"],"created_at":"2024-10-03T02:13:43.043Z","updated_at":"2025-04-07T06:07:02.693Z","avatar_url":"https://github.com/lucidrains.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"./graph-transformer.png\" width=\"500px\"\u003e\u003c/img\u003e\n\n## Graph Transformer - Pytorch\n\nImplementation of \u003ca href=\"https://arxiv.org/abs/2009.03509\"\u003eGraph Transformer\u003c/a\u003e in Pytorch, for potential use in replicating \u003ca href=\"https://github.com/lucidrains/alphafold2\"\u003eAlphafold2\u003c/a\u003e. This was recently used by both \u003ca href=\"https://www.biorxiv.org/content/10.1101/2021.06.02.446809v1\"\u003eCosta et al\u003c/a\u003e and \u003ca href=\"https://www.biorxiv.org/content/10.1101/2021.06.14.448402v1\"\u003eBakers lab\u003c/a\u003e for transforming MSA and pair-wise embedding into 3d coordinates.\n\n## Install\n\n```bash\n$ pip install graph-transformer-pytorch\n```\n\n## Usage\n\n```python\nimport torch\nfrom graph_transformer_pytorch import GraphTransformer\n\nmodel = GraphTransformer(\n    dim = 256,\n    depth = 6,\n    edge_dim = 512,             # optional - if left out, edge dimensions is assumed to be the same as the node dimensions above\n    with_feedforwards = True,   # whether to add a feedforward after each attention layer, suggested by literature to be needed\n    gated_residual = True,      # to use the gated residual to prevent over-smoothing\n    rel_pos_emb = True          # set to True if the nodes are ordered, default to False\n)\n\nnodes = torch.randn(1, 128, 256)\nedges = torch.randn(1, 128, 128, 512)\nmask = torch.ones(1, 128).bool()\n\nnodes, edges = model(nodes, edges, mask = mask)\n\nnodes.shape # (1, 128, 256) - project to R^3 for coordinates\n```\n\nIf you want it to handle an adjacency matrix\n\n```python\nimport torch\nfrom graph_transformer_pytorch import GraphTransformer\n\nmodel = GraphTransformer(\n    dim = 256,\n    depth = 6,\n    edge_dim = 512,\n    with_feedforwards = True,\n    gated_residual = True,\n    rel_pos_emb = True,\n    accept_adjacency_matrix = True  # set this to True\n)\n\nnodes = torch.randn(2, 128, 256)\nadj_mat = torch.randint(0, 2, (2, 128, 128))\nmask = torch.ones(2, 128).bool()\n\nnodes, edges = model(nodes, adj_mat = adj_mat, mask = mask)\n\nnodes.shape # (1, 128, 256) - project to R^3 for coordinates\n```\n\n## Citations\n\n```bibtex\n@article {Costa2021.06.02.446809,\n    author  = {Costa, Allan and Ponnapati, Manvitha and Jacobson, Joseph M. and Chatterjee, Pranam},\n    title   = {Distillation of MSA Embeddings to Folded Protein Structures with Graph Transformers},\n    year    = {2021},\n    doi     = {10.1101/2021.06.02.446809},\n    publisher = {Cold Spring Harbor Laboratory},\n    URL     = {https://www.biorxiv.org/content/early/2021/06/02/2021.06.02.446809},\n    eprint  = {https://www.biorxiv.org/content/early/2021/06/02/2021.06.02.446809.full.pdf},\n    journal = {bioRxiv}\n}\n```\n\n```bibtex\n@article {Baek2021.06.14.448402,\n    author  = {Baek, Minkyung and DiMaio, Frank and Anishchenko, Ivan and Dauparas, Justas and Ovchinnikov, Sergey and Lee, Gyu Rie and Wang, Jue and Cong, Qian and Kinch, Lisa N. and Schaeffer, R. Dustin and Mill{\\'a}n, Claudia and Park, Hahnbeom and Adams, Carson and Glassman, Caleb R. and DeGiovanni, Andy and Pereira, Jose H. and Rodrigues, Andria V. and van Dijk, Alberdina A. and Ebrecht, Ana C. and Opperman, Diederik J. and Sagmeister, Theo and Buhlheller, Christoph and Pavkov-Keller, Tea and Rathinaswamy, Manoj K and Dalwadi, Udit and Yip, Calvin K and Burke, John E and Garcia, K. Christopher and Grishin, Nick V. and Adams, Paul D. and Read, Randy J. and Baker, David},\n    title   = {Accurate prediction of protein structures and interactions using a 3-track network},\n    year    = {2021},\n    doi     = {10.1101/2021.06.14.448402},\n    publisher = {Cold Spring Harbor Laboratory},\n    URL     = {https://www.biorxiv.org/content/early/2021/06/15/2021.06.14.448402},\n    eprint  = {https://www.biorxiv.org/content/early/2021/06/15/2021.06.14.448402.full.pdf},\n    journal = {bioRxiv}\n}\n```\n\n```bibtex\n@misc{shi2021masked,\n    title   = {Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification}, \n    author  = {Yunsheng Shi and Zhengjie Huang and Shikun Feng and Hui Zhong and Wenjin Wang and Yu Sun},\n    year    = {2021},\n    eprint  = {2009.03509},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.LG}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fgraph-transformer-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucidrains%2Fgraph-transformer-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fgraph-transformer-pytorch/lists"}