{"id":30181895,"url":"https://github.com/snap-stanford/relgt","last_synced_at":"2025-08-12T09:21:30.339Z","repository":{"id":302768584,"uuid":"984295752","full_name":"snap-stanford/relgt","owner":"snap-stanford","description":"Relational Graph Transformer","archived":false,"fork":false,"pushed_at":"2025-07-04T05:16:24.000Z","size":244,"stargazers_count":27,"open_issues_count":0,"forks_count":1,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-07-04T06:25:13.468Z","etag":null,"topics":["graph-neural-networks","graph-transformer","relational-databases","relational-deep-learning","transformers"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2505.10960","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snap-stanford.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-15T17:43:42.000Z","updated_at":"2025-07-04T05:21:32.000Z","dependencies_parsed_at":"2025-07-04T06:35:27.401Z","dependency_job_id":null,"html_url":"https://github.com/snap-stanford/relgt","commit_stats":null,"previous_names":["snap-stanford/relgt"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/snap-stanford/relgt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-stanford%2Frelgt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-stanford%2Frelgt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-stanford%2Frelgt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-stanford%2Frelgt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snap-stanford","download_url":"https://codeload.github.com/snap-stanford/relgt/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snap-stanford%2Frelgt/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270034871,"owners_count":24515780,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-12T02:00:09.011Z","response_time":80,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph-neural-networks","graph-transformer","relational-databases","relational-deep-learning","transformers"],"created_at":"2025-08-12T09:21:28.392Z","updated_at":"2025-08-12T09:21:30.324Z","avatar_url":"https://github.com/snap-stanford.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Relational Graph Transformer (RelGT)\n\n[![arXiv](https://img.shields.io/badge/arXiv-2505.10960-b31b1b.svg)](https://arxiv.org/abs/2505.10960)\n\u003e[Vijay Prakash Dwivedi](http://vijaydwivedi.com.np), [Sri Jaladi](https://www.linkedin.com/in/srijaladi), [Yangyi Shen](https://www.linkedin.com/in/yangyi-shen-232514264/), [Federico Lopez](https://fedelopez77.github.io), [Charilaos I. Kanatsoulis](https://sites.google.com/site/harikanats/), [Rishi Puri](https://www.linkedin.com/in/rishi-puri-1726b1147/), [Matthias Fey](https://rusty1s.github.io/#/), [Jure Leskovec](https://cs.stanford.edu/people/jure/)\n\nThis repository implements the [Relational Graph Transformer (RelGT)](https://arxiv.org/abs/2505.10960), the first graph transformer architecture specifically designed for multi-table relational data represented as heterogeneous temporal graphs.\n\n# Overview\n\n![Overview of the architecture.](arch.png)\n\nRelational Deep Learning (RDL) is a promising approach for building state-of-the-art predictive models on multi-table relational data. While Graph Neural Networks (GNNs) have been commonly used for this task, they struggle with capturing complex structural patterns and long-range dependencies. Our Relational Graph Transformer (RelGT) addresses the unique challenges of applying transformers to relational entity graphs through:\n\n1. A novel multi-element tokenization strategy that decomposes each node into five components (features, type, hop distance, time, and local structure)\n2. An architecture that combines local attention over sampled subgraphs with global attention to learnable centroids\n3. Efficient encoding of heterogeneity, temporality, and topology without expensive precomputation\n\n# Environment Setup\n\nWe provide environment setup instructions using Micromamba (a lightweight, faster alternative to Conda):\n\n```\n# Install Micromamba\nmkdir ~/micromamba\nwget -qO- https://micromamba.snakepit.net/api/micromamba/linux-64/latest | tar -xvj -C ~/micromamba\necho 'export PATH=~/micromamba/bin:$PATH' \u003e\u003e ~/.bashrc\nsource ~/.bashrc\n\n# Create and activate environment\nmicromamba create -n gt python=3.12\nmicromamba activate gt\n\n# Install PyTorch with CUDA\nmicromamba install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia\n\n# Install uv for faster pip installations\ncurl -LsSf https://astral.sh/uv/install.sh | sh \n\n# Install PyTorch Geometric and its dependencies\nuv pip install pyg-lib -f https://data.pyg.org/whl/torch-2.5.0+cu121.html\nuv pip install torch-scatter torch-sparse torch-cluster torch-spline-conv -f https://data.pyg.org/whl/torch-2.5.0+cu121.html\nmicromamba install pyg -c pyg\n\n# Install other dependencies\nmicromamba install -c conda-forge wandb absl-py tensorboard einops matplotlib progressbar pandas numba networkx scikit-network ipykernel tqdm\nuv pip install kmeans-pytorch torchviz fastcluster opentsne ogb kmedoids relbench pytorch_frame[full] sentence-transformers h5py pynvml\n```\n\n# Experiments\n\nTo reproduce our experimental results on the RelBench benchmark:\n\n```\n# Choose the script to run specific experiment, for example:\nbash expts/run-\u003cexperiment\u003e.sh\n```\n\n**Notes**\n\n- While the scripts are configured to run experiments optimally with 8 GPUs on NVIDIA A100 GPU (80GB) server, they can be adjusted according to specific user environments.\n- Optionally, the main file supports Distributed Training using PyTorch's DDP, but may not be fully optimized.\n\n# Citation\n\n```bibtex\n@article{dwivedi2025relational,\n  title={Relational Graph Transformer},\n  author={Dwivedi, Vijay Prakash and Jaladi, Sri and Shen, Yangyi and L{\\'o}pez, Federico and Kanatsoulis, Charilaos I and Puri, Rishi and Fey, Matthias and Leskovec, Jure},\n  journal={arXiv preprint arXiv:2505.10960},\n  year={2025}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnap-stanford%2Frelgt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnap-stanford%2Frelgt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnap-stanford%2Frelgt/lists"}