{"id":29248890,"url":"https://github.com/hkuds/gpt-st","last_synced_at":"2025-10-25T05:46:10.577Z","repository":{"id":206430736,"uuid":"715584884","full_name":"HKUDS/GPT-ST","owner":"HKUDS","description":"[NeurIPS'2023] \"GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks\"","archived":false,"fork":false,"pushed_at":"2024-06-10T06:40:34.000Z","size":70767,"stargazers_count":53,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-06-10T08:37:32.579Z","etag":null,"topics":["generative-model","graph-neural-networks","pre-trained-model","spatio-temporal-prediction","urban-computing"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2311.04245","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HKUDS.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-07T12:47:26.000Z","updated_at":"2024-06-10T06:40:37.000Z","dependencies_parsed_at":null,"dependency_job_id":"256518a0-fc95-4bed-bc36-3512b4bbf236","html_url":"https://github.com/HKUDS/GPT-ST","commit_stats":null,"previous_names":["hkuds/gpt-st"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HKUDS/GPT-ST","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FGPT-ST","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FGPT-ST/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FGPT-ST/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FGPT-ST/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HKUDS","download_url":"https://codeload.github.com/HKUDS/GPT-ST/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FGPT-ST/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263421915,"owners_count":23464052,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["generative-model","graph-neural-networks","pre-trained-model","spatio-temporal-prediction","urban-computing"],"created_at":"2025-07-04T00:09:17.623Z","updated_at":"2025-10-25T05:46:10.568Z","avatar_url":"https://github.com/HKUDS.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks\n\nA pytorch implementation for the paper: [GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks](https://arxiv.org/abs/2311.04245v1)\u003cbr /\u003e  \n\nZhonghang Li, Lianghao Xia, Yong Xu, Chao Huang* (*Correspondence)\u003cbr /\u003e  \n\n**[Data Intelligence Lab](https://sites.google.com/view/chaoh/home)@[University of Hong Kong](https://www.hku.hk/)**, [South China University of Technology](https://www.scut.edu.cn/en/), PAZHOU LAB  \n\nThis repository hosts the code, data, and model weights of **GPT-ST**. Furthermore, it also includes the code for the baselines used in the paper.\n\n## Introduction\n\n\u003cp style=\"text-align: justify\"\u003e\nGPT-ST is a generative pre-training framework for improving the spatio-temporal prediction performance of downstream models. The framework is built upon two key designs: (i) We propose a spatio-temporal mask autoencoder as a pre-training model for learning spatio-temporal dependencies. The model incorporates customized parameter learners and hierarchical spatial pattern encoding networks, which specifically designed to capture spatio-temporal customized representations and intra- and inter-cluster region semantic relationships. (ii) We introduce an adaptive mask strategy as part of the pre-training mechanism. This strategy guides the mask autoencoder in learning robust spatio-temporal representations and facilitates the modeling of different relationships, ranging from intra-cluster to inter-cluster, in an easy-to-hard training manner.  \n\u003c/p\u003e\n\n![The detailed framework of the proposed GPT-ST.](https://github.com/LZH-YS1998/GPT-ST_img/blob/main/fig3.png)\n\n\n## Code structure\n* **conf**: This folder includes parameter settings for GPT-ST (`GPTST_pretrain`) as well as all other baseline models.\n* **data**: The documentation encompasses all the datasets utilized in our work, alongside prefabricated files and the corresponding file generation codes necessary for certain baselines.\n* **lib**: Including a series of initialization methods for data processing, as follows:\n\t* `Params_xxx.py`: To configure the parameters for GPT-ST and the baseline models。\n\t* `TrainInits.py`: Training initialization, including settings of optimizer, device, random seed, etc.\n\t* `add_window.py`: Time series slicing。\n\t* `dataloader.py` and `load_dataset.py`: Load, split, generate data, etc.\n\t* `logger.py`: For output printing。\n\t* `metrics.py`: Method for calculating evaluation indicators。\n\t* `normalization.py`: Normalizationmethod。\n\t* `predifineGraph.py`: Predefined graph generation method。\n* **model**: Includes the implementation of GPT-ST and all baseline models, along with the necessary code to support the framework's execution. The `args.py` script is utilized to generate the required prefabricated data and parameter configurations for different baselines. Additionally, the `SAVE` folder serves as the storage location for saving the pre-trained models.\n\n\n## Environment requirement\nThe code can be run in the following environments, other version of required packages may also work.\n* python==3.9.12\n* numpy==1.23.1\n* pytorch==1.9.0\n* cudatoolkit==11.1.1  \n\nOr you can install the required environment, which can be done by running the following commands:\n```\n# cteate new environmrnt\nconda create -n GPT-ST python=3.9.12\n\n# activate environmrnt\nconda activate GPT-ST\n\n# Torch with CUDA 11.1\npip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html\n\n# Install required libraries\npip install -r requirements.txt\n```\n\n\n## Run the codes \n* First, enter \"data\" folder to unzip all zip files, and then enter \"model\" folder:\n```\ncd model\n```\n* To test different models in various modes, you can execute the Run.py code. There are some examples:\n```\n# Evaluate the performance of STGCN enhanced by GPT-ST on the PEMS08 dataset\npython Run.py -dataset PEMS08 -mode eval -model STGCN\n\n# Evaluate the performance of ASTGCN enhanced by GPT-ST on the METR_LA dataset\npython Run.py -dataset METR_LA -mode eval -model ASTGCN\n\n# Evaluate the original performance of CCRNN on the NYC_TAXI dataset\npython Run.py -dataset NYC_TAXI -mode ori -model CCRNN\n\n# Pretrain from scratch on NYC_BIKE dataset, checkpoint will be saved in model/SAVE/NYC_BIKE/new_pretrain_model.pth\npython Run.py -dataset NYC_BIKE -mode pretrain\n```\n\n* Parameter setting instructions. The parameter settings consist of two parts: the pre-training model and the baseline model. To avoid any confusion arising from potential overlapping parameter names, we employ a hyphen (-) to specify the parameters of GPT-ST and use a double hyphen (--) to specify the parameters of the baseline model. Here is an example:\n```\n# Set first_layer_embedding_size and out_layer_dim to 32 in STFGNN\npython Run.py -model STFGNN -mode eval -dataset PEMS08 --first_layer_embedding_size 32 --out_layer_dim 32\n```\n\n\n\n## Citation\n```\n@inproceedings{\nli2023gptst,\nauthor={Zhonghang Li and Lianghao Xia and Yong Xu and Chao Huang},\ntitle = {GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks},\nbooktitle = {Advances in Neural Information Processing Systems},\npages = {70229-70246},\nyear={2023}\n}\n```\n\n## Acknowledgements\nWe developed our code framework drawing inspiration from [AGCRN](https://github.com/LeiBAI/AGCRN) and [STEP](https://github.com/zezhishao/STEP). Furthermore, the implementation of the baselines primarily relies on a combination of the code released by the original author and the code from [LibCity](https://github.com/LibCity/Bigscity-LibCity). We extend our heartfelt gratitude for their remarkable contribution.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhkuds%2Fgpt-st","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhkuds%2Fgpt-st","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhkuds%2Fgpt-st/lists"}