{"id":29248713,"url":"https://github.com/hkuds/urbangpt","last_synced_at":"2025-07-04T00:08:38.838Z","repository":{"id":223601519,"uuid":"760978768","full_name":"HKUDS/UrbanGPT","owner":"HKUDS","description":"[KDD'2024] \"UrbanGPT: Spatio-Temporal Large Language Models\"","archived":false,"fork":false,"pushed_at":"2025-04-18T16:48:30.000Z","size":16209,"stargazers_count":345,"open_issues_count":11,"forks_count":45,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-04-19T05:39:37.365Z","etag":null,"topics":["fundation-models","instruction-tuning","large-language-models","pre-trained-model","smart-cities","spatio-temporal-prediction","urban-computing","urban-data-science"],"latest_commit_sha":null,"homepage":"https://urban-gpt.github.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HKUDS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-21T02:24:49.000Z","updated_at":"2025-04-18T16:48:34.000Z","dependencies_parsed_at":"2024-05-19T02:25:44.598Z","dependency_job_id":"e2d7907c-8b5a-4de8-9d40-7d9e553d0499","html_url":"https://github.com/HKUDS/UrbanGPT","commit_stats":null,"previous_names":["hkuds/urbangpt"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HKUDS/UrbanGPT","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FUrbanGPT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FUrbanGPT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FUrbanGPT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FUrbanGPT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HKUDS","download_url":"https://codeload.github.com/HKUDS/UrbanGPT/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HKUDS%2FUrbanGPT/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263421919,"owners_count":23464051,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fundation-models","instruction-tuning","large-language-models","pre-trained-model","smart-cities","spatio-temporal-prediction","urban-computing","urban-data-science"],"created_at":"2025-07-04T00:08:37.397Z","updated_at":"2025-07-04T00:08:38.624Z","avatar_url":"https://github.com/HKUDS.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UrbanGPT: Spatio-Temporal Large Language Models\n\n\u003cimg src='urbanGPTarticle cover.jpeg' /\u003e\n\nA pytorch implementation for the paper: [UrbanGPT: Spatio-Temporal Large Language Models]\u003cbr /\u003e  \n\n[Zhonghang Li](https://scholar.google.com/citations?user=__9uvQkAAAAJ), [Lianghao Xia](https://akaxlh.github.io/), [Jiabin Tang](https://tjb-tech.github.io/), [Yong Xu](https://scholar.google.com/citations?user=1hx5iwEAAAAJ), [Lei Shi](https://harryshil.github.io/), [Long Xia](https://scholar.google.com/citations?user=NRwerBAAAAAJ), [Dawei Yin](https://www.yindawei.com/), [Chao Huang](https://sites.google.com/view/chaoh)* (*Correspondence)\u003cbr /\u003e  \n\n**[Data Intelligence Lab](https://sites.google.com/view/chaoh/home)@[University of Hong Kong](https://www.hku.hk/)**, [South China University of Technology](https://www.scut.edu.cn/en/), Baidu Inc  \n\n-----\n\n\u003ca href='https://urban-gpt.github.io/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-Green'\u003e\u003c/a\u003e\n\u003ca href='https://github.com/HKUDS/UrbanGPT'\u003e\u003cimg src='https://img.shields.io/badge/Demo-Page-purple'\u003e\u003c/a\u003e \n\u003ca href='https://arxiv.org/abs/2403.00813'\u003e\u003cimg src='https://img.shields.io/badge/Paper-PDF-orange'\u003e\u003c/a\u003e \n[![YouTube](https://badges.aleen42.com/src/youtube.svg)](https://www.youtube.com/watch?v=4BIbQt-EIAM)\n • 🌐 \u003ca href=\"https://zhuanlan.zhihu.com/p/684785925\" target=\"_blank\"\u003e中文博客\u003c/a\u003e\n\nThis repository hosts the code, data, and model weights of **UrbanGPT**.\n\n-----\n## 🎉 News \n\n- [x] 🚀🔥 [2024.05] 🎯🎯📢📢 Exciting News! We are thrilled to announce that our 🌟UrbanGPT🌟 has been accepted by KDD'2024! 🎉🎉🎉 Thanks to all the team members 🤗\n\n🎯🎯📢📢 We upload the **models** and **data** used in our UrbanGPT on 🤗 **Huggingface**. We highly recommend referring to the table below for further details: \n\n| 🤗 Huggingface Address                                        | 🎯 Description                                                |\n| ------------------------------------------------------------ | ------------------------------------------------------------ |\n| [https://huggingface.co/bjdwh/UrbanGPT](https://huggingface.co/bjdwh/UrbanGPT/tree/main) | It's the checkpoint of our UrbanGPT based on Vicuna-7B-v1.5-16k tuned on instruction data [train-data](https://huggingface.co/datasets/bjdwh/ST_data_urbangpt/tree/main/train_data) |\n| [https://huggingface.co/datasets/bjdwh/ST_data_urbangpt](https://huggingface.co/datasets/bjdwh/ST_data_urbangpt) | We released a portion of the instruction dataset for evaluation. |\n| [https://huggingface.co/datasets/bjdwh/UrbanGPT_ori_stdata](https://huggingface.co/datasets/bjdwh/UrbanGPT_ori_stdata) | We released the original dataset used in UrbanGPT. |\n\n- [x] [2023.02.23] 🚀🚀 Release the code of UrbanGPT.\n- [x] [2023.02.29] Add video.\n- [x] [2023.03.05] Release the full paper.\n- [x] [2023.03.11] Upload the new checkpoint of our UrbanGPT.\n- [x] [2023.06.07] Release instruction generation codes and the original dataset used in UrbanGPT.\n\n## 👉 TODO \n- [ ] Release baselines codes.\n- [ ] ...\n\n\n-----------\n\n## Introduction\n\n\u003cp style=\"text-align: justify\"\u003e\nIn this work, we present a spatio-temporal large language model that can exhibit exceptional generalization capabilities across a wide range of downstream urban tasks. \nTo achieve this objective, we present the UrbanGPT, which seamlessly integrates a spatio-temporal dependency encoder with the instruction-tuning paradigm. \nThis integration enables large language models (LLMs) to comprehend the complex inter-dependencies across time and space, facilitating more comprehensive and accurate predictions under data scarcity. \nExtensive experimental findings highlight the potential of building LLMs for spatio-temporal learning, particularly in zero-shot scenarios.\n\u003c/p\u003e\n\n![The detailed framework of the proposed UrbanGPT.](https://github.com/urban-gpt/urban-gpt.github.io/blob/main/images/urbangpt_framework.png)\n\n\n### Demo Video\nhttps://github.com/HKUDS/UrbanGPT/assets/90381931/9cd094b4-8fa3-486f-890d-631a08b19b4a\n\n-----------\n\u003cspan id='Usage'/\u003e\n\n## Getting Started\n\n\u003cspan id='all_catelogue'/\u003e\n\n### Table of Contents:\n* \u003ca href='#Code Structure'\u003e1. Code Structure\u003c/a\u003e\n* \u003ca href='#Environment'\u003e2. Environment \u003c/a\u003e\n* \u003ca href='#Training UrbanGPT'\u003e3. Training UrbanGPT \u003c/a\u003e\n  * \u003ca href='#Prepare Pre-trained Checkpoint'\u003e3.1. Prepare Pre-trained Checkpoint\u003c/a\u003e\n  * \u003ca href='#Instruction Tuning'\u003e3.2. Instruction Tuning\u003c/a\u003e\n* \u003ca href='#Evaluating UrbanGPT'\u003e4. Evaluating UrbanGPT\u003c/a\u003e\n  * \u003ca href='#Preparing Checkpoints and Data'\u003e4.1. Preparing Checkpoints and Data\u003c/a\u003e\n  * \u003ca href='#Running Evaluation'\u003e4.2. Running Evaluation\u003c/a\u003e\n  * \u003ca href='#Evaluation Metric Calculation'\u003e4.3. Evaluation Metric Calculation\u003c/a\u003e\n* \u003ca href='#Instructions Generation'\u003e5. Instructions Generation \u003c/a\u003e\n****\n\n\n\u003cspan id='Code Structure'/\u003e\n\n### 1. Code Structure \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\n\n```\n.\n|   README.md\n|   urbangpt_eval.sh\n|   urbangpt_train.sh\n|   \n+---checkpoints\n|   \\---st_encoder\n|           pretrain_stencoder.pth\n|           \n+---playground\n|   |   inspect_conv.py\n|   |   \n|   +---test_embedding\n|   |       README.md\n|   |       test_classification.py\n|   |       test_semantic_search.py\n|   |       test_sentence_similarity.py\n|   |       \n|   \\---test_openai_api\n|           anthropic_api.py\n|           openai_api.py\n|           \n+---tests\n|       test_openai_curl.sh\n|       test_openai_langchain.py\n|       test_openai_sdk.py\n|       \n\\---urbangpt\n    |   constants.py\n    |   conversation.py\n    |   utils.py\n    |   __init__.py\n    |   \n    +---eval\n    |   |   run_urbangpt.py                     # evaluation\n    |   |   run_vicuna.py\n    |   |   \n    |   \\---script\n    |           run_model_qa.yaml\n    |           \n    +---model\n    |   |   apply_delta.py\n    |   |   apply_lora.py\n    |   |   builder.py\n    |   |   compression.py\n    |   |   convert_fp16.py\n    |   |   make_delta.py\n    |   |   model_adapter.py\n    |   |   model_registry.py\n    |   |   monkey_patch_non_inplace.py\n    |   |   STLlama.py                          # model\n    |   |   utils.py\n    |   |   __init__.py\n    |   |   \n    |   \\---st_layers\n    |           args.py\n    |           ST_Encoder.conf\n    |           ST_Encoder.py                   # ST-Encoder\n    |           __init__.py\n    |           \n    +---protocol\n    |       openai_api_protocol.py\n    |       \n    +---serve\n    |   |   api_provider.py\n    |   |   bard_worker.py\n    |   |   cacheflow_worker.py\n    |   |   cli.py\n    |   |   controller.py\n    |   |   controller_graph.py\n    |   |   gradio_block_arena_anony.py\n    |   |   gradio_block_arena_named.py\n    |   |   gradio_css.py\n    |   |   gradio_patch.py\n    |   |   gradio_web_server.py\n    |   |   gradio_web_server_graph.py\n    |   |   gradio_web_server_multi.py\n    |   |   huggingface_api.py\n    |   |   inference.py\n    |   |   model_worker.py\n    |   |   model_worker_graph.py\n    |   |   openai_api_server.py\n    |   |   register_worker.py\n    |   |   test_message.py\n    |   |   test_throughput.py\n    |   |   __init__.py\n    |   |   \n    |   +---examples\n    |   |       extreme_ironing.jpg\n    |   |       waterview.jpg\n    |   |       \n    |   +---gateway\n    |   |       nginx.conf\n    |   |       README.md\n    |   |       \n    |   \\---monitor\n    |           basic_stats.py\n    |           clean_battle_data.py\n    |           elo_analysis.py\n    |           hf_space_leaderboard_app.py\n    |           monitor.py\n    |           \n    \\---train\n            llama2_flash_attn_monkey_patch.py\n            llama_flash_attn_monkey_patch.py\n            stchat_trainer.py\n            train_lora.py\n            train_mem.py\n            train_st.py                         # train\n            \n```\n\n\n\u003cspan id='Environment'/\u003e\n\n### 2.Environment \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\nPlease first clone the repo and install the required environment, which can be done by running the following commands:\n```shell\nconda create -n urbangpt python=3.9.13\n\nconda activate urbangpt\n\n# Torch with CUDA 11.7\npip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2\n\n# To support vicuna base model\npip3 install \"fschat[model_worker,webui]\"\n\n# To install pyg and pyg-relevant packages\npip install torch_geometric\npip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.0.1+cu117.html\n\n# Clone our UrabnGPT or download it\ngit clone https://github.com/HKUDS/UrbanGPT.git\ncd UrbanGPT\n\n# Install required libraries\n# (The recommendation is to install separately using the following method)\npip install deepspeed\npip install ray\npip install einops\npip install wandb\n# （There is a version compatibility issue between \"flash-attn\" and \"transformers\". Please refer to the flash-attn [GitHub URL](https://github.com/Dao-AILab/flash-attention) for more information.）\npip install flash-attn==2.3.5  # or download from (https://github.com/Dao-AILab/flash-attention/releases, e.g. flash_attn-2.3.5+cu117torch2.0cxx11abiFALSE-cp39-cp39-linux_x86_64.whl)\npip install transformers==4.34.0\n\n# （or you can install according to the requirements file.）\npip install -r requirements.txt\n```\n\n\u003cspan id='Training UrbanGPT'/\u003e\n\n### 3. Training UrbanGPT \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\n\n\u003cspan id='Prepare Pre-trained Checkpoint'/\u003e\n\n#### 3.1. Preparing Pre-trained Checkpoint  \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\nUrabnGPT is trained based on following excellent existing models.\nPlease follow the instructions to prepare the checkpoints.\n\n- `Vicuna`:\n  Prepare our base model Vicuna, which is an instruction-tuned chatbot and base model in our implementation. Please download its weights [here](https://github.com/lm-sys/FastChat#model-weights). We generally utilize v1.5 and v1.5-16k model with 7B parameters. You should update the 'config.json' of vicuna, for example, the 'config.json' in v1.5-16k can be found in [config.json](https://huggingface.co/datasets/bjdwh/checkpoints/blob/main/train_config/config.json)\n\n- `Spatio-temporal Encoder`:\n  We employ a simple TCNs-based spatio-temporal encoder to encode the spatio-temporal dependencies. The weights of [st_encoder](./checkpoints/st_encoder/pretrain_stencoder.pth) are pre-trained through a typical multi-step spatio-temporal prediction task.\n\n- `Spatio-temporal Train Data`:\n  We utilize pre-training data consisting of New York City's taxi, bike, and crime data, including spatio-temporal statistics, recorded timestamps, and information about regional points of interest (POIs). These data are organized in [train_data](https://huggingface.co/datasets/bjdwh/ST_data_urbangpt/tree/main/train_data). Please download it and put it at ./UrbanGPT/ST_data_urbangpt/train_data\n\n\u003cspan id='Instruction Tuning'/\u003e\n\n#### 3.2. Instruction Tuning \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\n\n* **Start tuning:** After the aforementioned steps, you could start the instruction tuning by filling blanks at [urbangpt_train.sh](urbangpt_train.sh). There is an example as below: \n\n```shell\n# to fill in the following path to run our UrbanGPT!\nmodel_path=./checkpoints/vicuna-7b-v1.5-16k\ninstruct_ds=./ST_data_urbangpt/train_data/multi_NYC.json\nst_data_path=./ST_data_urbangpt/train_data/multi_NYC_pkl.pkl\npretra_ste=ST_Encoder\noutput_model=./checkpoints/UrbanGPT\n\nwandb offline\npython -m torch.distributed.run --nnodes=1 --nproc_per_node=8 --master_port=20001 \\\n    urbangpt/train/train_mem.py \\\n    --model_name_or_path ${model_path} \\\n    --version v1 \\\n    --data_path ${instruct_ds} \\\n    --st_content ./TAXI.json \\\n    --st_data_path ${st_data_path} \\\n    --st_tower ${pretra_ste} \\\n    --tune_st_mlp_adapter True \\\n    --st_select_layer -2 \\\n    --use_st_start_end \\\n    --bf16 True \\\n    --output_dir ${output_model} \\\n    --num_train_epochs 3 \\\n    --per_device_train_batch_size 4 \\\n    --per_device_eval_batch_size 4 \\\n    --gradient_accumulation_steps 1 \\\n    --evaluation_strategy \"no\" \\\n    --save_strategy \"steps\" \\\n    --save_steps 2400 \\\n    --save_total_limit 1 \\\n    --learning_rate 2e-3 \\\n    --weight_decay 0. \\\n    --warmup_ratio 0.03 \\\n    --lr_scheduler_type \"cosine\" \\\n    --logging_steps 1 \\\n    --tf32 True \\\n    --model_max_length 2048 \\\n    --gradient_checkpointing True \\\n    --lazy_preprocess True \\\n    --report_to wandb\n\n```\n\n\u003cspan id='Evaluating UrbanGPT'/\u003e\n\n## 4. Evaluating UrbanGPT \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\n\n\u003cspan id='Preparing Checkpoints and Data'/\u003e\n\n#### 4.1. Preparing Checkpoints and Data \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\n\n* **Checkpoints:** You could try to evaluate UrbanGPT by using your own model or our released checkpoints.\n* **Data:** We split test sets for NYC-taxi datasets and make the instruction data for evaluation. Please refer to the [evaluating](https://huggingface.co/datasets/bjdwh/ST_data_urbangpt).\n\n\u003cspan id='Running Evaluation'/\u003e\n\n#### 4.2. Running Evaluation \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\n\nYou could start the second stage tuning by filling blanks at [urbangpt_eval.sh](urbangpt_eval.sh). There is an example as below: \n```\n# to fill in the following path to evaluation!\noutput_model=./checkpoints/tw2t_multi_reg-cla-gird\ndatapath=./ST_data_urbangpt/NYC_taxi_cross-region/NYC_taxi.json\nst_data_path=./ST_data_urbangpt/NYC_taxi_cross-region/NYC_taxi_pkl.pkl\nres_path=./result_test/cross-region/NYC_taxi\nstart_id=0\nend_id=51920\nnum_gpus=8\n\npython ./urbangpt/eval/run_urbangpt.py --model-name ${output_model}  --prompting_file ${datapath} --st_data_path ${st_data_path} --output_res_path ${res_path} --start_id ${start_id} --end_id ${end_id} --num_gpus ${num_gpus}\n```\n\n#### 4.3. Evaluation Metric Calculation \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\n\n\u003cspan id='Evaluation Metric Calculation'/\u003e\n\nYou can use [result_test.py](./metric_calculation/result_test.py) to calculate the performance metrics of the predicted results. \n\n---------\n\n\n## 5. Instructions Generation \u003ca href='#all_catelogue'\u003e[Back to Top]\u003c/a\u003e\n\n\u003cspan id='Instructions Generation'/\u003e\n\nYou can use the code in [instruction_generate.py](./instruction_generate/instruction_generate.py) to generate the specific instructions you need. For example: \n\n```\n-dataset_name: Choose the dataset. # NYC_multi(for training)    NYC_taxi NYC_bike NYC_crime1 NYC_crime2 CHI_taxi (for testing)\n# Only one of the following options can be set to True\n-for_zeroshot: for zero-shot prediction or not.\n-for_supervised: for supervised prediction or not.\n-for_ablation: for ablation study or not.\n\n# Create the instruction data for traning\npython instruction_generate.py -dataset_name NYC_multi\n\n# Create instruction data for the NYC_taxi dataset to facilitate testing in the zero-shot setting of UrbanGPT\npython instruction_generate.py -dataset_name NYC_taxi -for_zeroshot True\n```\n\n---------\n\n\u003c!--\n## Contact\nFor any questions or feedback, feel free to contact [Zhonghang Li](mailto:bjdwh.zzh@gmail.com).\n--\u003e\n\n## Citation\n\nIf you find UrbanGPT useful in your research or applications, please kindly cite:\n\n```\n@misc{li2024urbangpt,\n      title={UrbanGPT: Spatio-Temporal Large Language Models}, \n      author={Zhonghang Li and Lianghao Xia and Jiabin Tang and Yong Xu and Lei Shi and Long Xia and Dawei Yin and Chao Huang},\n      year={2024},\n      eprint={2403.00813},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL}\n}\n```\n\n\n\n## Acknowledgements\nYou may refer to related work that serves as foundations for our framework and code repository, \n[Vicuna](https://github.com/lm-sys/FastChat). We also partially draw inspirations from [GraphGPT](https://github.com/HKUDS/GraphGPT). The design of our website and README.md was inspired by [NExT-GPT](https://next-gpt.github.io/), and the design of our system deployment was inspired by [gradio](https://www.gradio.app) and [Baize](https://github.com/project-baize/baize-chatbot). Thanks for their wonderful works.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhkuds%2Furbangpt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhkuds%2Furbangpt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhkuds%2Furbangpt/lists"}