{"id":14961244,"url":"https://github.com/mit-spark/lp2","last_synced_at":"2025-10-24T20:31:27.939Z","repository":{"id":237300457,"uuid":"791338950","full_name":"MIT-SPARK/LP2","owner":"MIT-SPARK","description":"Long-term Human Trajectory Prediction using 3D DSGs","archived":false,"fork":false,"pushed_at":"2025-02-07T16:58:37.000Z","size":21995,"stargazers_count":29,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-07T17:36:36.610Z","etag":null,"topics":["dsg","human","llm","long-term","motion","prediction","probabilistic","trajectory"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MIT-SPARK.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-24T14:37:34.000Z","updated_at":"2025-02-07T16:58:40.000Z","dependencies_parsed_at":"2025-02-07T17:39:20.700Z","dependency_job_id":null,"html_url":"https://github.com/MIT-SPARK/LP2","commit_stats":null,"previous_names":["mit-spark/lp2"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FLP2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FLP2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FLP2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FLP2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MIT-SPARK","download_url":"https://codeload.github.com/MIT-SPARK/LP2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238035385,"owners_count":19405682,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dsg","human","llm","long-term","motion","prediction","probabilistic","trajectory"],"created_at":"2024-09-24T13:24:16.249Z","updated_at":"2025-10-24T20:31:22.875Z","avatar_url":"https://github.com/MIT-SPARK.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![](https://github.com/MIT-SPARK/LP2/blob/main/assets/project_page_title.gif?raw=true \"Our method, LP^2, predicts a spatio-temporal distribution over long-term (up to 60 s) human trajectories in complex environments by reasoning about their interactions with the scene, represented as a 3D Dynamic Scene Graph.\")\n\n# LP\u003csup\u003e2\u003c/sup\u003e: Language-based Probabilistic Long-term Prediction\nThis repository contains the code for LP\u003csup\u003e2\u003c/sup\u003e, our approach for Long-term Human Trajectory Prediction using 3D Dynamic Scene Graphs.\n\n\u003e This project was supported by Amazon, Lockheed Martin, and the Swiss National Science Foundation (SNSF).\n\n# Table of Contents\n**Credits**\n* [Paper](#paper)\n* [Video](#video)\n\n**Setup**\n* [OpenAI API Setup](#openai-api-setup)\n* [Installation](#installation)\n* [Dataset](#dataset)\n\n\n **Examples**\n* [Running LP\u003csup\u003e2\u003c/sup\u003e](#running)\n* [Visualization](#visualization) \n\n# Paper\nIf you find this useful for your research, please consider citing our paper:\n\n* Nicolas Gorlo, Lukas Schmid, and Luca Carlone, \"**Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs**\", in _IEEE Robotics and Automation Letters_, vol. 9, no. 11, pp. 10978-10985, 2024. [ [Paper](https://ieeexplore.ieee.org/document/10720207) | [Preprint](https://arxiv.org/abs/2405.00552) | [Video](https://www.youtube.com/watch?v=mzumT3T0dYw) ]\n  ```bibtex\n   @ARTICLE{Gorlo2024LP2,\n    author={Gorlo, Nicolas and Schmid, Lukas and Carlone, Luca},\n    journal={IEEE Robotics and Automation Letters},\n    title={Long-Term Human Trajectory Prediction Using 3D Dynamic Scene Graphs},\n    year={2024},\n    volume={9},\n    number={12},\n    pages={10978-10985},\n    doi={10.1109/LRA.2024.3482169}\n    }\n  ```\n# Video\n\nAn overview of our approach is available on [YouTube](https://www.youtube.com/watch?v=mzumT3T0dYw):\n\n[\u003cimg src=https://github.com/MIT-SPARK/LP2/assets/36043993/0dd28295-3b72-468b-8420-56477f910e8b alt=\"Youtube Video\"\u003e](https://www.youtube.com/watch?v=mzumT3T0dYw)\n\n# OpenAI API Setup \nOur method uses the OpenAI API to predict interactions between humans and the scene. To use the OpenAI API, you need to create an account and obtain an API key. You can find more information on the OpenAI API [here](https://platform.openai.com/docs/overview).\nSpecifically, you need to setup an account and buy Pay-as-you-go credits to be able to use the API.\nOur code relies on the following environment variables to access the OpenAI API. These will be associated with your personal account:\n```bash\nexport OPENAI_API_KEY=???\nexport OPENAI_API_ORG=???\n```\nOnce your account is set up, the keys are available in the [OpenAI profile tab](https://platform.openai.com/settings/profile?tab=api-keys).\n\n# Installation \n1. Clone the repository\n```bash\ngit clone git@github.com:MIT-SPARK/LP2.git\ncd LP2\n```\n2. Create a python 3.10 environment (using [pyenv](https://github.com/pyenv/pyenv))\n```bash\npyenv install 3.10.15\npyenv virtualenv 3.10.15 LP2\npyenv activate LP2\n```\n3. Install the required packages\n```bash\npip install -r requirements.txt\n```\n\n# Dataset\nTo download the dataset, run:\n```bash\npython scripts/download_data.py\n```\n\n# Running\nThe main script of our codebase will first run the pipeline for each trajectory in the selected split of the dataset. Then, it will evaluate the predictions and save the results in the `output` folder. To add additional steps or load checkpoints created while running the method, specify the corresponding parameters in the config file (e.g., `'global_config/global/load_checkpoints'`, `'project_config/global/animate'`).\nFeel free to adjust the configurations in the `config` folder or choose another scene config to select the part of the dataset to predict on. \nThe following command will run the pipeline for the LP\u003csup\u003e2\u003c/sup\u003e method on the office scene.\n\n```bash\npython src/lhmp/main.py --run_pipeline \"y\" \\\n                            --global_config_file \"config/global_config.yaml\" \\\n                            --method_config_file \"config/method_configs/project_config_LP2.yaml\" \\\n                            --scene_config_file \"config/scene_configs/scene_config_office.yaml\"\n```\n\nTo run LP\u003csup\u003e2\u003c/sup\u003e on your own data, you can bring the data into the same format as the downloaded data and create a new scene config file specifying the paths to the data.\nNote, that only the parameters `scene_graph_path`, `room_labels_path`, and `trajectory_dir` are required to run the method. The hierarchical 3D scene graph is in the [Spark-DSG](https://github.com/MIT-SPARK/Spark-DSG) json format.\n\n# Visualization\nTo plot the negative log likelihood of predictions, use the following command:\n```bash\npython3 scripts/visualize_all.py --methods \"LP2\" \"LP2_instance\" \\\n                                 --scenes \"office\" \"home\" \\\n                                 --n_past_interactions 2 \\\n                                 --time_lim 60.0 \\\n                                 --plot_std False\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-spark%2Flp2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmit-spark%2Flp2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-spark%2Flp2/lists"}