{"id":13445301,"url":"https://github.com/kaanakan/stretchbev","last_synced_at":"2025-03-20T20:32:21.464Z","repository":{"id":46885731,"uuid":"515623423","full_name":"kaanakan/stretchbev","owner":"kaanakan","description":"Official implementation of our ECCV paper \"StretchBEV: Stretching Future Instance Prediction Spatially and Temporally\"","archived":false,"fork":false,"pushed_at":"2022-10-06T13:10:58.000Z","size":160,"stargazers_count":42,"open_issues_count":4,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-08-01T05:13:17.758Z","etag":null,"topics":["autonomous-driving","autonomous-vehicles","deep-learning","eccv2022","future-prediction","generative-models","instance-segmentation","latent-variable-models","pytorch","segmentation","state-space-models","stochastic","vae"],"latest_commit_sha":null,"homepage":"https://kuis-ai.github.io/stretchbev/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kaanakan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-19T14:40:10.000Z","updated_at":"2024-07-30T05:34:27.000Z","dependencies_parsed_at":"2023-01-19T11:16:21.491Z","dependency_job_id":null,"html_url":"https://github.com/kaanakan/stretchbev","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaanakan%2Fstretchbev","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaanakan%2Fstretchbev/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaanakan%2Fstretchbev/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaanakan%2Fstretchbev/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kaanakan","download_url":"https://codeload.github.com/kaanakan/stretchbev/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221807658,"owners_count":16883629,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autonomous-driving","autonomous-vehicles","deep-learning","eccv2022","future-prediction","generative-models","instance-segmentation","latent-variable-models","pytorch","segmentation","state-space-models","stochastic","vae"],"created_at":"2024-07-31T05:00:30.092Z","updated_at":"2024-10-28T08:30:36.572Z","avatar_url":"https://github.com/kaanakan.png","language":"Python","funding_links":[],"categories":["五、Summary of multi-task learning methods under BEV"],"sub_categories":["4. Summary of multimodal fusion methods under BEV"],"readme":"# StretchBEV: Stretching Future Instance Prediction Spatially and Temporally (ECCV 2022)\n\n\u003c!-- [![report](https://img.shields.io/badge/CVF-paper-orange)](https://openaccess.thecvf.com/content/ICCV2021/html/Akan_SLAMP_Stochastic_Latent_Appearance_and_Motion_Prediction_ICCV_2021_paper.html) --\u003e\n[![report](https://img.shields.io/badge/ArXiv-Paper-red)](https://arxiv.org/abs/2203.13641)\n[![report](https://img.shields.io/badge/Project-Page-blue)](https://kuis-ai.github.io/stretchbev/)\n[![report](https://img.shields.io/badge/Pretrained-Models-yellow)](https://github.com/kaanakan/stretchbev/releases/tag/v1.0)\n[![report](https://img.shields.io/badge/Presentation-Video-brightgreen)](https://www.youtube.com/watch?v=2SiUNs6BMVk)\n\u003c!-- [![report](https://img.shields.io/badge/Supplementary-Material-brightgreen)](https://kuis-ai.github.io/stretchbev/data/docs/StretchBEV_supp.pdf) --\u003e\n\n\n\u003e [**StretchBEV: Stretching Future Instance Prediction Spatially and Temporally**](https://arxiv.org/abs/2203.13641),            \n\u003e [Adil Kaan Akan](https://kaanakan.github.io), \n\u003e [Fatma Guney](https://mysite.ku.edu.tr/fguney/),      \n\u003e *European Conference on Computer Vision (ECCV), 2022* \n\n\u003cp float=\"center\"\u003e\n  \u003cimg src=\"https://kuis-ai.github.io/stretchbev/data/figures/comp/2sec_gifs/test_4042_2sec.gif\" width=\"100%\" /\u003e\n\u003c/p\u003e\n\n## Features\n\nStretchBEV is a future instance prediction network in Bird's-eye view representation. It earns temporal dynamics in a latent space through stochastic residual updates at each time step. By sampling from a learned distribution at each time step, we obtain more diverse future predictions that are also more accurate compared to previous work, especially stretching both spatially further regions in the scene and temporally over longer time horizons\n\n\n## Requirements\n\nAll models were trained with Python 3.7.10 and PyTorch 1.7.0\n\nA list of required Python packages is available in the `environment.yml` file.\n\n\n\n## Datasets\n\nFor preparations of datasets, we followed [FIERY](https://github.com/wayveai/fiery). Please follow [this link](https://github.com/wayveai/fiery/blob/master/DATASET.md) below if you want to construct the datasets.\n\n\n## Training\n\nTo train the model on NuScenes:\n\n- First, you need to download [`static_lift_splat_setting.ckpt`](https://github.com/wayveai/fiery/releases/download/v1.0/static_lift_splat_setting.ckpt) and copy it to this directory.\n- Run `python train.py --config fiery/configs/baseline.yml DATASET.DATAROOT ${NUSCENES_DATAROOT}`.\n\nThis will train the model on 4 GPUs, each with a batch of size 2. To train on single GPU add the flag `GPUS 1`, and to change the batch size use the flag `BATCHSIZE ${DESIRED_BATCHSIZE}`.\n\n\n## Evaluation\n\nTo evaluate a trained model on NuScenes:\n\n- Download [pre-trained weights](https://github.com/wayveai/fiery/releases/download/v1.0/stretchbev.ckpt).\n- Run `python evaluate.py --checkpoint ${CHECKPOINT_PATH} --dataroot ${NUSCENES_DATAROOT}`.\n\n### Pretrained weights\n\nYou can download the pretrained weights from the releases of this repository or the links below.\n\n[Normal setting weight](https://github.com/wayveai/fiery/releases/download/v1.0/stretchbev.ckpt)\n\n[Fishing setting weight](https://github.com/wayveai/fiery/releases/download/v1.0/stretchbev_fishing.ckpt)\n\n\n\n## How to Cite\n\nPlease cite the paper if you benefit from our paper or the repository:\n\n```\n@InProceedings{Akan2022ECCV,\n            author    = {Akan, Adil Kaan and G\\\"uney, Fatma},\n            title     = {StretchBEV: Stretching Future Instance Prediction Spatially and Temporally},\n            journal = {European Conference on Computer Vision (ECCV)},\n            year      = {2022},\n            }\n```\n\n## Acknowledgments\n\nWe would like to thank FIERY and SRVP authors for making their repositories public. This repository contains several code segments from [FIERY's repository](https://github.com/wayveai/fiery) and [SRVP's repository](https://github.com/edouardelasalles/srvp). We appreciate the efforts by [Berkay Ugur Senocak](https://github.com/4turkuaz) for cleaning the code before release.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaanakan%2Fstretchbev","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkaanakan%2Fstretchbev","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaanakan%2Fstretchbev/lists"}