{"id":20373519,"url":"https://github.com/megengine/gyroflow","last_synced_at":"2025-04-12T06:51:48.424Z","repository":{"id":40603737,"uuid":"397143683","full_name":"MegEngine/GyroFlow","owner":"MegEngine","description":"The official MegEngine implementation of the ICCV 2021 paper: GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning","archived":false,"fork":false,"pushed_at":"2023-07-29T01:16:53.000Z","size":3989,"stargazers_count":48,"open_issues_count":0,"forks_count":4,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-12T06:51:42.501Z","etag":null,"topics":["gyroscope","megengine","optical-flow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MegEngine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-17T07:00:50.000Z","updated_at":"2025-03-26T05:05:25.000Z","dependencies_parsed_at":"2022-08-24T23:41:07.121Z","dependency_job_id":null,"html_url":"https://github.com/MegEngine/GyroFlow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MegEngine%2FGyroFlow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MegEngine%2FGyroFlow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MegEngine%2FGyroFlow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MegEngine%2FGyroFlow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MegEngine","download_url":"https://codeload.github.com/MegEngine/GyroFlow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248530604,"owners_count":21119595,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gyroscope","megengine","optical-flow"],"created_at":"2024-11-15T01:18:49.718Z","updated_at":"2025-04-12T06:51:48.366Z","avatar_url":"https://github.com/MegEngine.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [ICCV 2021] GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning\n\n\u003ch4 align=\"center\"\u003e Haipeng Li\u003csup\u003e1\u003c/sup\u003e, Kunming Luo\u003csup\u003e1\u003c/sup\u003e, Shuaicheng Liu\u003csup\u003e2,1\u003c/sup\u003e\u003c/h4\u003e\n\u003ch4 align=\"center\"\u003e 1. Megvii Research, 2. University of Electronic Science and Technology of China\u003c/h4\u003e\n\nThis is the official implementation of our ICCV2021 paper [GyroFlow](https://openaccess.thecvf.com/content/ICCV2021/html/Li_GyroFlow_Gyroscope-Guided_Unsupervised_Optical_Flow_Learning_ICCV_2021_paper.html). We also provide a PyTorch version, check at [GyroFlow-PyTorch](https://github.com/lhaippp/GyroFlow-PyTorch)\n\n\n## Abstract\nExisting optical flow methods are erroneous in challenging scenes, such as fog, rain, and night because the basic optical flow assumptions such as brightness and gradient constancy are broken. To address this problem, we present an unsupervised learning approach that fuses gyroscope into optical flow learning. Specifically, we first convert gyroscope readings into motion fields named gyro field. Second, we design a self-guided fusion module to fuse the background motion extracted from the gyro field with the optical flow and guide the network to focus on motion details. To the best of our knowledge, this is the first deep learning-based framework that fuses gyroscope data and image content for optical flow learning. To validate our method, we propose a new dataset that covers regular and challenging scenes. Experiments show that our method outperforms the state-of-art methods in both regular and challenging scenes.\n\n## Presentation video\n[[Youtube](https://www.youtube.com/watch?v=6gh40PyWdHM)][[Bilibili](https://www.bilibili.com/video/BV1Tr4y127kd/)].\n\n## Dependencies\n\n* MegEngine==1.6.0\n* Other requirements please refer to`requirements.txt`.\n\n## Data Preparation\n\n2021.11.15: We release the GOF_Train V1 that contains 2000 samples.\n\n2022.06.22: We release the PyTorch Version, welcome to have a try.\n\n2023.07.28: Please check to [GyroFlow-PyTorch](https://github.com/lhaippp/GyroFlow-PyTorch) for Data Preparation \n\n## Training and Evaluation\n\n### Training\n\nTo train the model, you can just run:\n\n```\npython train.py --model_dir experiments\n```\n\n### Evaluation\n\nLoad the pretrained checkpoint and run:\n\n```\npython evaluate.py --model_dir experiments --restore_file experiments/val_model_best.pkl\n```\n\nWe've updated the GOF (both trainset and testset), so the performance is a little bit different from the results reported in our paper.\n\nMegEngine checkpoint can be download via [Google Drive].\n\n## Citation\n\nIf you think this work is useful for your research, please kindly cite:\n\n```\n@InProceedings{Li_2021_ICCV,\n    author    = {Li, Haipeng and Luo, Kunming and Liu, Shuaicheng},\n    title     = {GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning},\n    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},\n    month     = {October},\n    year      = {2021},\n    pages     = {12869-12878}\n}\n```\n\n## Acknowledgments\n\nIn this project we use (parts of) the official implementations of the following works:\n\n* [ARFlow](https://github.com/lliuz/ARFlow)\n* [UpFlow](https://github.com/coolbeam/UPFlow_pytorch)\n* [RAFT](https://github.com/princeton-vl/RAFT)\n* [DeepOIS](https://github.com/lhaippp/DeepOIS)\n\nWe thank the respective authors for open sourcing their methods.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmegengine%2Fgyroflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmegengine%2Fgyroflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmegengine%2Fgyroflow/lists"}