{"id":13816453,"url":"https://github.com/haofeixu/aanet","last_synced_at":"2025-05-15T15:32:39.135Z","repository":{"id":40518672,"uuid":"250151347","full_name":"haofeixu/aanet","owner":"haofeixu","description":"[CVPR'20] AANet: Adaptive Aggregation Network for Efficient Stereo Matching","archived":false,"fork":false,"pushed_at":"2022-11-15T01:45:00.000Z","size":3208,"stargazers_count":533,"open_issues_count":4,"forks_count":102,"subscribers_count":16,"default_branch":"master","last_synced_at":"2024-11-19T13:38:20.700Z","etag":null,"topics":["cost-aggregation","cost-volume","cvpr2020","deformable-convolution","edge-preserving","stereo-matching","stereo-vision"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/haofeixu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-26T03:24:38.000Z","updated_at":"2024-11-11T03:30:16.000Z","dependencies_parsed_at":"2022-06-29T21:28:54.692Z","dependency_job_id":null,"html_url":"https://github.com/haofeixu/aanet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haofeixu%2Faanet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haofeixu%2Faanet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haofeixu%2Faanet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haofeixu%2Faanet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/haofeixu","download_url":"https://codeload.github.com/haofeixu/aanet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254367688,"owners_count":22059556,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cost-aggregation","cost-volume","cvpr2020","deformable-convolution","edge-preserving","stereo-matching","stereo-vision"],"created_at":"2024-08-04T05:00:42.122Z","updated_at":"2025-05-15T15:32:36.720Z","avatar_url":"https://github.com/haofeixu.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# AANet\n\nPyTorch implementation of our paper: \n\n**[AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://arxiv.org/abs/2004.09548)**, [CVPR 2020](http://cvpr2020.thecvf.com/)\n\nAuthors: [Haofei Xu](https://haofeixu.github.io/) and [Juyong Zhang](http://staff.ustc.edu.cn/~juyong/)\n\n**11/15/2022 Update: Check out our new work: [Unifying Flow, Stereo and Depth Estimation](https://haofeixu.github.io/unimatch/) and code: [unimatch](https://github.com/autonomousvision/unimatch) for performing stereo matching with our new GMStereo model. The CUDA op in AANet is no longer required. [10 pretrained GMStereo models](https://github.com/autonomousvision/unimatch/blob/master/MODEL_ZOO.md) with different speed-accuracy trade-offs are also released. Check out our [Colab](https://colab.research.google.com/drive/1r5m-xVy3Kw60U-m5VB-aQ98oqqg_6cab?usp=sharing) and [HuggingFace](https://huggingface.co/spaces/haofeixu/unimatch) demo to play with GMStereo in your browser!**\n\nWe propose a sparse points based intra-scale cost aggregation (ISA) module and a cross-scale cost aggregation (CSA) module for efficient and accurate stereo matching. \n\nThe implementation of improved version **AANet+ (stronger performance \u0026 slightly faster speed)** is also included in this repo.\n\n\u003cp align=\"center\"\u003e\u003cimg width=80% src=\"assets/overview.png\"\u003e\u003c/p\u003e\n\n## Highlights\n\n- **Modular design**\n\n  We decompose the end-to-end stereo matching framework into five components: \n\n  **feature extraction**, **cost volume construction**, **cost aggregation**, **disparity computation** and **disparity refinement.** \n\n  One can easily construct a customized stereo matching model by combining different components.\n\n- **High efficiency**\n\n  Our method can run at **60ms** for a KITTI stereo pair (384x1248 resolution)!\n\n- **Full framework**\n\n  All codes for training, validating, evaluating, inferencing and predicting on any stereo pair are provided!\n\n## Installation\n\nOur code is based on PyTorch 1.2.0, CUDA 10.0 and python 3.7. \n\nWe recommend using [conda](https://www.anaconda.com/distribution/) for installation: \n\n```shell\nconda env create -f environment.yml\n```\n\nAfter installing dependencies, build deformable convolution:\n\n```shell\ncd nets/deform_conv \u0026\u0026 bash build.sh\n```\n\n## Dataset Preparation\n\nDownload [Scene Flow](https://lmb.informatik.uni-freiburg.de/resources/datasets/SceneFlowDatasets.en.html), [KITTI 2012](http://www.cvlibs.net/datasets/kitti/eval_stereo_flow.php?benchmark=stereo) and [KITTI 2015](http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=stereo) datasets. \n\nOur folder structure is as follows:\n\n```\ndata\n├── KITTI\n│   ├── kitti_2012\n│   │   └── data_stereo_flow\n│   ├── kitti_2015\n│   │   └── data_scene_flow\n└── SceneFlow\n    ├── Driving\n    │   ├── disparity\n    │   └── frames_finalpass\n    ├── FlyingThings3D\n    │   ├── disparity\n    │   └── frames_finalpass\n    └── Monkaa\n        ├── disparity\n        └── frames_finalpass\n```\n\nIf you would like to use the pseudo ground truth supervision introduced in our paper, you can download the pre-computed disparity on KITTI 2012 and KITTI 2015 training set here: [KITTI 2012](https://drive.google.com/open?id=1ZJhraqgY1sL4UfHBrVojttCbvNAXfdj0), [KITTI 2015](https://drive.google.com/open?id=14NGQp9CwIVNAK8ZQ6GSNeGraFGtVGOce). \n\nFor KITTI 2012, you should place the unzipped file `disp_occ_pseudo_gt` under `kitti_2012/data_stereo_flow/training` directory. \n\nFor KITTI 2015, you should place `disp_occ_0_pseudo_gt` under `kitti_2015/data_scene_flow/training`.\n\nIt is recommended to symlink your dataset root to `$AANET/data`:\n\n```shell\nln -s $YOUR_DATASET_ROOT data\n```\n\nOtherwise, you may need to change the corresponding paths in the scripts.\n\n## Model Zoo\n\nAll pretrained models are available in the [model zoo](MODEL_ZOO.md).\n\nWe assume the downloaded weights are located under the `pretrained` directory. \n\nOtherwise, you may need to change the corresponding paths in the scripts.\n\n## Inference\n\nTo generate prediction results on the test set of Scene Flow and KITTI dataset, you can run [scripts/aanet_inference.sh](scripts/aanet_inference.sh). \n\nThe inference results on KITTI dataset can be directly submitted to the online evaluation server for benchmarking.\n\n## Prediction\n\nWe also support predicting on any rectified stereo pairs. [scripts/aanet_predict.sh](scripts/aanet_predict.sh) provides an example usage.\n\n## Training\n\nAll training scripts on Scene Flow and KITTI datasets are provided in [scripts/aanet_train.sh](scripts/aanet_train.sh). \n\nNote that we use 4 NVIDIA V100 GPUs (32G) with batch size 64 for training, you may need to tune the batch size according to your hardware. \n\nWe support using tensorboard to monitor and visualize the training process. You can first start a tensorboard session with\n\n```shell\ntensorboard --logdir checkpoints\n```\n\nand then access [http://localhost:6006](http://localhost:6006) in your browser.\n\n- **How to train on my own data?**\n\n  You can first generate a filename list by creating a data reading function in [filenames/generate_filenames.py](filenames/generate_filenames.py) (an example on KITTI dataset is provided), and then create a new dataset dictionary in [dataloader/dataloader.py](dataloader/dataloader.py).\n\n- **How to develop new components?**\n\n  Our framework is flexible to develop new components, e.g., new feature extractor, cost aggregation module or refinement architecture. You can 1) create a new file (e.g., `my_aggregation.py`) under `nets` directory, 2) import the module in `nets/aanet.py` and 3) use it in the model definition.\n\n## Evaluation\n\nTo enable fast experimenting, evaluation runs on-the-fly without saving the intermediate results. \n\nWe provide two types of evaluation setting:\n\n- After training, evaluate the model with best validation results\n- Evaluate a pretrained model\n\nCheck [scripts/aanet_evaluate.sh](scripts/aanet_evaluate.sh) for an example usage.\n\n## Citation\n\nIf you find our work useful in your research, please consider citing our paper:\n\n```\n@inproceedings{xu2020aanet,\n  title={AANet: Adaptive Aggregation Network for Efficient Stereo Matching},\n  author={Xu, Haofei and Zhang, Juyong},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={1959--1968},\n  year={2020}\n}\n```\n\n\n## Acknowledgements\n\nPart of the code is adopted from previous works: [PSMNet](https://github.com/JiaRenChang/PSMNet), [GwcNet](https://github.com/xy-guo/GwcNet) and [GA-Net](https://github.com/feihuzhang/GANet). We thank the original authors for their awesome repos. The deformable convolution op is taken from [mmdetection](https://github.com/open-mmlab/mmdetection). The FLOPs counting code is modified from [pytorch-OpCounter](https://github.com/Lyken17/pytorch-OpCounter). The code structure is partially inspired by [mmdetection](https://github.com/open-mmlab/mmdetection) and our previous work [rdn4depth](https://github.com/haofeixu/rdn4depth).\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaofeixu%2Faanet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhaofeixu%2Faanet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaofeixu%2Faanet/lists"}