{"id":18488216,"url":"https://github.com/choyingw/cfcnet","last_synced_at":"2025-07-12T13:08:19.512Z","repository":{"id":112372087,"uuid":"207868978","full_name":"choyingw/CFCNet","owner":"choyingw","description":"NeurIPS 2019: Deep RGB-D Canonical Correlation Analysis For Sparse Depth Completion","archived":false,"fork":false,"pushed_at":"2021-04-19T19:03:51.000Z","size":33334,"stargazers_count":37,"open_issues_count":1,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-09T15:47:14.267Z","etag":null,"topics":["3d","canonical-correlation-analysis","computer-vision","deep-neural-networks","depth-completion","depth-estimation","multimodal-deep-learning","neurips-2019","nips-2019"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/choyingw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-11T17:29:53.000Z","updated_at":"2024-09-17T13:34:04.000Z","dependencies_parsed_at":"2023-05-14T01:00:29.391Z","dependency_job_id":null,"html_url":"https://github.com/choyingw/CFCNet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/choyingw/CFCNet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/choyingw%2FCFCNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/choyingw%2FCFCNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/choyingw%2FCFCNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/choyingw%2FCFCNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/choyingw","download_url":"https://codeload.github.com/choyingw/CFCNet/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/choyingw%2FCFCNet/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264995622,"owners_count":23694997,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","canonical-correlation-analysis","computer-vision","deep-neural-networks","depth-completion","depth-estimation","multimodal-deep-learning","neurips-2019","nips-2019"],"created_at":"2024-11-06T12:51:25.621Z","updated_at":"2025-07-12T13:08:19.505Z","avatar_url":"https://github.com/choyingw.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep RGB-D Canonical Correlation Analysis for Sparse Depth Completion\nThis is the official PyTorch implemenation for our NeurIPS 2019 paper by Yiqi Zhong\\*, Cho-Ying Wu\\*, Suya You, Ulrich Neumann (\\*Equal Contribution) at USC \n\nPaper: [\u003ca href=\"https://arxiv.org/abs/1906.08967\"\u003eArxiv\u003c/a\u003e].\n\n\u003cimg src='images/500.gif'\u003e\n\nCheck out the whole video demo [\u003ca href=\"https://www.youtube.com/watch?v=6HCWipHkv60\"\u003eYoutube\u003c/a\u003e].\n\n**Also check our newest work on depth estimation/completion using sensor fusion \u003ca href=\"https://github.com/choyingw/SCADC-DepthCompletion\"\u003eSCADC\u003c/a\u003e!**\n\n# Prerequisites\n\tLinux\n\tPython 3\n\tPyTorch 1.0+ (Orginally developed upder v1.0, testing on v1.5 is also fine)\n\tNVIDIA GPU + CUDA CuDNN\n\tOther common libraries: matplotlib, cv2, PIL\n\n# Getting Started\n\nData Preparation: \n\tPlease refer to [\u003ca href=\"http://www.cvlibs.net/datasets/kitti/index.php\"\u003eKITTI\u003c/a\u003e] or [\u003ca href=\"https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html\"\u003eNYU Depth V2\u003c/a\u003e] and process them into h5 files. \u003ca href=\"https://github.com/fangchangma/sparse-to-dense.pytorch\"\u003eHere\u003c/a\u003e also provides preprocessed data.\n\n# Tutorial:\n\n1. Create a folder and a subfolder 'checkpoint/kitti'\n2. Download the pretrained weights: [\u003ca href=\"https://drive.google.com/file/d/1rFvrqQ1Qf5bT_WSmtZZP5c-FKAhRHKUn/view?usp=sharing\"\u003eNYU-Depth 500 points training\u003c/a\u003e] [\u003ca href=\"https://drive.google.com/open?id=1RJZMnohlp9OVSkxkSUWm7psnbW2mRunH\"\u003eKITTI 500 points training\u003c/a\u003e] and put the .pth under 'checkpoint/kitti/'\n3. Prepare data in the previous \"getting started\" section\n4. Run \"python3 evaluate.py --name kitti --checkpoints_dir ./checkpoint/ --test_path [path ot the testing file] \"\n4. You'll see completed depth maps are saved under 'vis/'\n\n# Train/Evaluation:\n\nFor training, please run\n\n\tpython3 train_depth_complete.py --name kitti --checkpoints_dir [path to save_dir] --train_path [train_data_dir] --test_path [test_data_dir]\n\nIf you use the preprocessed data from \u003ca href=\"https://github.com/fangchangma/sparse-to-dense.pytorch\"\u003ehere\u003c/a\u003e. The train/test data path should be ./kitti/train or ./kitti/val/ under your data directory.\n\nIf you want to use your data, please make your data into h5 dataset. (See dataloaders/dataloader.py) \n\nOther specifications: `--continue_train` would load the lastest saved ckpt. Also set --epoch_count to tell what's the next epoch_number. Otherwise, will start from epoch 0. Set hyperparameters by `--lr`, `--batch_size`, `--weight_decay`, or others. Please refer to the options/base_options.py and options/options.py\n\nNote that the default batch size is 4 during the training and use gpu:0. You can set larger batch size (--batch_size=xx) with more gpus (--gpu_ids=\"0,1,2,3\") to attain larger batch size training.\n\nExample command:\n\n\tpython3 train_depth_complete.py --name kitti --checkpoints_dir ./checkpoints --lr 0.001 --batch_size 4 --train_path './kitti/train/' --test_path './kitti/val/' --continue_train --epoch_count [next_epoch_number]\n\t\nFor evalutation, please run\n\n\tpython3 evaluate.py --name kitti --checkpoints_dir [path to save_dir to load ckpt] --test_path [test_data_dir] [--epoch [epoch number]]\n\nThis will load the latest checkpoint to evaluate. Add `--epoch` to specify which epoch checkpoint you want to load.\n\n# Update: 02/10/2020\n\n1.Fix several bugs and take off redundant options.\n\n2.Release Orb sparsifier\n\n3.Pretrain models release: [\u003ca href=\"https://drive.google.com/file/d/1rFvrqQ1Qf5bT_WSmtZZP5c-FKAhRHKUn/view?usp=sharing\"\u003eNYU-Depth 500 points training\u003c/a\u003e] [\u003ca href=\"https://drive.google.com/open?id=1RJZMnohlp9OVSkxkSUWm7psnbW2mRunH\"\u003eKITTI 500 points training\u003c/a\u003e]\n\n\n# Update: 04/19/2021\n\n1. Revise README and add a tutorial\n2. Several minor revisions\n\n\nIf you find our work useful, please consider to cite our work.\n\n\t@inproceedings{zhong2019deep,\n\t  title={Deep rgb-d canonical correlation analysis for sparse depth completion},\n\t  author={Zhong, Yiqi and Wu, Cho-Ying and You, Suya and Neumann, Ulrich},\n\t  booktitle={Advances in Neural Information Processing Systems},\n\t  pages={5332--5342},\n\t  year={2019}\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchoyingw%2Fcfcnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchoyingw%2Fcfcnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchoyingw%2Fcfcnet/lists"}