{"id":33174743,"url":"https://github.com/zhou13/neurvps","last_synced_at":"2025-11-20T20:01:41.567Z","repository":{"id":41176264,"uuid":"195754756","full_name":"zhou13/neurvps","owner":"zhou13","description":"Neural Vanishing Point Scanning via Conic Convolution","archived":false,"fork":false,"pushed_at":"2024-08-13T16:01:15.000Z","size":951,"stargazers_count":176,"open_issues_count":0,"forks_count":23,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-08-13T19:12:56.776Z","etag":null,"topics":["cnn","conic-convolution","deep-learning","deep-neural-networks","detected-vanishing-points","lsd","neurips-2019","paper","pytorch","vanishing-points"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhou13.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-08T07:01:12.000Z","updated_at":"2024-08-13T16:01:18.000Z","dependencies_parsed_at":"2022-07-21T11:33:14.081Z","dependency_job_id":null,"html_url":"https://github.com/zhou13/neurvps","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zhou13/neurvps","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhou13%2Fneurvps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhou13%2Fneurvps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhou13%2Fneurvps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhou13%2Fneurvps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhou13","download_url":"https://codeload.github.com/zhou13/neurvps/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhou13%2Fneurvps/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":285503668,"owners_count":27182913,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-20T02:00:05.334Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cnn","conic-convolution","deep-learning","deep-neural-networks","detected-vanishing-points","lsd","neurips-2019","paper","pytorch","vanishing-points"],"created_at":"2025-11-16T02:00:33.687Z","updated_at":"2025-11-20T20:01:41.557Z","avatar_url":"https://github.com/zhou13.png","language":"Python","funding_links":[],"categories":["References"],"sub_categories":["Papers - Scene level"],"readme":"# NeurVPS: Neural Vanishing Point Scanning via Conic Convolution\n\nThis repository contains the official PyTorch implementation of the paper: _[Yichao Zhou](https://yichaozhou.com), [Haozhi Qi](http://haozhi.io), [Jingwei Huang](http://stanford.edu/~jingweih/), [Yi Ma](https://people.eecs.berkeley.edu/~yima/). [\"NeurVPS: Neural Vanishing Point Scanning via Conic Convolution\"](https://arxiv.org/abs/1910.06316)_. NeurIPS 2019.\n\n## Introduction\n\nNeurVPS is an end-to-end trainable deep network with _geometry-inspired_ convolutional operators for detecting vanishing points in images. With the power of data-driven approaches and geometrical priors, NeurVPS is able to outperform the previous state-of-the-art vanishing point detection methods such as [LSD/J-Linkage](https://github.com/simbaforrest/vpdetection) and [Contour (TMM17)](https://github.com/heiheiknight/vpdet_tmm17).\n\n## Main Results\n\n### Qualitative Measures\n\n| [SceneCity Urban 3D (SU3)](https://arxiv.org/abs/1905.07482) | [Natural Scene (TMM17)](https://faculty.ist.psu.edu/zzhou/projects/vpdetection/) | [ScanNet](http://www.scan-net.org/) |\n| ------------------------------------------------------------ | -------------------------------------------------------------------------------- | ----------------------------------- |\n| ![blend](figs/su3.png)                                       | ![tmm17](figs/tmm17.png)                                                         | ![scannet](figs/scannet.png)        |\n\nRandom sampled results can be found in the [supplementary material](https://yichaozhou.com/publication/1905neurvps/appendix.pdf) of the paper.\n\n### Quantitative Measures\n\n| [SceneCity Urban 3D (SU3)](https://arxiv.org/abs/1905.07482) | [Natural Scene (TMM17)](https://faculty.ist.psu.edu/zzhou/projects/vpdetection/) | [ScanNet](http://www.scan-net.org/)           |\n| ------------------------------------------------------------ | -------------------------------------------------------------------------------- | --------------------------------------------- |\n| \u003cimg src=\"figs/su3_AA6.svg\" width=\"260\"\u003e                     | \u003cimg src=\"figs/tmm17_AA12.svg\" width=\"260\"\u003e                                      | \u003cimg src=\"figs/scannet_AA20.svg\" width=\"260\"\u003e |\n\nHere, the x-axis represents the angle accuracy of the detected vanishing points and the y-axis represents the percentage of the results whose error is less than that. Our conic convolutional networks outperform all the baseline methods and previous state-of-the-art vanishing point detection approaches, while naive CNN implementations might under-perform those traditional methods, especially in the high-accuracy regions.\n\n### Code Structure\n\nBelow is a quick overview of the function of each file.\n\n```bash\n########################### Data ###########################\ndata/                           # default folder for placing the data\n    su3/                        # folder for SU3 dataset\n    tmm17/                      # folder for TMM17 dataset\n    scannet-vp/                 # folder for ScanNet dataset\nlogs/                           # default folder for storing the output during training\n########################### Code ###########################\nconfig/                         # neural network hyper-parameters and configurations\n    su3.yaml                    # default parameters for SU3 dataset\n    tmm17.yaml                  # default parameters for TMM17 dataset\n    scannet.yaml                # default parameters for scannet dataset\ndataset/                        # all scripts related to data generation\n    su3.py                      # script for pre-processing the SU3 dataset to npz\nmisc/                           # misc scripts that are not important\n    find-radius.py              # script for generating figure grids\nneurvps/                        # neurvps module so you can \"import neurvps\" in other scripts\n    models/                     # neural network architectures\n        cpp/                    # CUDA kernel for deformable convolution\n        deformable.py           # python wrapper for deformable convolution layers\n        conic.py                # conic convolution layers\n        hourglass_pose.py       # backbone network\n        vanishing_net.py        # main network\n    datasets.py                 # reading the training data\n    trainer.py                  # trainer\n    config.py                   # global variables for configuration\n    utils.py                    # misc functions\ntrain.py                        # script for training the neural network\neval.py                         # script for evaluating a dataset from a checkpoint\n```\n\n## Reproducing Results\n\n### Installation\n\nFor the ease of reproducibility, you are suggested to install [miniconda](https://docs.conda.io/en/latest/miniconda.html) (or [anaconda](https://www.anaconda.com/distribution/) if you prefer) on a Linux environment before following executing the following commands.\n\nNote that as of 2025/08/08, pytorch deprecated the support of conda.  The old instruction up to CUDA 12.4 can still be found in https://pytorch.org/get-started/previous-versions/.\n\n```bash\ngit clone https://github.com/zhou13/neurvps\ncd neurvps\nconda create -y -n neurvps\nsource activate neurvps\n# Replace pytorch installation with the matching CUDA version in https://pytorch.org/get-started/previous-versions/\nconda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.4 -c pytorch -c nvidia\nconda install -y tensorboardx -c conda-forge\nconda install -y pyyaml docopt matplotlib scikit-image opencv tqdm\nmkdir data logs\n```\n\n### Downloading the Processed Datasets\n\nMake sure `curl` is installed on your system and execute\n\n```bash\ncd data\nwget https://huggingface.co/yichaozhou/neurvps/resolve/main/Data/scannet-vp/scannet-vp.z0{1..7}\nwget https://huggingface.co/yichaozhou/neurvps/resolve/main/Data/scannet-vp/scannet-vp.zip\nzip -FF scannet-vp.zip --out scannet-vp-full.zip\nunzip -FF scannet-vp-full.zip\nwget https://huggingface.co/yichaozhou/neurvps/resolve/main/Data/su3.tar.xz\nwget https://huggingface.co/yichaozhou/neurvps/resolve/main/Data/tmm17.tar.xz\ntar xf su3.tar.xz\ntar xf tmm17.tar.xz\nrm *.tar.xz *.z*\ncd ..\n```\n\nAlternatively, you can download the processed datasets manually from our [Hugging Face repo](https://huggingface.co/yichaozhou/neurvps/tree/main/Data) and proceed\naccordingly.\n\n### Training\n\nExecute the following commands to train the neural networks from scratch on 2 GPUs (GPU 0 and GPU 1, specified by `-d 0,1`) with the default parameters:\n\n```bash\npython ./train.py -d 0,1 --identifier su3 config/su3.yaml\npython ./train.py -d 0,1 --identifier tmm17 config/tmm17.yaml\npython ./train.py -d 0,1 --identifier scannet config/scannet.yaml\n```\n\nThe checkpoints and logs will be written to `logs/` accordingly.\n\n**Note:** For TMM17 dataset, due to its small size the model is more senstive to the initialization. You may need to train it multiple times to reach the same performance as the pre-trained model.  For SU3, it has been reported that it is possible to achieve higher performance with 4-GPU training than the reported one in the paper, though the training process is more volatile. \n\n### Pre-trained Models\n\nYou can download our reference pre-trained models from our [Hugging Face repo](https://huggingface.co/yichaozhou/neurvps/tree/main/Pretrained). Those pre-trained models should be able to reproduce the numbers in our paper.\n\n### Evaluation\n\nExecute the following commands to compute and plot the angular accuracy (AA) curves with trained network checkpoints:\n\n```bash\npython eval.py -d 0 logs/YOUR_LOG/config.yaml logs/YOUR_LOG/checkpoint_best.pth.tar\n```\n\n### FAQ\n\n#### I want a quick tutorial on vanishing points.\n\n**A:** I have a blog post on [Vanishing Points and “Gaussian Sphere Representation”](https://yichaozhou.com/post/20190402vanishingpoint/).\n\n#### I do not understand your format of vanishing points.\n\n**A:** Uncomment [these lines](https://github.com/zhou13/neurvps/blob/edac1a85d5a4f75079da6734957e3e92927e1fcc/neurvps/datasets.py#L71-L83) or [these lines](https://github.com/zhou13/neurvps/blob/edac1a85d5a4f75079da6734957e3e92927e1fcc/neurvps/datasets.py#L125-L127) to visualize vanishing points overlaid with 2D images.\n\n#### What is the unit of focal length in the yaml and why do I need it?\n\n**A:** The focal length in our implementation is in the unit of 2/w pixel (w is the image width. only a square image is supported). This follows the convention of the OpenGL projection matrix so that to make it resolution invariant. The focal length is used for uniform sampling of the position of vanishing points. If it is not known, you can set it to some common focal length for your categories of images, as we do in [config/tmm17.yaml](https://github.com/zhou13/neurvps/blob/master/config/tmm17.yaml).\n\nYou can also check the function `to_label` and `to_pixel`, which use the focal length to convert the 3D line direction from and to a 2D vanishing point.\n\n#### I have a question. How could I get help?\n\n**A:** You can post an issue on Github, which might help other people that have the same question. You can also send me an email if you think that is more appropriate.\n\n### Acknowledgement\n\nWe thank Yikai Li from SJTU and Jiajun Wu from MIT for pointing out a bug in the data augmentation for the TMM17 Natural Scene dataset. This work is supported by a research grant from Sony Research.\n\n### Citing NeurVPS\n\nIf you find NeurVPS useful in your research, please consider citing:\n\n```\n@inproceedings{zhou2019neurvps,\n author={Zhou, Yichao and Qi, Haozhi and Huang, Jingwei and Ma, Yi},\n title={{NeurVPS}: Neural Vanishing Point Scanning via Conic Convolution},\n booktitle={{NeurIPS}},\n year={2019}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhou13%2Fneurvps","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhou13%2Fneurvps","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhou13%2Fneurvps/lists"}