{"id":13716682,"url":"https://github.com/puzzlepaint/camera_calibration","last_synced_at":"2025-05-07T06:30:51.975Z","repository":{"id":43038844,"uuid":"226195096","full_name":"puzzlepaint/camera_calibration","owner":"puzzlepaint","description":"Accurate geometric camera calibration with generic camera models","archived":false,"fork":false,"pushed_at":"2024-07-01T12:09:02.000Z","size":7834,"stargazers_count":715,"open_issues_count":34,"forks_count":119,"subscribers_count":29,"default_branch":"master","last_synced_at":"2024-11-14T04:34:58.238Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/puzzlepaint.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-05T21:50:32.000Z","updated_at":"2024-11-12T18:19:15.000Z","dependencies_parsed_at":"2024-01-05T23:52:04.837Z","dependency_job_id":"1994ff16-b2a8-4ada-bbb1-13e8403b7307","html_url":"https://github.com/puzzlepaint/camera_calibration","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puzzlepaint%2Fcamera_calibration","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puzzlepaint%2Fcamera_calibration/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puzzlepaint%2Fcamera_calibration/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puzzlepaint%2Fcamera_calibration/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/puzzlepaint","download_url":"https://codeload.github.com/puzzlepaint/camera_calibration/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252826657,"owners_count":21810161,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T00:01:13.291Z","updated_at":"2025-05-07T06:30:49.291Z","avatar_url":"https://github.com/puzzlepaint.png","language":"C++","funding_links":[],"categories":["8. Tutorials","Awesome SLAM Tools","\u003ca name=\"cpp\"\u003e\u003c/a\u003eC++","Calibration software"],"sub_categories":["8.5 Calibration","Calibration Tools"],"readme":"## Accurate geometric camera calibration ##\n\n* [Overview](#overview)\n* [About](#about)\n* [Building](#building)\n* [How to use](#how-to-use)\n  * [Obtaining a calibration pattern](#obtaining-a-calibration-pattern)\n  * [Calibrating a camera with live input](#calibrating-a-camera-with-live-input)\n  * [Calibrating a camera from images in a folder](#calibrating-a-camera-from-images-in-a-folder)\n  * [Calibrating a stereo camera and computing depth images](#calibrating-a-stereo-camera-and-computing-depth-images)\n  * [Which camera model to choose?](#which-camera-model-to-choose)\n  * [How to obtain and verify good calibration results](#how-to-obtain-and-verify-good-calibration-results)\n  * [How to use generic camera models in your application](#how-to-use-generic-camera-models-in-your-application)\n  * [Reference on calibration report visualizations](#reference-on-calibration-report-visualizations)\n\n\n\n## Overview ##\n\nThis repository contains a tool for **accurate geometric camera calibration**,\ni.e., establishing a mapping between image pixels and the pixels' 3D observation\ndirections respectively lines. In particular, it supports calibration with\n**generic** camera models, which fit nearly every camera and allow for highly\naccurate calibration. The tool also includes support to calibrate\n**fixed camera rigs** and additionally supports estimating\n**accurate depth images for stereo cameras** such as the Intel D435 or the\nOccipital Structure Core.\n\nThe requirements on the camera are:\n\n* The camera must be near-central, i.e., all observation lines must approximately pass\n  through the same 3D point. This is because a central camera model is used for initialization.\n  There is support for a fully non-central model, but not for initializing directly with it\n  (however, re-implementations of Ramalingam and Sturm's initialization methods for\n  all combinations of central/non-central cameras and planar/non-planar calibrations patterns\n  are available in [applications/camera_calibration/src/camera_calibration/relative_pose_initialization](https://github.com/puzzlepaint/camera_calibration/tree/master/applications/camera_calibration/src/camera_calibration/relative_pose_initialization)).\n* The observation directions / lines must vary smoothly. For example,\n  there should not be a mirror within the field-of-view of the camera that ends\n  abruptly. This is because observation directions / lines are stored sparsely\n  and are interpolated smoothly.\n\nFor depth estimation and live feature detection, a CUDA-capable graphics card is required.\n\nThe application has been tested on Ubuntu Linux only.\n\n\n\n## About ##\n\nThis repository contains the\n[Camera calibration application](https://github.com/puzzlepaint/camera_calibration/tree/master/applications/camera_calibration)\nand the library it is based on,\n[libvis](https://github.com/puzzlepaint/camera_calibration/tree/master/libvis).\nThe library is work-in-progress and it is not recommended to use it for other projects at this point.\n\nThe application and library code is licensed under the BSD license, but please\nalso notice the licenses of the included or externally used third-party components.\n\nIf you use the provided code for research, please cite the paper describing the approach:\n\n[Thomas Schöps, Viktor Larsson, Marc Pollefeys, Torsten Sattler, \"Why Having 10,000 Parameters in Your Camera Model is Better Than Twelve\", arXiv 2019.](https://arxiv.org/abs/1912.02908)\n\n\n\n## Building ##\n\nBuilding has been tested on Ubuntu 14.04 and Ubuntu 18.04 (with gcc).\n\nThe following external dependencies are required.\n\n| Dependency   | Version(s) known to work |\n| ------------ | ------------------------ |\n| [Boost](https://www.boost.org/) | 1.54.0 |\n| [CUDA](https://developer.nvidia.com/cuda-downloads) | 10.1 |\n| [Eigen](http://eigen.tuxfamily.org/index.php?title=Main_Page) | 3.3.7 |\n| [GLEW](http://glew.sourceforge.net/build.html) | 1.10.0 |\n| [OpenGV](https://github.com/laurentkneip/opengv) | Commit 306a54e6c6b94e2048f820cdf77ef5281d4b48ad |\n| [Qt](https://www.qt.io/) | 5.12.0; minimum version: 5.8 |\n| [SuiteSparse](http://faculty.cse.tamu.edu/davis/suitesparse.html) | 4.2.1 |\n| [zlib](https://zlib.net/) | - |\n\nThe following external dependencies are optional.\n\n| Dependency   | Purpose |\n| ------------ | ------- | \n| [librealsense2](https://github.com/IntelRealSense/librealsense) | Live input from RealSense D400 series depth cameras (tested with the D435 only). |\n| [Structure SDK](https://structure.io/developers) | Live input from Structure Core cameras (tested with the color version only). To use this, set the SCSDK_ROOT CMake variable to the SDK path. |\n\nAfter obtaining all dependencies, the application can be built with CMake, for example as follows:\n\n```bash\nmkdir build\ncd build\ncmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_CUDA_FLAGS=\"-arch=sm_61\" ..\nmake -j camera_calibration  # Reduce the number of threads if running out of memory, e.g., -j3\n```\n\nIf you intend to use the depth estimation or live feature detection functionalities,\nmake sure to specify suitable CUDA architecture(s) in CMAKE_CUDA_FLAGS.\nCommon settings would either be the CUDA architecture of your graphics card only (in case\nyou only intend to run the compiled application on the system it was compiled on), or a range of virtual\narchitectures (in case the compiled application is intended for distribution).\nSee the [corresponding CUDA documentation](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#options-for-steering-gpu-code-generation-gpu-architecture).\n\n\n\n## How to use ##\n\n### Obtaining a calibration pattern ###\n\nThis is a prerequisite for calibration.\n\nThe first step is to choose a suitable pattern. Ideally, the density of features\non the pattern is chosen to be appropriate for the resolution of the camera to\nbe calibrated. For example, a high-resolution camera can observe many features\nat the same time, so a high feature density helps in quickly obtaining enough\ncalibration data. However, this pattern may not be well-suited for a low-resolution\ncamera, which cannot sharply observe all features at the same time. It should\nalso be considered that high numbers of features (either due to high density, or\ndue to using multiple patterns at the same time) significantly increase the time\nrequired to perform the calibration.\n\nSome readily usable patterns with different feature densities, generated for\nDIN A4 sized paper, are included in the\n[patterns](https://github.com/puzzlepaint/camera_calibration/tree/master/applications/camera_calibration/patterns) folder.\nEach pattern consists of a PDF file for display and a YAML file that describes the pattern content.\nThe YAML file later needs to be passed to the camera calibration program such that it can detect\nthe corresponding pattern.\n\nIf the provided patterns are not sufficient, you can\ngenerate additional patterns with the pattern generation script [scripts/create_calibration_pattern.py](https://github.com/puzzlepaint/camera_calibration/tree/master/applications/camera_calibration/scripts/create_calibration_pattern.py). The script uses [ReportLab](https://www.reportlab.com/) to generate the PDF file,\nwhich may be installed like: `sudo pip[3] install reportlab`. It also depends on\nnumpy. Call the script as follows to see its usage:\n`python[3] create_calibration_pattern.py -h`. Only the `--tag36h11_path` and\n`--output_base_path` arguments are mandatory.\n\nAfter deciding for one or multiple patterns, the second step is to choose how to\npresent the pattern(s) to the camera:\n\n* One way is to **print** the pattern(s). It is possible to use multiple printed patterns\n  at the same time. This may for example help to fully calibrate fisheye cameras\n  with a very large field-of-view, since the pattern geometry is not limited to\n  a plane then. In this case, make sure that each printed pattern uses a unique\n  AprilTag index. Note that the final pattern(s) must be rigid, fixed wrt. each\n  other, and each individual pattern should be near-planar.\n  Planarity is assumed for initialization purposes, but not for later stages in\n  the calibration. Thus, as long as initialization works, the final accuracy is not\n  negatively affected by non-planar patterns in any way.\n* Another way is to **show the pattern on a display** such as a computer monitor.\n  The application includes direct support to do this, while also showing the\n  feature coverage while not recording images. See below for how to use this.\n  This way, only a single pattern can be used at a time. If multiple patterns\n  are given, the current pattern can be changed with the arrow keys.\n\n\n\n### Calibrating a camera with live input ###\n\nLive input has the advantage that the coverage of the camera view with feature\ndetections is shown in real-time during recording, showing where additional data\nis still needed. However, this is only possible for cameras for which live\nsupport has been implemented. Currently, there is support for Intel RealSense\ncameras via librealsense2, for Occipital Structure Core cameras via the Structure\nSDK, and for many other kinds of cameras with video4linux2.\n\nTo use this mode of operation, start the application without arguments:\n\n```bash\n/path/to/camera_calibration/build/applications/camera_calibration/camera_calibration\n```\n\nThis will show a window that might look like this with a webcam and an Intel RealSense D435 camera attached:\n\n![Settings Window](applications/camera_calibration/doc/settings_window.png?raw=true)\n\nAt the top, all attached and detected cameras are listed. They are prefixed by the\nlibrary that they are detected with. A single camera may be detected by multiple\nlibraries; for example, here the three cameras on the D435 device were detected\nby librealsense and by video4linux2 (but in this case, they will only work with\nlibrealsense).\n\nIn this list, check the boxes for all cameras that should be used at the same\ntime. Note that at present, it is only possible to check multiple \"librealsense\"\ncameras or multiple \"Structure SDK\" cameras at the same time, but no other\ncameras or cameras used with different libraries.\n\nThe \"Live feature detection\" box should remain checked to give a live image of\nthe image coverage with feature detections. It should be unchecked if no\nCUDA-capable graphics card is available, or if recording data for other purposes.\n\nIn the text field above this box, the paths to the pattern YAML files that will\nbe used must be entered. If the mode which shows the pattern on screen will be used later,\nthis pattern must also be selected here.\n\nThe feature window extent should be set to suit the specific camera(s) used. It\nis recommended to shortly try out a few different values and choose the value\nwhich gives the most reliable feature detections. Common values are for example\n10, 15, and 20.\n\nSaving the recorded images is helpful in case you cannot run real-time feature\ndetection, or if you potentially want to process the images\nagain later with other settings. If you do not want to save the images, the\ncorresponding checkbox can be un-ticked.\n\nFor saving the recorded images, and a dataset file containing the features extracted in\nreal-time, specify a directory to save the dataset and images in at the bottom.\n\nFrom here on, there are two ways to start live operation:\n\n* Click \"Show pattern\" to display the selected patterns in fullscreen mode on\n  the computer screen. Note that in this mode, no automatic image recording\n  or live detection is done. Instead, an image is recorded and features are detected in it when pressing the\n  Space key. The live camera view is displayed while no image is recorded,\n  which is hidden while recording an image.\n  Note that for using this mode, you must start the application with the `--apriltags_directory`\n  parameter, specifying the path to a directory containing the \"tag36h11\" AprilTag\n  images. Those can be downloaded from [the corresponding repository](https://github.com/AprilRobotics/apriltag-imgs).\n* Click \"Start normal\" to use printed or otherwise externally shown patterns.\n  This will show a window which only shows the live images and the feature\n  coverage.\n\nTo end recording, simply close the recording window (use Escape or Alt+F4 in case of the\nfullscreen pattern display).\n\nRecording with live feature detection yields a file `dataset.bin` that can be further processed to calibrate\nthe camera as described in the second step of the section below. If only recording\nimages, proceed as described from the start of the section below.\n\n\n\n### Calibrating a camera from images in a folder ###\n\nThis mode of operation may be used for cameras for which live input is not possible,\nor after recording images live as described above.\n\n#### Feature extraction ####\n\nTo extract features and create a dataset file, the camera calibration program\ncan be first called as follows, for example. This assumes that the images have\nbeen placed in a folder `${DATASET}/images`.\n\n```bash\nexport CALIBRATION_PATH=/path/to/camera_calibration_root_folder\nexport DATASET=/path/to/dataset_folder\nexport HALF_WINDOW_SIZE=15  # Adjust to what gives the most detections for your camera, e.g., 10, 15, or 20\n${CALIBRATION_PATH}/build/applications/camera_calibration/camera_calibration \\\n    --pattern_files ${CALIBRATION_PATH}/applications/camera_calibration/patterns/pattern_resolution_17x24_segments_16_apriltag_0.yaml \\\n    --image_directories ${DATASET}/images \\\n    --dataset_output_path ${DATASET}/features_${HALF_WINDOW_SIZE}px.bin \\\n    --refinement_window_half_extent ${HALF_WINDOW_SIZE} \\\n    --show_visualizations  # optional for showing visualizations\n#   --no_cuda_feature_detection  # use this to disable using CUDA for feature detection\n```\n\n`--pattern_files` must be a comma-separated list of paths to YAML files\ndescribing the calibration pattern(s) used. `--image_directories` specifies the\npath to the directory containing the images. If calibrating a camera rig, multiple\ncomma-separated folders must be given. Images in different folders that have the\nsame file name are assumed to be recorded at the same time. `--dataset_output_path` gives the\npath to a file that will be created to store the extracted features. If you use\n`--show_visualizations`, the visualization window will remain open once the process has finished and\nneeds to be closed manually.\n\n#### Camera calibration ####\n\nAs a second step, the camera calibration program can be called to perform the\nactual calibration based on the extracted features, for example as follows\n(using the definitions from above):\n\n```bash\nexport CELL_SIZE=50  # Choose a suitable value for the camera's resolution\n${CALIBRATION_PATH}/build/applications/camera_calibration/camera_calibration \\\n    --dataset_files ${DATASET}/features_${HALF_WINDOW_SIZE}px.bin \\\n    --output_directory ${DATASET}/result_${HALF_WINDOW_SIZE}px_noncentral_generic_${CELL_SIZE} \\\n    --cell_length_in_pixels ${CELL_SIZE} \\\n    --model noncentral_generic \\\n    --num_pyramid_levels 4 \\\n    --show_visualizations  # optional for showing visualizations\n```\n\n`--dataset_files` must point to the dataset file with the extracted features.\nThe computed calibration files will be saved in the folder given with `--output_directory`.\n`--cell_length_in_pixels` specifies the desired cell length for generic camera models;\nsee below. The camera model to use must be given with `--model`. For generic\ncamera models, it can be helpful to use a multi-resolution pyramid during\ncalibration for better convergence. The number of pyramid levels can be given\nwith `--num_pyramid_levels`. Note that re-sampling for the `noncentral_generic`\nmodel is implemented in a somewhat inaccurate way, however. If you use\n`--show_visualizations`, the visualization window will remain open once the\nprocess has finished and needs to be closed manually.\n\nThe available camera models are as follows. See the corresponding section below for\nrecommendations on which model to choose.\n\n```\ncentral_generic\ncentral_thin_prism_fisheye\ncentral_opencv\ncentral_radial\nnoncentral_generic\n```\n\nFor generic camera models, a grid resolution respectively cell size must be chosen.\nCalibrated 3D observation directions or lines are stored at the corners of the resulting grid\nand are interpolated over the grid cells. Note that the given cell size is not used\ndirectly; rather, the closest cell size is chosen that yields an integer number of\ncells over the calibrated image area.\n\nThe grid resolution should be chosen to be appropriate for the camera's resolution.\nFor example, for a camera of resolution 2000x1000 pixels, a cell length of 40 might\nbe appropriate, while for a camera of resolution 640x480 pixels, a cell length of 10\nmight be appropriate. The points to consider are:\n\n* Denser grids may better model small details, improving the calibration.\n* To properly constrain the camera model parameters, multiple feature observations\n  should be recorded within each grid cell. I.e., with a denser grid, denser\n  feature observations are required to properly constrain the model and avoid\n  overfitting.\n* Denser grids increase the time required to calibrate the model. However, the\n  runtime performance impact when using the calibrated model should be negligible.\n\nThe output files contain some \"report\" files that allow to judge the quality of\nthe resulting calibration. See the section \"How to obtain good calibration results\"\nbelow.\n\n#### Refining existing calibrations ####\n\nIt is also possible to take an existing calibration and refine it, possibly after\nre-sampling to a different camera model. To do this, run the calibration program\nas specified above, but also give the directory in which the existing calibration\nis saved in with the `--state_directory` parameter. Note that re-sampling camera\nmodels is only implemented between different central models, from a central model\nto the non-central model, and (approximately) from the non-central model to a\ndifferent grid resolution, but not from the non-central model to a central model.\nFor example, for near-central cameras, this allows to calibrate the camera with\na central model first and then use the non-central model as last refinement step.\n\n#### Handling large datasets and many variables ####\n\nThe application computes the Schur complement during bundle adjustment while\nsolving for state updates. By default, it will fully store the off-diagonal\npart of the Hessian matrix in memory for its computation, which may become huge\nif there are many images and thus many pose variables to be optimized, as well\nas many intrinsics variables to be optimized. This may be very slow and/or\nexceed the available memory. To better handle such cases, the program allows to\nchange this behavior by specifying the `--schur_mode` parameter. It supports the\nfollowing options:\n\n* **dense**: This is the default behavior. Use this if you have sufficient\n  memory and cannot use the CUDA variant (or the latter does not help).\n* **dense_cuda**: Performs a large matrix multiplication during computation of\n  the Schur complement on the GPU with CUDA. This may improve the performance,\n  but does not reduce memory use.\n* **dense_onthefly**: Stores only a few rows of the off-diagonal part of the\n  Hessian at each time. The rows are stored densely. This requires more passes\n  over the residuals than the default option and is slower than it, but saves\n  memory.\n* **sparse**: Stores the off-diagonal part of the Hessian sparsely (but keeps\n  it in memory completely). This may be faster than the default if the matrix\n  is very sparse, and may potentially save some memory.\n* **sparse_onthefly**: Stores only a few rows of the off-diagonal part of the\n  Hessian at each time. The rows are stored sparsely. This requires more passes\n  over the residuals than the default option, but saves memory. It might be\n  well-suited if the matrix is very sparse.\n\nYou may need to try out which option works best for your case. If you do not\nrun into any issues with memory or performance, you may simply leave this\noption at its default.\n\n\n\n### Calibrating a stereo camera and computing depth images ###\n\nThis requires a fixed configuration of two cameras whose fields of view overlap.\nFor example, this is well-suited to calibrate active stereo cameras such as the\nIntel D435 or the Occipital Structure Core. However, it is also possible to\nput two arbitrary individual cameras next to each other to make a stereo rig.\nNote that this configuration needs to remain completely fixed though for the calibration\nto remain valid, and both cameras are supposed to take images at exactly the same\ntime; alternatively, the scene must be static, such that different recording times do\nnot matter.\n\nAlso note that at the moment, this supports only a single camera model at a time,\ndepending on which model the CUDA kernel for stereo depth estimation is compiled with.\nSee [libvis/src/libvis/cuda/pixel_corner_projector.cuh](https://github.com/puzzlepaint/camera_calibration/tree/master/libvis/src/libvis/cuda/pixel_corner_projector.cuh).\nBy default, it is the central-generic camera model.\n\nAnother limitation of the implementation (that should be trivial to fix if required)\nis that the calibration must have been made with exactly the two cameras that\nwill be used for stereo depth estimation (and no additional ones).\n\nIf using an active stereo camera, the active projection should be disabled for\ncalibration. The librealsense integration can do this if using a RealSense\ncamera for live input. For other cameras, the projector needs to be covered to\nblock the light.\n\nCalibration otherwise works as described in the sections above, either with live\ncamera input or based on recorded images.\n\nFor depth estimation, stereo images with the active projection turned on should\nbe recorded. Depth maps can then be computed for example as follows:\n\n```bash\nexport CALIBRATION_PATH=/path/to/camera_calibration\nexport CALIBRATION_RESULT=/path/to/calibration/result/folder\nexport STEREO_DATASET=/path/to/input/image/dataset\nexport IMAGE=image_filename_without_png\n${CALIBRATION_PATH}/build/applications/camera_calibration/camera_calibration \\\n    --stereo_depth_estimation \\\n    --state_directory ${CALIBRATION_RESULT} \\\n    --images ${STEREO_DATASET}/images0/${IMAGE}.png,${STEREO_DATASET}/images1/${IMAGE}.png \\\n    --output_directory ${STEREO_DATASET}/stereo_${IMAGE}\n```\n\nThis assumes that the stereo images have been recorded with the camera_calibration\nprogram, which places the images of the two cameras in the `images0` and `images1` folders.\n\nNote that the stereo depth estimation implementation has not at all been optimized\nand may thus take a very long time to compute.\n\n\n\n### Which camera model to choose? ###\n\nFor best results, choose one of the following models:\n\n* `central_generic` for assuming a central camera (all observation rays go through a single point), or\n* `noncentral_generic` for general non-central cameras.\n\nUsually, `noncentral_generic` is slightly more accurate than `central_generic`,\neven for near-central cameras. In general, it should always be at least as\naccurate as `central_generic`, unless a lack of data leads to overfitting.\n\nHowever, one should be aware of the implications: With a non-central camera model,\nimages in general cannot be undistorted to pinhole images (without knowing the\nscene geometry), and algorithms developed for central cameras might require adaptation.\nFor this reason, using a central camera model might be more convenient, even if\nbeing a little less accurate.\n\n\n\n### How to obtain and verify good calibration results ###\n\nSome tips to follow for getting good calibration results are:\n\n* Show the calibration pattern to the camera(s) from all angles and from different distances.\n  The calibration will fail if the pattern is only visible from straight above.\n  Showing the effect of perspective is necessary to calibrate the camera field-of-view.\n* Cover the whole camera image with feature detections. In particular, focus on the image corners,\n  as it is hardest to get sufficient detections there. If using a generic camera model,\n  each grid cell should contain several feature detections.\n* If using a rolling shutter camera, use a tripod and only take images from fixed\n  poses (not hand-held) to avoid introducing any rolling shutter distortions.\n\nAfter computing a calibration, the report files within the output directory\nallow judging the calibration quality.\n\n* In `report_cameraX_info.txt`, `reprojection_error_median` should usually be\n  significantly smaller than 0.1 pixels.\n* `report_cameraX_errors_histogram.png` should be a (more or less) small white\n  and round(-ish) dot in the center of the image, such as:\n  \n  ![Error histogram good example](applications/camera_calibration/doc/report_cameraX_errors_histogram_good_example_1.png?raw=true)\n  \n  This is another good example with a larger dot:\n  \n  ![Error histogram good example](applications/camera_calibration/doc/report_cameraX_errors_histogram_good_example_2.png?raw=true)\n  \n  If the dot is not round or is not centered, something is definitely wrong. Example:\n  \n  ![Error histogram good example](applications/camera_calibration/doc/report_cameraX_errors_histogram_bad_example_1.png?raw=true)\n  \n  Another bad example:\n  \n  ![Error histogram good example](applications/camera_calibration/doc/report_cameraX_errors_histogram_bad_example_2.png?raw=true)\n  \n  A possible reason for such failure cases is that the bundle adjustment is not converged yet and needs more iterations.\n  It could also be that the selected camera model does not fit to the camera at all, but that should be unlikely to happen if using a suitable generic model.\n* Most cells in `report_cameraX_error_magnitudes.png` should be more or less\n  green (if using outlier removal). If there are red points forming some systematic\n  pattern, something is probably wrong.\n* `report_camera0_error_directions.png` should show random colors. If there is\n  a systematic pattern, then the calibrated model does not fit the data tightly\n  (this is bound to happen if using parametric camera models!).\n  Good example (generic camera model):\n  \n  ![Error histogram good example](applications/camera_calibration/doc/report_cameraX_error_directions_good_example.jpg?raw=true)\n  \n  Bad example (parametric camera model):\n  \n  ![Error histogram good example](applications/camera_calibration/doc/report_cameraX_error_directions_bad_example_1.jpg?raw=true)\n  \n  Note that all kinds of different systematic patterns can show up here. Also,\n  even in good calibrations, weak patterns may remain. Another failure mode is\n  missing projections in areas where features were detected. This shows up as\n  areas of large Voronoi cells; example in the top-left corner:\n  \n  ![Error histogram good example](applications/camera_calibration/doc/report_cameraX_error_directions_bad_example_2.jpg?raw=true)\n  \n  This kind of failure probably means that the optimization process\n  used for projection does not find the optimum for points that project to this\n  area. This may be caused by unusual camera geometry, or by having too much noise\n  in the calibration, possibly caused by not enough feature detections.\n  For the example above, we can confirm the case of a noisy calibration by\n  zooming in on the top-left corner of the observation direction visualization:\n  \n  ![Error histogram good example](applications/camera_calibration/doc/noisy_calibration_example.png?raw=true)\n  \n  Here, the faint blue rims change their direction at the top left corner (zoom into the image to see this better).\n  This creates a \"trap\" for the optimization from which it cannot escape,\n  causing points near this corner to fail projection (since the optimization\n  would first overshoot and then go back, but it fails to go back in this case).\n\n\n\n### How to use generic camera models in your application ###\n\nAfter successful calibration, the calibrated intrinsic camera parameters are\nstored in the files `intrinsicsX.yaml` in the output folder.\n\nIn the [applications/camera_calibration/generic_models](https://github.com/puzzlepaint/camera_calibration/tree/master/applications/camera_calibration/generic_models) folder,\nthere are implementations for the central-generic and non-central generic camera\nmodels which can load these intrinsics YAML files. This should make it easy\nto use these camera models in other applications. These implementations support\n3D point projection to the image, pixel un-projection to a 3D direction respectively\nline, and computing Jacobians for the above operations with respect to the input\npoint or pixel.\n\nThese camera model implementations use the [Eigen](http://eigen.tuxfamily.org/index.php?title=Main_Page) library as a single dependency.\nEven this dependency should be easy to remove if desired, since only its\nmatrix and vector classes are used, but no advanced functionality that would be\nhard to substitute. See the [main file](https://github.com/puzzlepaint/camera_calibration/tree/master/applications/camera_calibration/generic_models/src/main.cc)\nof this implementation for some unit tests, which show by example how to use the\ncamera model classes. The camera models are also documented with Doxygen comments.\nHowever, note that these implementations have not been optimized; depending on the\napplication, it could be sensible to use different kinds of lookup tables to\nspeed up the operations.\n\nNote that the calibration program will not calibrate the whole image area, but\nonly the bounding rectangle of all feature detections. Due to the local window\nsize for feature refinement, features are not detected directly next to the image\nborders. If it was crucial to calibrate the whole image area, it would for example\nbe possible to extrapolate the calibration, or to tolerate some overlap of the feature\nrefinement window with regions outside of the image.\n\n\n\n### Reference on calibration report visualizations ###\n\n* `report_cameraX_error_directions.png`: Each pixel in this image is colored\n  according to the *direction* (disregarding the magnitude) of the reprojection\n  error of the closest residual (over all images used for calibration). This\n  allows judging whether there are any systematic patterns in the residual\n  error directions, even very small ones. This visualization is a Voronoi\n  diagram. It also allows judging whether there are too few feature detections\n  in some part of the image; those cause large Voronoi cells.\n* `report_cameraX_error_magnitudes.png`: Each pixel in this image is colored\n  according to the *magnitude* (disregarding the direction) of the reprojection\n  error of the closest residual (over all images used for calibration). Low\n  errors are colored green, high errors are colored red.\n* `report_cameraX_errors_histogram.png`: Shows a 2D histogram of all reprojection\n  errors. Allows judging whether the residual distribution is as expected (dot-shaped).\n* `report_cameraX_grid_point_locations.png`: Shows the locations of the grid points for\n  generic camera models that use a grid for interpolation.\n* `report_cameraX_line_offsets.png`: For the non-central-generic model, this image\n  visualizes the positions of the observation lines as follows: First, a 3D point\n  is determined which is as close as possible to all observation lines. For a\n  central camera, this would be the projection center. Then, for each pixel, the\n  closest point on the pixel's observation line to this 3D point is determined.\n  The 3D offset between those two points is directly translated into an RGB color\n  for this pixel in the visualization. This visualization allows judging whether\n  the lines follow some clear pattern (which suggests that the camera is\n  significantly non-central), or appear more or less random (which suggests that\n  the camera is mostly central). Note that the automatic scaling will usually\n  cause almost all areas of this visualization to be gray (since the extrema\n  will usually be only in small parts of the image). The contrast can be changed\n  with an image editing program such as GIMP to see the remaining structure.\n* `report_cameraX_observation_directions.png`: Visualizes the calibrated observation directions.\n  Each 3D direction is directly mapped to an RGB color in the visualization.\n  More structure is shown for the z direction, since by convention, this is\n  calibrated to be the 'forward' direction for each camera, and too little\n  structure might be visible if treating it the same as the other two dimensions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpuzzlepaint%2Fcamera_calibration","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpuzzlepaint%2Fcamera_calibration","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpuzzlepaint%2Fcamera_calibration/lists"}