{"id":13418754,"url":"https://github.com/patrikhuber/superviseddescent","last_synced_at":"2025-04-06T00:09:45.335Z","repository":{"id":25185401,"uuid":"28608742","full_name":"patrikhuber/superviseddescent","owner":"patrikhuber","description":"C++11 implementation of the supervised descent optimisation method","archived":false,"fork":false,"pushed_at":"2018-08-06T23:07:57.000Z","size":48496,"stargazers_count":403,"open_issues_count":11,"forks_count":188,"subscribers_count":43,"default_branch":"master","last_synced_at":"2025-03-29T23:09:21.623Z","etag":null,"topics":["computer-vision","landmark-detection","machine-learning","modern-cpp"],"latest_commit_sha":null,"homepage":"http://patrikhuber.github.io/superviseddescent/","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/patrikhuber.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-12-29T22:29:42.000Z","updated_at":"2025-03-12T06:12:45.000Z","dependencies_parsed_at":"2022-07-24T04:32:27.082Z","dependency_job_id":null,"html_url":"https://github.com/patrikhuber/superviseddescent","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patrikhuber%2Fsuperviseddescent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patrikhuber%2Fsuperviseddescent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patrikhuber%2Fsuperviseddescent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patrikhuber%2Fsuperviseddescent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/patrikhuber","download_url":"https://codeload.github.com/patrikhuber/superviseddescent/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247415967,"owners_count":20935387,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","landmark-detection","machine-learning","modern-cpp"],"created_at":"2024-07-30T22:01:06.620Z","updated_at":"2025-04-06T00:09:45.317Z","avatar_url":"https://github.com/patrikhuber.png","language":"C++","funding_links":[],"categories":["TODO scan for Android support in followings"],"sub_categories":[],"readme":"# superviseddescent: A C++11 implementation of the supervised descent optimisation method\n[![GitHub release](http://img.shields.io/github/release/patrikhuber/superviseddescent.svg?style=flat-square)][release]\n[![Apache License 2.0](https://img.shields.io/github/license/patrikhuber/superviseddescent.svg?style=flat-square)][license]\n\n[release]: https://github.com/patrikhuber/superviseddescent/releases\n[license]: https://github.com/patrikhuber/superviseddescent/blob/master/LICENSE\n\nsuperviseddescent is a C++11 implementation of the supervised descent method, which is a generic algorithm to perform optimisation of arbitrary functions.\n\nThe library contains an implementation of the _Robust Cascaded Regression_ facial landmark detection and features a pre-trained detection model.\n\nThere are two main advantages compared to traditional optimisation algorithms like gradient descent, L-BFGS and the like:\n* The function doesn't have to be differentiable. It works with arbitrary functions, for example with the HoG operator (which is a non-differentiable function)\n* It might be better at reaching a global optimum - give it a try!\n\nThe theory is based on the idea of _Supervised Descent Method and Its Applications to Face Alignment_, from X. Xiong \u0026 F. De la Torre, CVPR 2013 (http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=\u0026arnumber=6618919)\n\n## Features\n\n* Generic implementation, can be used to optimise arbitrary parameters and functions\n* Fast, using Eigen for matrix operations\n* Modern, clean C++11/14\n* Header only (compilation/installation only for the examples and tests)\n* Fully cross-platform compatible - learned models can be run even on phones\n\n## Usage\n\nIt is a header only library, and can thus be included directly into your project by just adding `superviseddescent/include` and `3rdparty/cereal-1.1.1/include` to your project's include directory.\n\n* Tested with the following compilers: gcc-4.8.2, clang-3.5, Visual Studio 2013 \u0026 2015 RC\n* Needed dependencies: OpenCV core (2.4.3), Eigen (3.2). For the apps, Boost (1.54.0) is also required. Older versions of the libraries might work as well.\n\n## Sample code\n\n### Robust Cascaded Regression landmark detection:\n\nWhile the method and this library can be used to approximate arbitrary functions, its prime example is facial landmark detection. The library contains an implementation of the _Robust Cascaded Regression_ landmark detection and a pre-trained model. Running it is as simple as:\n\n    rcr::detection_model rcr_model = rcr::load_detection_model(\"face_landmarks_model_rcr_22.bin\");\n    cv::Rect facebox = ...; // has to be detected by e.g. V\u0026J\n    LandmarkCollection\u003ccv::Vec2f\u003e landmarks = rcr_model.detect(image, facebox);\n\nThe full example is at [apps/rcr/rcr-detect.cpp](https://github.com/patrikhuber/superviseddescent/blob/master/apps/rcr/rcr-detect.cpp) and it is built with the library.\n\n### Simple example for generic optimisation - approximate sin(x):\n\nThe following examples describe using simple hello-world example how to use the library for generic function optimisation (for example for other computer vision tasks). We'll start with a simple function, `sin(x)`.\nDefine a function:\n\n    auto h = [](Mat value, size_t, int) { return std::sin(value.at\u003cfloat\u003e(0)); };\n\nGenerate training data (see `examples/simple_function.cpp` for the (in this case boring) code):\n* training labels `y_tr` in the interval [-1:0.2:1]\n* the inverse function values (the ground truth parameters) `x_tr`\n* fixed starting values of the parameters `x0` (a constant value in this case).\n\nConstruct and train the model, and (optionally) specify a callback function that prints the residual after each learned regressor:\n\n~~~{.cpp}\n\tvector\u003cLinearRegressor\u003e regressors(10);\n\tSupervisedDescentOptimiser\u003cLinearRegressor\u003e supervised_descent_model(regressors);\n\tauto print_residual = [\u0026x_tr](const Mat\u0026 current_predictions) {\n\t\tcout \u003c\u003c cv::norm(current_predictions, x_tr, cv::NORM_L2) / cv::norm(x_tr, cv::NORM_L2) \u003c\u003c endl;\n\t};\n\tsupervised_descent_model.train(x_tr, x0, y_tr, h, print_residual);\n~~~\t\n\nThe model can be tested on test data like so:\n~~~{.cpp}\n\tMat predictions = supervised_descent_model.test(x0_ts, y_ts, h);\n\tcout \u003c\u003c \"Test residual: \" \u003c\u003c cv::norm(predictions, x_ts_gt, cv::NORM_L2) / cv::norm(x_ts_gt, cv::NORM_L2) \u003c\u003c endl;\n~~~\n\nPredictions on new data can similarly be made with:\n~~~{.cpp}\nSupervisedDescentOptimiser::predict(Mat x0, Mat y, H h)\n~~~\nwhich returns the prediction result.\n\n\n### Pose estimation:\n\nUsing the `SupervisedDescentOptimiser` for 3D pose estimation from 2D landmarks works exactly in the same way.\n\n* `h` is the projection function that projects the 3D model to 2D coordinates, given the current parameters\n* `y` are the 2D landmarks, known (or detected) at training and testing time\n* `x` are the rotation and translation parameters (the 6 DOF of the model)\n* the initial parameters `x0` are all set to 0, with the exception of t_z being -2000.\n\n~~~{.cpp}\nclass ModelProjection\n{\npublic:\n\tModelProjection(Mat model) : model(model) {};\n\n\tMat operator()(Mat parameters, size_t regressor_level, int training_index = 0)\n\t{\n\t\t// parameters is a vector consisting of [R_x, R_y, R_z, t_x, t_y, t_z]\n\t\tprojected_points = project_model(parameters);\n\t\treturn projected_points;\n\t};\nprivate:\n\tMat model;\n}\n~~~\n\n~~~{.cpp}\nModelProjection projection(facemodel);\nsupervised_descent_model.train(x_tr, x0, y_tr, projection, print_residual);\n~~~\n\nTesting and prediction work analogous.\n\nFor the documented full working example, see `examples/landmark_detection.cpp`.\n\n### Landmark detection hello-world:\n\nIn contrast to the RCR, this is a simple-as-possible hello-world example to understand how the library works and to build your own applications.\n\nThe `SupervisedDescentOptimiser` can be used in the same way as before for landmark detection. In this case,\n\n* `h` is a feature transform that extracts image features like HOG or SIFT from the image (we thus make it a function object, to store the images)\n* we don't use the `y` values (the so called _template_) to train, because at testing, the HoG descriptors differ for each person (i.e. each persons face looks different)\n* the parameters `x` are the current 2D landmark locations\n* the initial parameters `x0` are computed by aligning the mean landmark to a detected face box.\n\n~~~{.cpp}\nclass HogTransform\n{\npublic:\n\tHogTransform(vector\u003cMat\u003e images, ...HoG parameters...) { ... };\n\t\n\tMat operator()(Mat parameters, size_t regressor_level, int training_index = 0)\n\t{\n\t\t// shortened, to get the idea across:\n\t\tMat hog_descriptors = extract_HOG_features(images[training_index], parameters);\n\t\treturn hog_descriptors;\n\t}\nprivate:\n\tvector\u003cMat\u003e images;\n}\n~~~\n\nTraining the model is the same, except we pass an empty `cv::Mat` instead of `y` values:\n~~~{.cpp}\nHogTransform hog(training_images, ...HoG parameters...);\nsupervised_descent_model.train(training_landmarks, x0, Mat(), hog, print_residual);\n~~~\n\nTesting and prediction work analogous.\n\nFor the documented full working example, see `examples/landmark_detection.cpp`.\n\n## Build the examples and tests\n\nBuilding of the examples and tests requires CMake\u003e=2.8.11, OpenCV (core, imgproc, highgui, objdetect) and Boost (system, filesystem, program_options).\n\n* copy `initial_cache.cmake.template` to `initial_cache.cmake`, edit the necessary paths\n* create a build directory next to the `superviseddescent` folder: `mkdir build; cd build`\n* `cmake -C ../superviseddescent/initial_cache.cmake -G \"\u003cyour favourite generator\u003e\" ../superviseddescent -DCMAKE_INSTALL_PREFIX=install/`\n* build using your favourite tools, e.g. `make; make install` or open the solution in Visual Studio.\n\n\n## Documentation\n\nDoxygen: http://patrikhuber.github.io/superviseddescent/doc/\n\nThe [examples](https://github.com/patrikhuber/superviseddescent/tree/master/examples) and the [Class List](http://patrikhuber.github.io/superviseddescent/doc/annotated.html) in doxygen are a good place to start.\n\n## License \u0026 contributions\n\nThis code is licensed under the Apache License, Version 2.0\n\nContributions are very welcome! (best in the form of pull requests.) Please use Github issues for any bug reports, ideas, and discussions.\n\nIf you use this code in your own work, please cite the following paper: _Fitting 3D Morphable Models using Local Features_, P. Huber, Z. Feng, W. Christmas, J. Kittler, M. Rätsch, IEEE International Conference on Image Processing (ICIP) 2015, Québec City, Canada (http://arxiv.org/abs/1503.02330).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpatrikhuber%2Fsuperviseddescent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpatrikhuber%2Fsuperviseddescent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpatrikhuber%2Fsuperviseddescent/lists"}