{"id":17222949,"url":"https://github.com/cheind/dest","last_synced_at":"2025-06-19T00:03:07.714Z","repository":{"id":66174966,"uuid":"48329193","full_name":"cheind/dest","owner":"cheind","description":":panda_face: One Millisecond Deformable Shape Tracking Library (DEST)","archived":false,"fork":false,"pushed_at":"2016-12-29T16:10:36.000Z","size":1659,"stargazers_count":291,"open_issues_count":4,"forks_count":108,"subscribers_count":27,"default_branch":"master","last_synced_at":"2025-05-20T09:06:25.136Z","etag":null,"topics":["face-alignment","face-detector","machine-learning","vision"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cheind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2015-12-20T16:44:36.000Z","updated_at":"2025-04-03T14:48:58.000Z","dependencies_parsed_at":"2023-02-21T01:30:20.432Z","dependency_job_id":null,"html_url":"https://github.com/cheind/dest","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/cheind/dest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cheind%2Fdest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cheind%2Fdest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cheind%2Fdest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cheind%2Fdest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cheind","download_url":"https://codeload.github.com/cheind/dest/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cheind%2Fdest/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260654616,"owners_count":23042670,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["face-alignment","face-detector","machine-learning","vision"],"created_at":"2024-10-15T04:06:48.222Z","updated_at":"2025-06-19T00:03:02.690Z","avatar_url":"https://github.com/cheind.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"## About this library\n\n**Deformable Shape Tracking (DEST)** is a C++ library providing high performance 2D shape tracking leveraging\nmachine learning methods. The video below shows the real-time capabilities of **DEST** in annotating video sequences\n / still images with facial landmarks.\n\n[![Watch on Youtube](http://img.youtube.com/vi/Hewjc0oyqPQ/0.jpg)](https://youtu.be/Hewjc0oyqPQ)\n\nThis **DEST** tracker was previously trained on thousands of training samples from available face databases.\n\n**DEST** features\n - A generic framework for learning arbitrary shape transformations.\n - A lightning fast landmark alignment module.\n - State of the art performance and accuracy.\n - Pre-trained trackers for a quick start.\n - Cross platform minimal disk footprint serialization.\n - Built in support for [IMM](http://www.imm.dtu.dk/~aam/datasets/datasets.html) and [ibug](http://ibug.doc.ic.ac.uk/resources/facial-point-annotations/) annotated face database import.\n\n## Using DEST\n\nUsing involves the following steps. First include **DEST**\n\n```cpp\n#include \u003cdest/dest.h\u003e\n```\n\nNext, load a trained tracker from disk\n\n```cpp\ndest::core::Tracker t;\nt.load(\"destcv.bin\");\n```\n\nNote that each [release](https://github.com/cheind/dest/releases) contains pre-trained tracker files. Assuming that our goal is to align face landmarks, we also need a face detector to provide a coarse estimate (rectangle) of the face area. **DEST** includes a convenience wrapper for OpenCV based face detection\n\n```cpp\n#include \u003cdest/face/face_detector.h\u003e\n\n//...\n\ndest::face::FaceDetector fd;\nfd.loadClassifiers(\"classifier_frontalface.xml\");\n```\n\nOpenCV uses Viola Jones algorithm for face detection. This algorithm requires a training phase. You can find application ready files in OpenCV or [here](etc/cv/). Use the face detector to find a face in the given image.\n\n```cpp\ndest::core::Rect r;\nfd.detectSingleFace(img, r);\n```\n\nHere `img` is either `dest::core::Image` or `cv::Mat`. Once we have a rough estimate of the face location, we need to find a shape normalizing transform. By default the following is used\n\n```cpp\ndest::core::Rect ur = dest::core::unitRectangle();\ndest::core::ShapeTransform shapeToImage;\n\nshapeToImage = dest::core::estimateSimilarityTransform(ur, r);\n```\n\nFinally, invoke the tracker to get the face landmarks\n\n```cpp\ndest::core::Shape s = t.predict(img, shapeToImage);\n```\n\nThe shape `s` contains the landmark locations in columns (x,y) for the given image. The number of landmarks depends on the data used during training.\n\nNote, you need to use same shape normalization procedure during tracking as in training. This also holds true for the way rough\nestimates (face detector in this example) are generated.\n\n## Building from source\n**DEST** requires the following pre-requisites\n\n - [CMake](www.cmake.org) - for generating cross platform build files\n - [Eigen 3.x](http://eigen.tuxfamily.org) - for linear algebra calculations\n\nOptionally, you need\n\n - [OpenCV 2.x / 3.x](www.opencv.org) - for image processing related functions\n - A compiler with [OpenMP](https://en.wikipedia.org/wiki/OpenMP) capabilities.  \n\nTo build follow these steps\n\n  1. Fork or download a [release](https://github.com/cheind/dest/releases) of this repository. We recommend releases as those include pre-trained trackers.\n  1. Point CMake to the source directory.\n  1. Click CMake Configure and select your toolchain.\n  1. Specify `DEST_EIGEN_DIR`.\n  1. Select `DEST_WITH_OPENCV` if required. When selected you will be asked to specify `OpenCV_DIR` next time you run Configure. Set OpenCV_DIR to the directory containing the file `OpenCVConfig.cmake`.\n  1. Select `DEST_WITH_OPENMP` if required.\n  1. Select `DEST_VERBOSE` if verbose logging is required.\n  1. Click CMake Generate.\n  1. Open generated solution and build `ALL_BUILD`.\n\n#### When is OpenCV is required?\nOpenCV is required during training and when running the demo samples. **DEST** comes with its own Eigen based image type, OpenCV is mainly used for convenience functions such as image loading and rendering.\n\n#### Any other dependencies?\nYes, those are inline [included](ext/) and are header only. **DEST** makes use of Google flatbuffers for serialization, tinydir for enumerating files and TCLAP for command line parsing.\n\n#### Supported platforms\nAlthough **Deformable Shape Tracking** should build across multiple platforms and architectures, tests are carried out on these systems\n - Windows 8/10 MSVC10 / MSVC12 x64\n - OS X 10.10 XCode 7.x x64\n\nIf the build should fail for a specific platform, don't hesitate to create an [issue](https://github.com/cheind/dest/issues).\n\n## Using the tools\n**DEST** comes with a set of handy tools to train and evaluate and trackers. The tools below require OpenCV support. Make sure to enable it before building the library.\n\n#### dest_align\n`dest_align` is a command line tool to test a previously trained tracker on sample images. It shows intermediate steps and is thus best used for debugging. Its main application is the face alignment.\n\nTo run `dest_align` on a single image type\n\n```\n\u003e dest_align -t destcv.bin -d classifier_frontalface.xml image.png\n```\n\nHere `destcv.bin` is a pre-trained tracker file and `classifier_frontalface.xml` contains trained HAAR classifiers for\nface detection. When run, you should see an image with annotated landmarks. This is the initial situation before alignment.\nUse any key to cycle through cascades.\n\nType `dest_align --help` for detailed help.\n\n#### dest_track_video\n`dest_track_video` is a command line tool to track faces over multiple frames.\n\n```\n\u003e dest_track_video -t destcv.bin -d classifier_frontalface.xml video.avi\n```\n\nThis tool can also handle camera input. Specify a numeric device id, such as `0`, to open a physical device.\n\n**DEST** requires a rough estimate (global similarity transform) of the target shape. Here we use an OpenCV\nface detector for exactly this job. It works great but has the drawback of being slow compared to\n`dest::core::Tracker`. For this reason `dest_track_video` supports a `--detect-rate` parameter.\nIf set to 1, the face detector will be invoked in all frames. Setting it to bigger values will run the face detector\nonly every n-th frame. Between detection frames, the tool tracks the face through to simulation a face detector\nbased on the previous tracking results.\n\nType `dest_track_video --help` for detailed help.\n\n#### dest_train\n`dest_train` allows you to train your own tracker. This step requires a training database. **DEST**\ncomes with a set of importers for common face databases. You can use your own\ndatabase as well: all you need to train are images, landmarks and initial estimates\n(usually rectangles) to provide a rough estimate of the shape.\n\nTo train a tracker using a supported database format type\n\n```\n\u003e dest_train --rectangles rectangles.csv --load-mirrowed --load-max-size 640 directory\n```\n\nHere `directory` is the directory containing the shape database. `rectangles.csv`\nprovide estimates of rough shape location and size. `dest_train` makes no assumption on\nhow those are generated, but make sure that you use the same method during training and\nrunning the tracker later on. In case you want to go with OpenCV face detector rectangles,\nyou can use `dest_generate_rects_viola_jones` to generate the rectangles. The IO format for\n`rectangles.csv` is documented at `dest::io::importRectangles`.\n\nType `dest_train --help` for detailed help.\n\n#### dest_evaluate\n`dest_evaluate` can is a tool used to evaluate a previously trained tracker. It loads a\ntest database and and computes tracker statistics. These statistics include the mean Euclidean\ndistance between target and estimated shape landmarks normalized by the inter-ocular distance\nwhen the loaded database contains faces. Here is how you invoke it\n\n```\n\u003e dest_evaluate --rectangles rectangles.csv -t destcv.bin database\n```\n\nWhen using\n - a pre-trained tracker from our [release]([release](https://github.com/cheind/dest/releases)\n - on the [ibug annotated HELEN](http://ibug.doc.ic.ac.uk/download/annotations/helen.zip) test database\n - using OpenCV Viola Jones estimated face rectangles\n\nyou should see roughly the following output\n\n```\nLoading ibug database. Found 330 candidate entries.\nSuccessfully loaded 330 entries from database.\nAverage normalized error: 0.0451457  \n```\n\n#### dest_gen_rects\n`dest_gen_rects` is a utility to generate face rectangles for a training\ndatabase using OpenCVs Viola Jones algorithm. These rectangles can be fed into `dest_train`\nfor learning. Note, if your application comes with a face detector built in, you may want\nto use your face detector to generate these rectangles.\n\nType `dest_gen_rects --help` for detailed help.\n\n## References\n\n 1. \u003ca name=\"Kazemi14\"\u003e\u003c/a\u003eKazemi, Vahid, and Josephine Sullivan. \"One millisecond face alignment with an ensemble of regression trees.\" Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on. IEEE, 2014.\n 1. Viola, Paul, and Michael J. Jones. \"Robust real-time face detection.\" International journal of computer vision 57.2 (2004): 137-154.\n 1. Chrysos, Grigoris, et al. \"Offline deformable face tracking in arbitrary videos.\" Proceedings of the IEEE International Conference on Computer Vision Workshops. 2015.\n 1. Gower, John C. \"Generalized procrustes analysis.\" Psychometrika 40.1 (1975): 33-51.\n\n# License\n\n**DEST** is licensed under 'three-clause' BSD license.\n\n```\nCopyright (c) 2015/2016, Christoph Heindl\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without modification,\nare permitted provided that the following conditions are met:\n\n1. Redistributions of source code must retain the above copyright notice, this\nlist of conditions and the following disclaimer.\n\n2. Redistributions in binary form must reproduce the above copyright notice,\nthis list of conditions and the following disclaimer in the documentation and/or\nother materials provided with the distribution.\n\n3. Neither the name of the copyright holder nor the names of its contributors\nmay be used to endorse or promote products derived from this software without\nspecific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND\nANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\nWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR\nANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\nLOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON\nANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\nSOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n```\n\n**DEST** uses third party libraries that are distributed under their [own terms](ext/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcheind%2Fdest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcheind%2Fdest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcheind%2Fdest/lists"}