{"id":13816083,"url":"https://github.com/tzolov/mtcnn-java","last_synced_at":"2025-04-21T22:30:35.835Z","repository":{"id":145796513,"uuid":"148781815","full_name":"tzolov/mtcnn-java","owner":"tzolov","description":"Java MTCNN face detection using ND4J and trained TensorFlow models","archived":false,"fork":false,"pushed_at":"2018-11-26T14:42:04.000Z","size":9046,"stargazers_count":62,"open_issues_count":11,"forks_count":25,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-08-04T05:01:33.529Z","etag":null,"topics":["face-detection","java","mtcnn","nd4j","spring-cloud-stream","tensorflow"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tzolov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-09-14T11:53:18.000Z","updated_at":"2024-05-28T16:03:42.000Z","dependencies_parsed_at":"2024-01-23T07:23:05.465Z","dependency_job_id":null,"html_url":"https://github.com/tzolov/mtcnn-java","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tzolov%2Fmtcnn-java","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tzolov%2Fmtcnn-java/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tzolov%2Fmtcnn-java/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tzolov%2Fmtcnn-java/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tzolov","download_url":"https://codeload.github.com/tzolov/mtcnn-java/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223881080,"owners_count":17219248,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["face-detection","java","mtcnn","nd4j","spring-cloud-stream","tensorflow"],"created_at":"2024-08-04T05:00:33.378Z","updated_at":"2024-11-09T20:33:52.461Z","avatar_url":"https://github.com/tzolov.png","language":"Java","readme":"# MTCNN Face Detection for Java, using Tensorflow and ND4J\n\n[ ![Download](https://api.bintray.com/packages/big-data/maven/mtcnn-java/images/download.svg) ](https://bintray.com/big-data/maven/mtcnn-java/_latestVersion)\n\n\u003e Note: This is still Work In Progress!\n\n`Java` and `Tensorflow` implementation of the [MTCNN Face Detector](https://arxiv.org/abs/1604.02878). Based on David Sandberg's [FaceNet's MTCNN](https://github.com/davidsandberg/facenet/tree/master/src/align) \n`python` implementation and the original [Zhang, K et al. (2016) ZHANG2016](https://arxiv.org/abs/1604.02878) paper and [Matlab implementation](https://github.com/kpzhang93/MTCNN_face_detection_alignment).\n\n[\u003cimg align=\"right\" src=\"https://raw.githubusercontent.com/tzolov/mtcnn-java/master/src/test/resources/docs/scdf-face-detection-2.gif\" alt=\"scdf-mtcnn\" hspace=\"10\" vspace=\"10\"\u003e\u003c/img\u003e](https://github.com/tzolov/computer-vision/blob/master/spring-cloud-starter-stream-processor-face-detection-mtcnn/README.adoc) It reuses the `PNet`, `RNet` and `ONet` Tensorflow models build in [FaceNet's MTCNN](https://github.com/davidsandberg/facenet/tree/master/src/align) and \ninitialized with the original [weights](https://github.com/kpzhang93/MTCNN_face_detection_alignment/tree/master/code/codes/MTCNNv2/model). [Here](https://github.com/davidsandberg/facenet/pull/866) \nyou can find how to freeze the TF models.\n\n\u003e Note that the required Tensorflow models are already pre-bundled with this project! No need to download or freeze those by yourself.\n\nThe MTCNN technique involves significant amount of linear algebra operations, such as multi-dimensional array computations and so. \nTherefore the [ND4J](https://deeplearning4j.org/docs/latest/nd4j-overview) library is leveraged for implementing all (pre)processing steps \nrequired for flowing the data through the `PNet`, `RNet` and `ONet` Tensorflow networks. Furthermore [JavaCV](https://github.com/bytedeco/javacv) is \nleveraged for image manipulation and the `ND4J-Tensorflow` GraphRunner is used to inferring the pre-trained tensorflow models. \nThe combination of those libraries allows to exchange processing states between the `ND4J`, `JavaCV` and `Tensorflow` with little data churn. \nIt also provides off-heap memory management and native support for GPU and BLAS processor features.         \n\n\u003e NOTE: You can find the original NumPy/Python code snippets as inline comments in front of the equivalent ND4J java implementations.\n\n## Quick Start\n\nThe [FaceDetectionSample1.java](./src/test/java/net/tzolov/cv/mtcnn/sample/FaceDetectionSample1.java) demonstrates how to use `MtcnnService` for detecting faces in images.\n![Input Image](./src/test/resources/docs/AnnotatedImage.png)\n\nHere is the essence this sample:\n\n```java\n// 1. Create face detection service.\nMtcnnService mtcnnService = new MtcnnService(30, 0.709, new double[] { 0.6, 0.7, 0.7 });\n\ntry (InputStream imageInputStream = new DefaultResourceLoader() .getResource(\"classpath:/pivotal-ipo-nyse.jpg\").getInputStream()) {\n    // 2. Load the input image (you can use http:/, file:/ or classpath:/ URIs to resolve the input image\n    BufferedImage inputImage = ImageIO.read(imageInputStream);\n    // 3. Run face detection\n    FaceAnnotation[] faceAnnotations = mtcnnService.faceDetection(inputImage);\n    // 4. Augment the input image with the detected faces\n    BufferedImage annotatedImage = MtcnnUtil.drawFaceAnnotations(inputImage, faceAnnotations);\n    // 5. Store face-annotated image\n    ImageIO.write(annotatedImage, \"png\", new File(\"./AnnotatedImage.png\"));\n    // 6. Print the face annotations as JSON\n    System.out.println(\"Face Annotations (JSON): \" + new ObjectMapper().writeValueAsString(faceAnnotations));\n}\n```\nIt takes an input image detect the faces, produces json annotations and augments the image with the faces. \n\nThe face annotation json format looks like this:\n\n```json\n[ {\n  \"bbox\" : { \"x\" : 331, \"y\" : 92, \"w\" : 58, \"h\" : 71 }, \"confidence\" : 0.9999871253967285,\n  \"landmarks\" : [ {\n    \"type\" : \"LEFT_EYE\", \"position\" : { \"x\" : 346, \"y\" : 120 } }, {\n    \"type\" : \"RIGHT_EYE\", \"position\" : { \"x\" : 374, \"y\" : 119 } }, {\n    \"type\" : \"NOSE\", \"position\" : { \"x\" : 359, \"y\" : 133 } }, {\n    \"type\" : \"MOUTH_LEFT\", \"position\" : { \"x\" : 347, \"y\" : 147 } }, {\n    \"type\" : \"MOUTH_RIGHT\", \"position\" : { \"x\" : 371, \"y\" : 147 },\n  } ]\n}, { ... \n```\n## Rea-Time Face Detection with Spring Cloud Data Flow \nThe [spring-cloud-starter-stream-processor-face-detection-mtcnn](https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-face-detection-mtcnn) is \nSpring Cloud Data Flow processor that allows detecting and faces in real time from input image or video streams.\n\n## Maven setup\n\nUse the following dependency to add the `mtcnn` utility to your project \n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003enet.tzolov.cv\u003c/groupId\u003e\n  \u003cartifactId\u003emtcnn\u003c/artifactId\u003e\n  \u003cversion\u003e0.0.4\u003c/version\u003e\n\u003c/dependency\u003e\n```\nAlso register `jcentral` to your list of maven repository (it is available out of the box for Gradle).\n```xml\n\u003crepositories\u003e\n    \u003crepository\u003e\n      \u003cid\u003ejcenter\u003c/id\u003e\n      \u003curl\u003ehttps://jcenter.bintray.com/\u003c/url\u003e\n    \u003c/repository\u003e\n\u003c/repositories\u003e\n```\n\n## Benchmarking\n\nUse this [MtcnnServiceBenchmark](https://github.com/tzolov/mtcnn-java/blob/master/src/test/java/net/tzolov/cv/mtcnn/beanchmark/MtcnnServiceBenchmark.java) to perform some basic benchmarking. You can change the image URI to test\nthe performance with different images.   \n\n## Caveats\n\nThe ND4J, DataVec, ND4J-Tensorflow and JavaCV all rely on core libraries written in C++ (for performance reasons) and wrapped with JNI (e.g JavaCPP) wrapper. \nThis approach has many great advantages such as support for GPU and BLAS math features, off-heap data management, low latency minimal data churn. \n Still the approach apparently can induce significant jar footprint. By default if you bundle support for OS platforms (e.g. linux, android, windows, macos ..) \n the target Spring Boot jar can grow to the insane 1GB jar footprint!\n\nIf you know what your target platform is going to be then you can remedy this situation by setting the `-Djavacpp.platform=` property. For example `-Djavacpp.platform=macosx-x86_64` for MacOS target platform.\n\nAnother possible solution might be to try to substitute the ND4J, DataVec and JavaCV stack using the newly released \n[]Tensorflow Java Ops API (ver. 1.10+)](https://github.com/tensorflow/tensorflow/releases/tag/v1.10.0) later appear to have the superset of what above stack \nprovides but in a single C++ library with much small footprint. \n  \n     \n","funding_links":[],"categories":["Java","人工智能"],"sub_categories":["计算机视觉"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftzolov%2Fmtcnn-java","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftzolov%2Fmtcnn-java","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftzolov%2Fmtcnn-java/lists"}