{"id":19487389,"url":"https://github.com/elggem/ros_people_model","last_synced_at":"2025-07-30T23:33:28.792Z","repository":{"id":74402875,"uuid":"117931790","full_name":"elggem/ros_people_model","owner":"elggem","description":"ROS package for visual perception of people and associated attributes.","archived":false,"fork":false,"pushed_at":"2020-07-17T12:35:37.000Z","size":2697,"stargazers_count":7,"open_issues_count":0,"forks_count":6,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-25T18:39:58.116Z","etag":null,"topics":["architecture","classifier","detection","dlib","emotion","face","perception","recognition","robotics","ros","sensor","social","tensorflow","visual"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elggem.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-18T04:21:49.000Z","updated_at":"2023-11-15T18:12:47.000Z","dependencies_parsed_at":null,"dependency_job_id":"e87deb22-b1f1-4348-8c4e-e7e840db61a9","html_url":"https://github.com/elggem/ros_people_model","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/elggem/ros_people_model","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elggem%2Fros_people_model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elggem%2Fros_people_model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elggem%2Fros_people_model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elggem%2Fros_people_model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elggem","download_url":"https://codeload.github.com/elggem/ros_people_model/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elggem%2Fros_people_model/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267960767,"owners_count":24172507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["architecture","classifier","detection","dlib","emotion","face","perception","recognition","robotics","ros","sensor","social","tensorflow","visual"],"created_at":"2024-11-10T20:45:41.701Z","updated_at":"2025-07-30T23:33:28.772Z","avatar_url":"https://github.com/elggem.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ros_people_model\n\n![Screenshot Debug Output](https://raw.githubusercontent.com/elggem/ros_peoplemodel/master/images/model_debug.png)\n\nThis is a collection of perception scripts primarily aimed at use in a  social robotics context. It combines various Deep Learning classifiers to build a model of people and their attributes. Working so far are modules for:\n\n  - Detection of [68 Face landmarks](http://dlib.net/face_landmark_detection.py.html) to be used by classifiers based on Dlib 68 Landmark\n  - Face ID using the [128D vector embedding from Dlib](https://github.com/davisking/dlib/blob/master/examples/dnn_face_recognition_ex.cpp), in addition with some simple clustering logic\n  - Emotion recognition using [iCog Emopy](https://github.com/mitiku1/Emopy)\n  - Eyes closed detection based on [iCog EyeState Detection](https://github.com/mitiku1/EyeStateDetection)\n\n![Diagram Architecture](https://raw.githubusercontent.com/elggem/ros_peoplemodel/master/images/arch.png)\n\nIn addition this repository contains some config files and experimental scripts for use of this package on Hanson Robotics humanoid robots.\n\n## API \u0026 message types\n\nCurrently, there are two publishers in `ros_peoplemodel`. One that outputs the integrated raw perceptions from the various modules at the topic `/vis_dlib_faces`, the other one applies some probabilistic smoothing of position and attributes and is published on `/faces`. You can demo this by launching the `vis_debug.py` script for raw perceptions and `model_debug.py` for probabilistic model.\n\nRaw output at `/vis_dlib_faces` outputs raw array of Feature.msg messages, which include the following information. Note that all attributes can be empty in case of detection failure.\n\n```\n# Required fields:\nsensor_msgs/Image crop\nsensor_msgs/RegionOfInterest roi\n\n# Optional fields\ngeometry_msgs/Point[]     shapes\nstring                    face_id\nfloat32[]                 emotions\nfloat32[]                 eye_states\n```\n\nProbabilistic output at `/faces` outputs the following Feature.msg message. Note the added information of position and certainty added in this stage. Also note that all attributes can be empty in case of detection failure.\n\n```\n# Required fields:\nsensor_msgs/Image         crop\n\n# World coordinates:\ngeometry_msgs/Point       position\nfloat32                   certainty\n\n# Attributes\ngeometry_msgs/Point[]     shapes\nstring                    face_id\nfloat32[]                 emotions\nfloat32[]                 eye_states\n```\n\n## Dependencies\n\nMost of the modules used depend on GPU accelerated Dlib or Tensorflow. In order to use it please do the following:\n\n1. Install compatible NVIDIA drivers, [CUDA](https://developer.nvidia.com/cuda-90-download-archive) and [cuDNN](https://developer.nvidia.com/cudnn).\n2. Install [Dlib](http://dlib.net/compile.html) from source using graphics acceleration support (after compilation and all dependencies are installed, follow the instructions for Compiling Dlib's Python Interface)\n3. Install tensorflow-gpu.\n4. Install additional dependencies `numpy`, `scikit-images`, `opencv`.\n\nUpon first launch the system will download the necessary classifier models and extract them to the home folder of the user in `~/.dlib`.\n\n## Usage\n\nIn `launch/` there are several scripts to test the architecture:\n\n  - `webcam_single.launch`: Can be used to launch the /camera node that will publish camera image.\n  - `perception.launch`: Launches the entire architecture as described above.\n  - In addition `scripts/vis_debug.py` will show a window for debugging output of the raw visual perception scripts.\n  - In addition `scripts/model_debug.py` will show a window for debugging the output of model_people node, which fuses the various visual classifiers into a model of perceived faces.\n\n## Roadmap\n\n  - Integration with a classifier for speaking detection\n  - Eventual integration with directional microphone to map what is spoken to individual faces.\n  - Integration with OpenPose and various classifiers for\n    - Body pose estimation (sitting, standing, waving, etc.)\n    - Hand pose estimation (open palm, fist, etc.)\n  - Integration wih Masked RCNN architectures to detect various categories of objects in the hands of people.\n  - Better packaging and setup.py\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felggem%2Fros_people_model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felggem%2Fros_people_model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felggem%2Fros_people_model/lists"}