{"id":14977904,"url":"https://github.com/ionhedes/ptdmproject","last_synced_at":"2026-03-06T17:30:50.644Z","repository":{"id":55513728,"uuid":"324159926","full_name":"ionhedes/PTDMProject","owner":"ionhedes","description":"This is a Face Motion Capture based school project for a Measurements Course","archived":false,"fork":false,"pushed_at":"2021-01-08T21:26:49.000Z","size":136,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-02T21:13:23.146Z","etag":null,"topics":["mocap","raspberry-pi","raspberry-pi-camera"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ionhedes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-24T13:15:46.000Z","updated_at":"2024-01-16T05:31:33.000Z","dependencies_parsed_at":"2022-08-15T02:10:31.642Z","dependency_job_id":null,"html_url":"https://github.com/ionhedes/PTDMProject","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ionhedes%2FPTDMProject","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ionhedes%2FPTDMProject/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ionhedes%2FPTDMProject/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ionhedes%2FPTDMProject/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ionhedes","download_url":"https://codeload.github.com/ionhedes/PTDMProject/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241570921,"owners_count":19984002,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mocap","raspberry-pi","raspberry-pi-camera"],"created_at":"2024-09-24T13:56:31.122Z","updated_at":"2025-11-27T17:03:18.506Z","avatar_url":"https://github.com/ionhedes.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# PTDMProject\nThis is a Face Motion Capture based school project for a Measurements course\n## Table of contents\n* [General Info](#general-info)\n* [Technologies](#technologies)\n* [Components](#components)\n* [Setting up the Raspberry](#setting-up-the-raspberry)\n* [Building the project](#building-the-project)\n* [Development Status](#development-status)\n* [Credits and Bibliography](#credits-and-bibliography)\n\n## General Info\nThe goal of this project is to create a facial motion capture detection system like the one in the illustration below and use it for home automation based on facial gestures, using self-sourced or open-sourced components and prototyping boards (Raspberry, Arduino, etc...).\n\n![Facial Capture Helmet](https://www.norpix.com/applications/images/face-camera-helmet.jpg)\n\nIt is developed by students **Ioan-Alexandru Hedeș** and **Grămescu Dan-Mihai**, as a project assignment for the second year Measurement Devices and Techniques course, at the Polytechnic University of Timișoara, Romania.\n\n## Technologies\nFor this project the following software technologies are used:\n\n* [OpenCV 4.5.0](https://opencv.org/) - for image processing and pattern recognition;\n* [3DF Zephyr](https://www.3dflow.net/3df-zephyr-photogrammetry-software/) - for designing Face Motion Capture models from real-life pictures;\n* [3DS Max](https://www.autodesk.com/products/3ds-max/overview?support=ADVANCED\u0026plc=3DSMAX\u0026term=1-YEAR\u0026quantity=1) - for designing different parts for our motion capture helmet;\n* [Visual Studio Code](https://code.visualstudio.com/) - for coding remotely on the RPi through SSH;\n\n## Components\n* Raspberry Pi 4B (it should work with whatever version of the board you have) (+official charger)\n* Arduino Uno Prototyping Board\n* Pi NoIR V2 Camera\n* nRF24L01 Wireless Transceiver Module\n* microSD card (minimum 32GB)\n* WS2812B Circular 16 LED Strip\n* Several breadboards\n* Bike helmet\n* Straight aluminium rods (diameter 12mm)\n* 3D-printed resin joints\n\n## Setting up the Raspberry\n\n**WARNING**: before continuing, it is necessary to use the official Raspberry Pi 4 Charger, it is the *only* one capable of feeding the board enough current for this project.\n\n### Setting up Raspbian OS\n[Link to installation steps](https://www.raspberrypi.org/documentation/installation/installing-images/)\n\n### Enabling Raspberry native interfacing through different protocols\n* SSH into the Pi, or connect it to a screen and fire a terminal and type ```sudo raspi-config```.\n* Select ```Interfacing Options``` and enable ```Camera```, ```SPI```, ```Remote GPIO```, ```VNC``` (if you want a remote-controlled desktop environment), ```Serial```.\n\n\n### Installing OpenCV 4.5.0 on Raspberry PI\n[Link to installation steps](https://qengineering.eu/install-opencv-4.1-on-raspberry-pi-4.html)\n\n**Note**: By default, by referencing the Makefile downloadable from the given link, OpenCV will be installed under\n```CMAKE_INSTALL_PREFIX=/usr/local```\n\nIn the following lines, we will be referring to the installation path as to ```CMAKE_INSTALL_PREFIX```, just in case you want to install it elsewhere.\n\nOpenCV will work out of the box if you want to use Python3, but for C++, the following tweak has to be made:\n* After a default installation, the ```.hpp``` headers are located in ```CMAKE_INSTALL_PREFIX/include/opencv4/opencv2/```.\nHowever, headers expect themselves to have an absolute path that looks like this: ```CMAKE_INSTALL_PREFIX/include/opencv2/```. Including and using them in source files will **not be possible** with the current folder tree. \nUse ```sudo cp -r CMAKE_INSTALL_PREFIX/include/opencv4/opencv2 CMAKE_INSTALL_PREFIX/include/``` to fix this issue.\n\n### Installing libraries for interfacing RF24 with the Raspberry Pi\n[Link to installation steps](https://www.hackster.io/wirekraken/connecting-an-nrf24l01-to-raspberry-pi-9c0a57)\n\n## Building the project\nA building script can be found on branch ```add-basic-camera-functionality```.\n\n## Development Status\n\n**Note**: this is all preliminary and may drastically change with time.\n\n### Face Recognition\nCode for facial recognition can be found on branch ```add-basic-camera-functionality```.\nIt was written using the [OpenCV Documentation](https://docs.opencv.org/).\n\nFor face detection, an [openly available trained model](https://raw.githubusercontent.com/kurnianggoro/GSOC2017/master/data/lbfmodel.yaml) was used.\n\n\n*29.12.2020* Currently, faces bigger than a certain threshold are detected, marked and the key positions on the face are tracked.\nThe next step is to create routines detecting some easy gestures (like blinking, or opening the mouth).\n![Current progress](https://scontent.fsbz1-2.fna.fbcdn.net/v/t1.15752-9/133664750_1670144359823310_7777849030707191115_n.png?_nc_cat=100\u0026ccb=2\u0026_nc_sid=ae9488\u0026_nc_ohc=h3VOXYiQByIAX9jmZlh\u0026_nc_ht=scontent.fsbz1-2.fna\u0026oh=6b05c37584147d4b1e4903a0dcc8b1b2\u0026oe=601A7B2D)\n\n### Hooking the LED Circles and RF24 module to GPIO\n#### LED Circles\n\nFor easier control of LEDs using the GPIO port, the C/C++ Raspberry native wiringPi library will be used.\n\n*05.01.2021* As per the [datasheet](https://cdn-shop.adafruit.com/datasheets/WS2812B.pdf) of  WS2812B LEDs, they need to be powered by a 3.5-5.3V power supply. The high logic level needed to address the LEDs is based on the power supply. However, I could not get the circle to work with the 3.3V based Raspberry device.  \n\n####RF24 radio transmission module\n\n*Not yet in development*\nSome candidate libraries for interfacing with this module will be tested after the helmet is built.\n\n### Helmet\nA cheap Bike Helmet bought from a store will be used as a base.\n\nThe camera and Raspberry will be placed in a case, mounted on iron rods connected to the helmet, with a fixed field of view on the face of the target.\n\nThe models for the components to be 3D printed can be found on branch ```add-body-3d-models```. All 3D models are self-sourced.\n\n#### The Case Design and first attempts at 3D printing it\n![Case_empty](https://cdn.discordapp.com/attachments/177419077688623104/796116953165070426/unknown.png)\n![Case_full](https://cdn.discordapp.com/attachments/177419077688623104/796117099603820574/unknown.png)\n\n*03.01.2021* The first rendition of the case suffered from a scaling problem during printing, so another one has to be done.\n![Printed case](https://scontent.fsbz1-1.fna.fbcdn.net/v/t1.15752-9/135365838_3436042619953274_2375842809621320780_n.jpg?_nc_cat=106\u0026ccb=2\u0026_nc_sid=ae9488\u0026_nc_ohc=xp0wdlD0JyEAX_awq9x\u0026_nc_ht=scontent.fsbz1-1.fna\u0026oh=b3369a8c6b3ff536a1cd384b4690f640\u0026oe=60199E13)\n\n*08.01.2021* The second one can house the RPi and Camera just fine, but not the LED circle. Now we have insight on how to make a better one to accomodate the wires without bends.\n![Prototype](https://scontent.fsbz1-2.fna.fbcdn.net/v/t1.15752-9/136663733_247118643483240_4503122424718875494_n.jpg?_nc_cat=100\u0026ccb=2\u0026_nc_sid=ae9488\u0026_nc_ohc=7jSSF-qAjYAAX8TBgqX\u0026_nc_ht=scontent.fsbz1-2.fna\u0026oh=9e15d24b37f87782322d6e45c3a5905c\u0026oe=601D636F)\n\n#### Ratchet Clutch \nUsed for adjusting the position of the camera case on the helmet (up/down).\n![Ratchet Clutch](https://cdn.discordapp.com/attachments/177419077688623104/797201097449406514/unknown.png)\n\n#### Joints for fixing the metal rods on the helmet\nAngular piece for joining two rods:\n![Angular joint](https://cdn.discordapp.com/attachments/177419077688623104/796131425077821510/unknown.png)\n\n\nClam for:\n* attaching the rods to the ratchet (2x);\n* attaching the raspberry case to the rods (2x);\nThese clams are screwed onto the helmet, or case respectively.\n\n![Clam](https://cdn.discordapp.com/attachments/177419077688623104/796132164999053312/unknown.png)\n\n### Bonus: creating an animated face model using our faces\nWhy only use our Facial Tracking System for home automation? Why not use the facial data to animate a model in computer graphics software?\n\nThis is a little side-quest of ours, trying to map our faces into 3DF Zephyr, to be animated later using the captured data from the Raspberry Camera:\n![Face angle 1](https://cdn.discordapp.com/attachments/177419077688623104/796115778133950464/unknown.png)\n![Face angle 2](https://cdn.discordapp.com/attachments/177419077688623104/796115847546273802/unknown.png)\n\n## Credits and Bibliography\n* [Menache, Alberto. Understanding Motion Capture For Computer Animation - Second Edition. Elsevier, 2011](https://www.elsevier.com/books/understanding-motion-capture-for-computer-animation/menache/978-0-12-381496-8)\n* [Kurnianggoro's FaceMark pre-trained model](https://raw.githubusercontent.com/kurnianggoro/GSOC2017/master/data/lbfmodel.yaml)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fionhedes%2Fptdmproject","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fionhedes%2Fptdmproject","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fionhedes%2Fptdmproject/lists"}