{"id":24757146,"url":"https://github.com/mathunder/tse_hand_project","last_synced_at":"2025-03-23T08:11:41.776Z","repository":{"id":218559281,"uuid":"746720241","full_name":"Mathunder/tse_hand_project","owner":"Mathunder","description":"This repository is a school assignment. The goal of this exercise is to create a game that captures the hand of the user and creates a 3D model of it.","archived":false,"fork":false,"pushed_at":"2024-01-24T16:26:26.000Z","size":29355,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-28T14:45:23.426Z","etag":null,"topics":["cpp","opencv","opengl","qtcreator"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mathunder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-22T14:42:56.000Z","updated_at":"2024-12-09T11:37:45.000Z","dependencies_parsed_at":"2025-01-28T14:49:37.661Z","dependency_job_id":null,"html_url":"https://github.com/Mathunder/tse_hand_project","commit_stats":null,"previous_names":["mathunder/tse_hand_project"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mathunder%2Ftse_hand_project","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mathunder%2Ftse_hand_project/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mathunder%2Ftse_hand_project/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mathunder%2Ftse_hand_project/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mathunder","download_url":"https://codeload.github.com/Mathunder/tse_hand_project/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245072271,"owners_count":20556353,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","opencv","opengl","qtcreator"],"created_at":"2025-01-28T14:36:06.269Z","updated_at":"2025-03-23T08:11:41.755Z","avatar_url":"https://github.com/Mathunder.png","language":"C++","readme":"\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./ressources/imgs/hand_pixel_art.png\" alt=\"Logo\" width=\"170\" height=\"200\"\u003e\n    \u003ch3 align=\"center\"\u003eHand Game\u003c/h3\u003e\n    \u003cp align=\"center\"\u003eHand tracking with a camera\u003c/p\u003e\n    \u003cbr /\u003e\n\u003c/div\u003e\n\u003cbr/\u003e\n\n\u003ch2 style=\"color: #2783ff;\"\u003e1. Presentation of the app\u003c/h2\u003e\n\n\u003ch3\u003eInteraction with the user\u003c/h3\u003e\n\n\u003cimg src=\"./ressources/imgs/app_layout.png\" alt=\"app layout\" width=\"400\" height=\"320\"\u003e\u003cbr/\u003e\nAs the application starts, the game is on pause. The user needs to click on the `Start` button to begin the game. The camera is not displayed if the player hasn't clicked on the start button.\n\n\u003cimg src=\"./ressources/imgs/app_game_over.png\" alt=\"app game over\" width=\"400\" height=\"320\"\u003e\u003cbr/\u003e\nWhen the user loses, a `Game Over` messages pops and the final score is shown. The user can choose to restart the game. The score is reset if the user loses.\n\n\u003ch3\u003eHand detection zone\u003c/h3\u003e\n\n\u003cdiv\u003e\n    \u003cimg src=\"./ressources/imgs/app_green.png\" alt=\"zone detection green\" width=\"200\" height=\"200\"\u003e\n    \u003cimg src=\"./ressources/imgs/app_orange.png\" alt=\"zone detection green\" width=\"200\" height=\"200\"\u003e\n\u003c/div\u003e\nThe detection zone is the zone where the back of the hand becomes green and turns progressively orange. The detection is made **at the end of this zone**. When the zone is over, the hand's color turns back to grey.\n\n\u003ch3\u003eDefault behavior\u003c/h3\u003e\n\nThe application has some default behavior that one must know in order to enjoy the game to its fullest :  \n- if no hand is detected, the default configuration is a full hand (5 fingers up)\n- due to the inconsistency of the hand detection algorithm, a hand configuration can sometimes be wrongly analyzed or detected\n\n\u003ch2 style=\"color: #2783ff;\"\u003e2. Structure\u003c/h2\u003e\n\n\u003ch3\u003eClass diagram\u003c/h3\u003e\n\n![class diagram](./ressources/imgs/Class_diagram.png)\n\n\u003ch3\u003eClass descprition\u003c/h3\u003e\n\n\u003e`Cube` handles the creation of a cube given some coordinates.\n\n\u003e`Hand` and `Wall` are the classes that create the hand and wall with cubes. They are similar in their functionnality and behavior. The idea is to identify the elements that will always be displayed. For example, the fingers will not always be drawn, but the base of the hand will be, so there is no need for it to be coded more than once.  \nA finger is to be drawn if the function that is assigned to this specific finger is given the argument `true` and vice versa.\n\n\u003e`Camera` handles the display of the camera.\n\n\u003e`Webcam` creates the image that is to be displayed by the `Camera` class. It also deals with the hand detection. \n\n\u003e`Scene` draws the 3D scene and emit signals if some conditions are met.\n\n\u003e`Mainwindow` handles the display of every object on the app window. This class receives all the signals and display the appropriate content.\n\n\u003ch3\u003eMultithreading\n\nIn order to have a fluid application, we chose to use `QtConcurrent` which allows to execute a task in an independant thread. The 3D scene and the hand detection are both executed in a thread, while the rest of the application is executed normally. Thus the application runs without any major drop in performances.\n\n\u003ch2 style=\"color: #2783ff;\"\u003e3. Supported hand configurations\u003c/h2\u003e\n\nThe supported hand position are the following :  \n\u003cdiv\u003e\n    \u003cimg src=\"./ressources/imgs/conf1.png\" width=\"200\" height=\"200\"\u003e\n    \u003cimg src=\"./ressources/imgs/conf2.png\" width=\"200\" height=\"200\"\u003e\n    \u003cimg src=\"./ressources/imgs/conf3.png\" width=\"200\" height=\"200\"\u003e\n    \u003cimg src=\"./ressources/imgs/conf4.png\" width=\"200\" height=\"200\"\u003e\n    \u003cimg src=\"./ressources/imgs/conf5.png\" width=\"200\" height=\"200\"\u003e\n    \u003cimg src=\"./ressources/imgs/conf6.png\" width=\"200\" height=\"200\"\u003e\n    \u003cimg src=\"./ressources/imgs/conf7.png\" width=\"200\" height=\"200\"\u003e\n    \u003cimg src=\"./ressources/imgs/conf8.png\" width=\"200\" height=\"200\"\u003e\n\u003c/div\u003e\n\n\u003ch2 style=\"color: #2783ff;\"\u003e4. Requirements checklist\u003c/h2\u003e\n\n- [x] wall built with cubes\n- [x] display of a wall with a hole in the shape of a hand\n- [x] point of view and light set so there is a 3D look\n- [x] TSE logo on the wall\n- [x] wall with texture\n- [x] hand built with cubes so that a hand shape is recognizable\n- [x] fingers are recognizable\n- [x] hand configuration detection\n- [x] update of the hand\n- [x] great latency\n- [x] webcam zone\n- [x] collision detection\n- [x] visual effect when collision (game over)\n- [x] score and restart button\n\n\u003ch2 style=\"color: #2783ff;\"\u003e5. Class header\u003c/h2\u003e\n\n`camera.h` :\n```c++\n#ifndef CAMERA_H\n#define CAMERA_H\n\n#include \"webcam.h\"\n#include \u003cQWidget\u003e\n#include \u003cQImage\u003e\n\nQT_BEGIN_NAMESPACE\nnamespace Ui {\nclass Camera;\n}\nQT_END_NAMESPACE\n\n/*\n * This class handles the display of the webcam.\n *\n * @author : Matthieu d'Hoop\n */\nclass Camera : public QWidget\n{\n    Q_OBJECT\npublic:\n    // Constructor\n    explicit Camera(QWidget *parent = nullptr);\n    // Display the image captionned by the webcam in the camera_label\n    void displayCamera();\n    // Returns the instance of the Webcam that is being used by the Camera\n    Webcam* getWebcam();\n    // Destructor\n    ~Camera();\n\nprivate:\n    Ui::Camera *ui;\n    Webcam *webcam;\n    bool analyze;\n    unsigned int nb_threads = std::thread::hardware_concurrency();\n\n};\n\n#endif // CAMERA_H\n```\n`cube.h` :\n```c++\n#ifndef CUBE_H\n#define CUBE_H\n\n#include \u003cGL/gl.h\u003e\n#include \u003cQtWidgets\u003e\n\n/*\n * This class handles the creation and display of a cube, given its position and/or its color.\n *\n * @author : Matthieu d'Hoop\n * @co-author : Alban Lemière\n */\nclass Cube {\nprivate:\n    GLfloat light_grey[3] = {0.7f, 0.7f, 0.7f};\n    GLfloat grey[3] = {0.5f, 0.5f, 0.5f};\n    GLfloat dark_grey[3] = {0.3f, 0.3f, 0.3f};\n    GLfloat black[3] = {0.f, 0.f, 0.f};\n    GLfloat blue[3] = {0.f, 0.f, 1.f};\n    float depth;\npublic:\n    // Constructor\n    Cube(float depth);\n    // Destructor\n    ~Cube();\n    // Draws a colored cube at the given coordinates\n    void drawCube(float bottom_left_corner_x, float bottom_left_corner_y, float bottom_left_corner_z, GLfloat color[]);\n    // Draws a cube that will be textured at the given coordinates\n    void drawCube(float bottom_left_corner_x, float bottom_left_corner_y, float bottom_left_corner_z);\n    // Draws a cube with the TSE logo on it at the given coordinates\n    void drawCubeTextureLogo(float bottom_left_corner_x, float bottom_left_corner_y, float bottom_left_corner_z, GLuint texture);\n};\n\n#endif // CUBE_H\n```\n\n`hand.h` :\n```c++\n\n#ifndef HAND_H\n#define HAND_H\n\n#include \"cube.h\"\n\n/*\n * This class handles the modelization of the hand, given the number of fingers that must be displayed.\n *\n * @author : Matthieu d'Hoop\n */\nclass Hand {\nprivate:\n    GLfloat light_grey[3] = {0.7f, 0.7f, 0.7f};\n\npublic:\n    // Constructor\n    Hand();\n    // Destructor\n    ~Hand();\n    // Draws the base oh the hand (the cubes that are always displayed no matter the configuration)\n    void drawBase();\n    // Draws the thumb (always called so a boolean is given as an argument)\n    void drawThumb(boolean is_drawn);\n    // Draws the index (always called so a boolean is given as an argument)\n    void drawIndex(boolean is_drawn);\n    // Draws the middle finger (always called so a boolean is given as an argument)\n    void drawMiddleFinger(boolean is_drawn);\n    // Draws the ring finger (always called so a boolean is given as an argument)\n    void drawRingFinger(boolean is_drawn);\n    // Draws the little finger (always called so a boolean is given as an argument)\n    void drawLittleFinger(boolean is_drawn);\n    // Sets the color of the back of the hand\n    void setColor(GLfloat col[3]);\n    Cube *cube;\n    GLfloat color[3];\n};\n\n#endif // HAND_H\n```\n\n`mainwindow.h` : \n```c++\n\n#ifndef MAINWINDOW_H\n#define MAINWINDOW_H\n\n#include \u003cQMainWindow\u003e\n#include \u003cQTimer\u003e\n#include \u003cQFuture\u003e\n\nQT_BEGIN_NAMESPACE\nnamespace Ui { class MainWindow; }\nQT_END_NAMESPACE\n\n\n/*\n * This class manages the general layout and events of the application.\n * It handles all the signals of the app.\n *\n * @author : Matthieu d'Hoop\n */\nclass MainWindow : public QMainWindow\n\n{\n    Q_OBJECT\n\npublic:\n    MainWindow(QWidget *parent = nullptr);\n    ~MainWindow();\n\nprivate:\n    Ui::MainWindow *ui;\n    QTimer *timer;\n    QFuture\u003cvoid\u003e future_scene;\n    QFuture\u003cvoid\u003e future_camera;\n    int score;\n    bool start;\n\nprivate slots:\n    // Updates the 3D scene\n    void update_scene();\n    // Updates the camera frame\n    void update_camera();\n    // Updates the game over state\n    void update_game_over();\n    // Updates the score of the player\n    void update_points();\n    // Start the hand detection\n    void analyze();\n    // Handles the event \"restart button clicked\"\n    void on_restart_button_clicked();\n    // Handles the event \"start button clicked\"\n    void on_start_button_clicked();\n};\n\n#endif // MAINWINDOW_H\n```\n\n`scene.h` : \n```c++\n#ifndef SCENE_H\n#define SCENE_H\n\n#include \u003cQOpenGLWidget\u003e\n#include \u003cGL/glu.h\u003e\n#include \"hand.h\"\n#include \u003cQTimer\u003e\n#include \"webcam.h\"\n#include \"wall.h\"\n\nnamespace Ui {\nclass Scene;\n}\n\n\n/*\n * This class draws the 3D scene and send signals if some conditions are met.\n *\n * @author : Matthieu d'Hoop\n * @co-author : Alban Lemière\n */\nclass Scene : public QOpenGLWidget\n{\n    Q_OBJECT\n\nprotected:\n    void initializeGL();\n    void paintGL();\n\npublic:\n    // Constructor\n    explicit Scene(QWidget *parent = nullptr);\n    // Allows to link the webcam that analyzes the hand and the 3D scene (link Webcam and Scene)\n    void setWebcam(Webcam *webcam);\n    // Destructor\n    ~Scene();\n\nprivate:\n    Ui::Scene *ui;\n    Hand *hand;\n    Wall *wall;\n    Webcam *webcam;\n    Cube *cube;\n    bool wall_finger[5] = {true, true, true, true, true};\n    int temp = 0;\n    bool one_by_one;\n    bool has_been_analyzed;\n    float compteur;\n    QImage mur;\n    GLuint mur_texture;\n\nsignals:\n    // The following signals must be sent when :\n    // A collision happened\n    void collisionOccured();\n    // The wall has been passed\n    void wallPassed();\n    // The hand must be analyzed\n    void analyze();\n};\n\n#endif // SCENE_H\n```\n\n`wall.h` :\n```c++\n#ifndef WALL_H\n#define WALL_H\n\n#include \"cube.h\"\n#include \u003cGL/glu.h\u003e\n#include \u003cQtWidgets\u003e\n\n/*\n * This class handles the modelization of the wall given the number of fingers that is randomly selected.\n *\n * @author : Alban Lemière\n * @co-author : Matthieu d'Hoop\n */\nclass Wall {\nprivate:\n    GLfloat light_grey[3] = {0.7f, 0.7f, 0.7f};\n    GLfloat grey[3] = {0.5f, 0.5f, 0.5f};\n    GLfloat dark_grey[3] = {0.3f, 0.3f, 0.3f};\n    GLfloat black[3] = {0.f, 0.f, 0.f};\n    GLfloat blue[3] = {0.f, 0.f, 1.f};\n    float position;\n    Cube *cube;\n\npublic:\n    Wall();\n    ~Wall();\n    // Draws the cubes that are always displayed no matter the configuration\n    void drawWallBase();\n    // Draws the hole for the thumb or not\n    void drawWallThumb(boolean is_drawn);\n    // Draws the index for the thumb or not\n    void drawWallIndex(boolean is_drawn);\n    // Draws the hole for the middle finger or not\n    void drawWallMiddleFinger(boolean middleWall_is_drawn, boolean indexWall_is_drawn, boolean ringWall_is_drawn);\n    // Draws the hole for the ring finger or not\n    void drawWallRingFinger(boolean is_drawn);\n    // Draws the hole for the ring finger or not\n    void drawWallLittleFinger(boolean littleWallis_drawn, boolean ringWall_is_drawn);\n    // Draws the logo of TSE\n    void drawLogo();\n    // Sets the position of the wall, allowing the translation\n    void setPosition(float pos);\n    // Gets the position of the wall, allowing to track its progress\n    float getPosition();\n    // Stores the hand configuration for the wall\n    bool configuration[8][5] = {\n        {true, true, true, true, true},\n        {false, true, true, true, true},\n        {true, true, false, false, false},\n        {true, true, true, false, false},\n        {false, false, false, false, false},\n        {false, true, true, true, false},\n        {false, true, false, false, false},\n        {true, true, false, false, true}\n    };\n};\n\n#endif // WALL_H\n```\n\n`webcam.h` :\n```c++\n#ifndef WEBCAM_H\n#define WEBCAM_H\n\n#include \"qimage.h\"\n#include \u003cfuture\u003e\n#include \u003copencv2/opencv.hpp\u003e\n#include \u003copencv2/highgui/highgui.hpp\u003e\n#include \u003cvector\u003e\n#include \u003copencv2/dnn.hpp\u003e\n#include \u003copencv2/imgproc.hpp\u003e\n#include \u003copencv2/highgui.hpp\u003e\n\nusing namespace cv;\nusing namespace cv::dnn;\n#include \u003ciostream\u003e\n\n/*\n * This class handles the hand detection and the generation of images for the video caption.\n *\n * @author : Matthieu d'Hoop\n */\nclass Webcam\n{\npublic:\n    Webcam();\n    ~Webcam();\n    // Generates the image from the webcam and returns it\n    QImage run();\n    // Analyzes the hand configuration\n    void analyze_hand();\n    // Gets the current instance of the Webcam\n    static Webcam* getInstance();\n    // Stores the detected hand configuration\n    bool finger[5] = {true, true, true, true, true};\n\nprivate:\n    cv::VideoCapture *window;\n    cv::Mat frame;\n    Mat *result;\n    QImage *img;\n    static Webcam* s_instance;\n    Net net;\n    String modelTxt = samples::findFile(\"../hand_app/pose_iter_102000.caffemodel\");\n    String modelBin = samples::findFile(\"../hand_app/pose_deploy.prototxt\");\n    String dataset = \"HAND\";\n    int W_in = 368;\n    int H_in = 368;\n    float thresh = 0.1;\n    float scale  = 0.003922;\n    int midx, npairs, nparts;\n\n    float finger_length[5];\n\n\n    const int POSE_PAIRS[3][20][2] = {\n        {   // COCO body\n            {1,2}, {1,5}, {2,3},\n            {3,4}, {5,6}, {6,7},\n            {1,8}, {8,9}, {9,10},\n            {1,11}, {11,12}, {12,13},\n            {1,0}, {0,14},\n            {14,16}, {0,15}, {15,17}\n        },\n        {   // MPI body\n            {0,1}, {1,2}, {2,3},\n            {3,4}, {1,5}, {5,6},\n            {6,7}, {1,14}, {14,8}, {8,9},\n            {9,10}, {14,11}, {11,12}, {12,13}\n        },\n        {   // hand\n            {0,1}, {1,2}, {2,3}, {3,4},         // thumb\n            {0,5}, {5,6}, {6,7}, {7,8},         // pinkie\n            {0,9}, {9,10}, {10,11}, {11,12},    // middle\n            {0,13}, {13,14}, {14,15}, {15,16},  // ring\n            {0,17}, {17,18}, {18,19}, {19,20}   // small\n        }\n    };\n};\n\n#endif\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathunder%2Ftse_hand_project","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmathunder%2Ftse_hand_project","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathunder%2Ftse_hand_project/lists"}