{"id":20346662,"url":"https://github.com/arthurfdlr/openhand-models","last_synced_at":"2025-09-09T08:34:02.754Z","repository":{"id":112736909,"uuid":"322399272","full_name":"ArthurFDLR/OpenHand-Models","owner":"ArthurFDLR","description":null,"archived":false,"fork":false,"pushed_at":"2021-01-09T00:48:36.000Z","size":80593,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-14T22:31:41.517Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ArthurFDLR.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-17T19:57:13.000Z","updated_at":"2021-01-09T00:48:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"f0701269-6dd9-4294-a61e-0ab20bf7751c","html_url":"https://github.com/ArthurFDLR/OpenHand-Models","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArthurFDLR%2FOpenHand-Models","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArthurFDLR%2FOpenHand-Models/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArthurFDLR%2FOpenHand-Models/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArthurFDLR%2FOpenHand-Models/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ArthurFDLR","download_url":"https://codeload.github.com/ArthurFDLR/OpenHand-Models/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241877421,"owners_count":20035403,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T22:13:46.025Z","updated_at":"2025-03-04T15:47:49.408Z","avatar_url":"https://github.com/ArthurFDLR.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Video real-time analysis](./.github/markdown/hand_gesture_analysed.gif)](https://youtu.be/FK-1G749cIo \"Hand gesture classification | OpenHand\")\n\n[**Watch full video here!**](https://youtu.be/FK-1G749cIo)\n\n\n# 🤙 OpenHand model design\n**For a better visualization of this notebook, please use Google Colab**\n\n\u003ca href=\"https://colab.research.google.com/github/ArthurFDLR/OpenHand-Models/blob/main/OpenHand-Models.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\n![GitHub](https://img.shields.io/github/license/ArthurFDLR/OpenHand-Models)\n\nThis Notebook can be used to create Neural Network classifiers running in the [OpenHand application](https://github.com/ArthurFDLR/OpenHand-App).\n\n\nFirst, we have to import several libraries to visualize our dataset and create a new model.\n\n\n```python\n# Visualization tools \nfrom IPython.display import display, HTML, Markdown\nfrom google.colab import widgets\n\n%matplotlib inline\nfrom matplotlib import pylab\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as mpatches\nfrom matplotlib.lines import Line2D\nplt.style.use('ggplot')\n\n# Computation tools \nfrom pathlib import Path\nimport numpy as np\nimport os\nimport pandas as pd\npd.set_option('display.max_columns', None)\npd.set_option('display.min_rows', 5)\n\n%tensorflow_version 2.x\nimport tensorflow\nfrom tensorflow import keras\n\nprint('Available GPU:')\n!nvidia-smi -L\nprint('\\nTensorFlow use GPU at: {}'.format(tensorflow.test.gpu_device_name()))\n```\n\n    Available GPU:\n    GPU 0: Tesla T4 (UUID: GPU-51d5ac3f-2cff-1a5e-cd46-c12b64020977)\n    \n    TensorFlow use GPU at: /device:GPU:0\n    \n\n## Dataset exploration\n\n\n\nWhile OpenHand already contains a basic dataset, you can easily create and import your own using the application!\n\n\n```python\ndataset_df = pd.read_csv(\"https://raw.githubusercontent.com/ArthurFDLR/OpenHand-App/master/Dataset/OpenHand_dataset.csv\")\nlabels = dataset_df.label.unique()\n\ndisplay(Markdown(\"## Complete dataset view\"))\ndisplay(dataset_df)\n\ndisplay(Markdown(\"## Number of samples per label and hand\"))\ndf_hand_labels = {hand_i : dataset_df.loc[dataset_df['hand'] == hand_i].groupby('label') for hand_i in ['left', 'right']}\ndisplay(\n    pd.DataFrame(\n        [df.size() for df in df_hand_labels.values()],\n        columns=labels,\n        index=df_hand_labels.keys(),\n        )\n)\n```\n\n\n### Complete dataset view\n\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003elabel\u003c/th\u003e\n      \u003cth\u003ehand\u003c/th\u003e\n      \u003cth\u003eaccuracy\u003c/th\u003e\n      \u003cth\u003ex0\u003c/th\u003e\n      \u003cth\u003ey0\u003c/th\u003e\n      \u003cth\u003ex1\u003c/th\u003e\n      \u003cth\u003ey1\u003c/th\u003e\n      \u003cth\u003ex2\u003c/th\u003e\n      \u003cth\u003ey2\u003c/th\u003e\n      \u003cth\u003ex3\u003c/th\u003e\n      \u003cth\u003ey3\u003c/th\u003e\n      \u003cth\u003ex4\u003c/th\u003e\n      \u003cth\u003ey4\u003c/th\u003e\n      \u003cth\u003ex5\u003c/th\u003e\n      \u003cth\u003ey5\u003c/th\u003e\n      \u003cth\u003ex6\u003c/th\u003e\n      \u003cth\u003ey6\u003c/th\u003e\n      \u003cth\u003ex7\u003c/th\u003e\n      \u003cth\u003ey7\u003c/th\u003e\n      \u003cth\u003ex8\u003c/th\u003e\n      \u003cth\u003ey8\u003c/th\u003e\n      \u003cth\u003ex9\u003c/th\u003e\n      \u003cth\u003ey9\u003c/th\u003e\n      \u003cth\u003ex10\u003c/th\u003e\n      \u003cth\u003ey10\u003c/th\u003e\n      \u003cth\u003ex11\u003c/th\u003e\n      \u003cth\u003ey11\u003c/th\u003e\n      \u003cth\u003ex12\u003c/th\u003e\n      \u003cth\u003ey12\u003c/th\u003e\n      \u003cth\u003ex13\u003c/th\u003e\n      \u003cth\u003ey13\u003c/th\u003e\n      \u003cth\u003ex14\u003c/th\u003e\n      \u003cth\u003ey14\u003c/th\u003e\n      \u003cth\u003ex15\u003c/th\u003e\n      \u003cth\u003ey15\u003c/th\u003e\n      \u003cth\u003ex16\u003c/th\u003e\n      \u003cth\u003ey16\u003c/th\u003e\n      \u003cth\u003ex17\u003c/th\u003e\n      \u003cth\u003ey17\u003c/th\u003e\n      \u003cth\u003ex18\u003c/th\u003e\n      \u003cth\u003ey18\u003c/th\u003e\n      \u003cth\u003ex19\u003c/th\u003e\n      \u003cth\u003ey19\u003c/th\u003e\n      \u003cth\u003ex20\u003c/th\u003e\n      \u003cth\u003ey20\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eleft\u003c/td\u003e\n      \u003ctd\u003e15.139381\u003c/td\u003e\n      \u003ctd\u003e0.403383\u003c/td\u003e\n      \u003ctd\u003e-0.366229\u003c/td\u003e\n      \u003ctd\u003e0.194999\u003c/td\u003e\n      \u003ctd\u003e-0.361383\u003c/td\u003e\n      \u003ctd\u003e-0.013385\u003c/td\u003e\n      \u003ctd\u003e-0.264461\u003c/td\u003e\n      \u003ctd\u003e-0.149077\u003c/td\u003e\n      \u003ctd\u003e-0.162692\u003c/td\u003e\n      \u003ctd\u003e-0.216922\u003c/td\u003e\n      \u003ctd\u003e-0.060923\u003c/td\u003e\n      \u003ctd\u003e0.083538\u003c/td\u003e\n      \u003ctd\u003e-0.002769\u003c/td\u003e\n      \u003ctd\u003e-0.027923\u003c/td\u003e\n      \u003ctd\u003e0.195922\u003c/td\u003e\n      \u003ctd\u003e-0.173307\u003c/td\u003e\n      \u003ctd\u003e0.094153\u003c/td\u003e\n      \u003ctd\u003e-0.207230\u003c/td\u003e\n      \u003ctd\u003e0.002077\u003c/td\u003e\n      \u003ctd\u003e0.136845\u003c/td\u003e\n      \u003ctd\u003e0.036000\u003c/td\u003e\n      \u003ctd\u003e-0.042462\u003c/td\u003e\n      \u003ctd\u003e0.200768\u003c/td\u003e\n      \u003ctd\u003e-0.153923\u003c/td\u003e\n      \u003ctd\u003e0.079615\u003c/td\u003e\n      \u003ctd\u003e-0.178153\u003c/td\u003e\n      \u003ctd\u003e-0.027000\u003c/td\u003e\n      \u003ctd\u003e0.199845\u003c/td\u003e\n      \u003ctd\u003e0.045692\u003c/td\u003e\n      \u003ctd\u003e0.010846\u003c/td\u003e\n      \u003ctd\u003e0.195922\u003c/td\u003e\n      \u003ctd\u003e-0.095769\u003c/td\u003e\n      \u003ctd\u003e0.079615\u003c/td\u003e\n      \u003ctd\u003e-0.120000\u003c/td\u003e\n      \u003ctd\u003e-0.012462\u003c/td\u003e\n      \u003ctd\u003e0.257999\u003c/td\u003e\n      \u003ctd\u003e0.040846\u003c/td\u003e\n      \u003ctd\u003e0.131999\u003c/td\u003e\n      \u003ctd\u003e0.137769\u003c/td\u003e\n      \u003ctd\u003e0.015692\u003c/td\u003e\n      \u003ctd\u003e0.113538\u003c/td\u003e\n      \u003ctd\u003e-0.057000\u003c/td\u003e\n      \u003ctd\u003e0.036000\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003eleft\u003c/td\u003e\n      \u003ctd\u003e14.065027\u003c/td\u003e\n      \u003ctd\u003e0.333266\u003c/td\u003e\n      \u003ctd\u003e-0.430984\u003c/td\u003e\n      \u003ctd\u003e0.169325\u003c/td\u003e\n      \u003ctd\u003e-0.368800\u003c/td\u003e\n      \u003ctd\u003e0.016691\u003c/td\u003e\n      \u003ctd\u003e-0.255737\u003c/td\u003e\n      \u003ctd\u003e-0.152903\u003c/td\u003e\n      \u003ctd\u003e-0.142674\u003c/td\u003e\n      \u003ctd\u003e-0.226394\u003c/td\u003e\n      \u003ctd\u003e-0.040918\u003c/td\u003e\n      \u003ctd\u003e0.186284\u003c/td\u003e\n      \u003ctd\u003e-0.006999\u003c/td\u003e\n      \u003ctd\u003e0.022344\u003c/td\u003e\n      \u003ctd\u003e0.106063\u003c/td\u003e\n      \u003ctd\u003e-0.113331\u003c/td\u003e\n      \u003ctd\u003e0.100410\u003c/td\u003e\n      \u003ctd\u003e-0.124638\u003c/td\u003e\n      \u003ctd\u003e0.043879\u003c/td\u003e\n      \u003ctd\u003e0.231509\u003c/td\u003e\n      \u003ctd\u003e0.049532\u003c/td\u003e\n      \u003ctd\u003e-0.005922\u003c/td\u003e\n      \u003ctd\u003e0.145635\u003c/td\u003e\n      \u003ctd\u003e-0.135944\u003c/td\u003e\n      \u003ctd\u003e0.094757\u003c/td\u003e\n      \u003ctd\u003e-0.203782\u003c/td\u003e\n      \u003ctd\u003e0.004307\u003c/td\u003e\n      \u003ctd\u003e0.208897\u003c/td\u003e\n      \u003ctd\u003e0.038226\u003c/td\u003e\n      \u003ctd\u003e0.016691\u003c/td\u003e\n      \u003ctd\u003e0.168248\u003c/td\u003e\n      \u003ctd\u003e-0.118985\u003c/td\u003e\n      \u003ctd\u003e0.111716\u003c/td\u003e\n      \u003ctd\u003e-0.147250\u003c/td\u003e\n      \u003ctd\u003e0.038226\u003c/td\u003e\n      \u003ctd\u003e0.191937\u003c/td\u003e\n      \u003ctd\u003e0.004307\u003c/td\u003e\n      \u003ctd\u003e0.027997\u003c/td\u003e\n      \u003ctd\u003e0.151288\u003c/td\u003e\n      \u003ctd\u003e-0.062453\u003c/td\u003e\n      \u003ctd\u003e0.128676\u003c/td\u003e\n      \u003ctd\u003e-0.113331\u003c/td\u003e\n      \u003ctd\u003e0.060838\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e...\u003c/th\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n      \u003ctd\u003e...\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e11202\u003c/th\u003e\n      \u003ctd\u003eOk\u003c/td\u003e\n      \u003ctd\u003eright\u003c/td\u003e\n      \u003ctd\u003e16.430834\u003c/td\u003e\n      \u003ctd\u003e-0.262744\u003c/td\u003e\n      \u003ctd\u003e-0.477646\u003c/td\u003e\n      \u003ctd\u003e-0.070791\u003c/td\u003e\n      \u003ctd\u003e-0.408216\u003c/td\u003e\n      \u003ctd\u003e0.100741\u003c/td\u003e\n      \u003ctd\u003e-0.355123\u003c/td\u003e\n      \u003ctd\u003e0.215096\u003c/td\u003e\n      \u003ctd\u003e-0.314281\u003c/td\u003e\n      \u003ctd\u003e0.268190\u003c/td\u003e\n      \u003ctd\u003e-0.253020\u003c/td\u003e\n      \u003ctd\u003e0.031312\u003c/td\u003e\n      \u003ctd\u003e-0.085572\u003c/td\u003e\n      \u003ctd\u003e0.190592\u003c/td\u003e\n      \u003ctd\u003e-0.048815\u003c/td\u003e\n      \u003ctd\u003e0.255937\u003c/td\u003e\n      \u003ctd\u003e-0.126413\u003c/td\u003e\n      \u003ctd\u003e0.255937\u003c/td\u003e\n      \u003ctd\u003e-0.244852\u003c/td\u003e\n      \u003ctd\u003e-0.038118\u003c/td\u003e\n      \u003ctd\u003e-0.020226\u003c/td\u003e\n      \u003ctd\u003e0.059900\u003c/td\u003e\n      \u003ctd\u003e0.179895\u003c/td\u003e\n      \u003ctd\u003e0.137498\u003c/td\u003e\n      \u003ctd\u003e0.306502\u003c/td\u003e\n      \u003ctd\u003e0.198760\u003c/td\u003e\n      \u003ctd\u003e0.408605\u003c/td\u003e\n      \u003ctd\u003e-0.132053\u003c/td\u003e\n      \u003ctd\u003e-0.024310\u003c/td\u003e\n      \u003ctd\u003e-0.111632\u003c/td\u003e\n      \u003ctd\u003e0.179895\u003c/td\u003e\n      \u003ctd\u003e-0.066707\u003c/td\u003e\n      \u003ctd\u003e0.310586\u003c/td\u003e\n      \u003ctd\u003e-0.038118\u003c/td\u003e\n      \u003ctd\u003e0.412689\u003c/td\u003e\n      \u003ctd\u003e-0.213735\u003c/td\u003e\n      \u003ctd\u003e-0.065151\u003c/td\u003e\n      \u003ctd\u003e-0.262744\u003c/td\u003e\n      \u003ctd\u003e0.106381\u003c/td\u003e\n      \u003ctd\u003e-0.262744\u003c/td\u003e\n      \u003ctd\u003e0.208484\u003c/td\u003e\n      \u003ctd\u003e-0.254576\u003c/td\u003e\n      \u003ctd\u003e0.310586\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e11203\u003c/th\u003e\n      \u003ctd\u003eOk\u003c/td\u003e\n      \u003ctd\u003eright\u003c/td\u003e\n      \u003ctd\u003e15.640315\u003c/td\u003e\n      \u003ctd\u003e-0.258065\u003c/td\u003e\n      \u003ctd\u003e-0.478518\u003c/td\u003e\n      \u003ctd\u003e-0.070381\u003c/td\u003e\n      \u003ctd\u003e-0.404227\u003c/td\u003e\n      \u003ctd\u003e0.105572\u003c/td\u003e\n      \u003ctd\u003e-0.353396\u003c/td\u003e\n      \u003ctd\u003e0.211144\u003c/td\u003e\n      \u003ctd\u003e-0.318205\u003c/td\u003e\n      \u003ctd\u003e0.273705\u003c/td\u003e\n      \u003ctd\u003e-0.259554\u003c/td\u003e\n      \u003ctd\u003e0.043011\u003c/td\u003e\n      \u003ctd\u003e-0.071871\u003c/td\u003e\n      \u003ctd\u003e0.203324\u003c/td\u003e\n      \u003ctd\u003e-0.040590\u003c/td\u003e\n      \u003ctd\u003e0.246334\u003c/td\u003e\n      \u003ctd\u003e-0.130522\u003c/td\u003e\n      \u003ctd\u003e0.265885\u003c/td\u003e\n      \u003ctd\u003e-0.251734\u003c/td\u003e\n      \u003ctd\u003e-0.039101\u003c/td\u003e\n      \u003ctd\u003e-0.013220\u003c/td\u003e\n      \u003ctd\u003e0.058651\u003c/td\u003e\n      \u003ctd\u003e0.182284\u003c/td\u003e\n      \u003ctd\u003e0.140763\u003c/td\u003e\n      \u003ctd\u003e0.303496\u003c/td\u003e\n      \u003ctd\u003e0.199414\u003c/td\u003e\n      \u003ctd\u003e0.409068\u003c/td\u003e\n      \u003ctd\u003e-0.132942\u003c/td\u003e\n      \u003ctd\u003e-0.021040\u003c/td\u003e\n      \u003ctd\u003e-0.113392\u003c/td\u003e\n      \u003ctd\u003e0.182284\u003c/td\u003e\n      \u003ctd\u003e-0.074291\u003c/td\u003e\n      \u003ctd\u003e0.303496\u003c/td\u003e\n      \u003ctd\u003e-0.043011\u003c/td\u003e\n      \u003ctd\u003e0.409068\u003c/td\u003e\n      \u003ctd\u003e-0.226784\u003c/td\u003e\n      \u003ctd\u003e-0.067961\u003c/td\u003e\n      \u003ctd\u003e-0.261975\u003c/td\u003e\n      \u003ctd\u003e0.107992\u003c/td\u003e\n      \u003ctd\u003e-0.265885\u003c/td\u003e\n      \u003ctd\u003e0.209654\u003c/td\u003e\n      \u003ctd\u003e-0.261975\u003c/td\u003e\n      \u003ctd\u003e0.303496\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e11204 rows × 45 columns\u003c/p\u003e\n\u003c/div\u003e\n\n\n\n### Number of samples per label and hand\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003cth\u003e5\u003c/th\u003e\n      \u003cth\u003e6\u003c/th\u003e\n      \u003cth\u003e7\u003c/th\u003e\n      \u003cth\u003e8\u003c/th\u003e\n      \u003cth\u003e9\u003c/th\u003e\n      \u003cth\u003eChef\u003c/th\u003e\n      \u003cth\u003eHelp\u003c/th\u003e\n      \u003cth\u003eSuper\u003c/th\u003e\n      \u003cth\u003eVIP\u003c/th\u003e\n      \u003cth\u003eWater\u003c/th\u003e\n      \u003cth\u003eMetal\u003c/th\u003e\n      \u003cth\u003eDislike\u003c/th\u003e\n      \u003cth\u003eLoser\u003c/th\u003e\n      \u003cth\u003ePhone\u003c/th\u003e\n      \u003cth\u003eShaka\u003c/th\u003e\n      \u003cth\u003eStop\u003c/th\u003e\n      \u003cth\u003eSpoke\u003c/th\u003e\n      \u003cth\u003ePowerFist\u003c/th\u003e\n      \u003cth\u003eHorns\u003c/th\u003e\n      \u003cth\u003eFightFist\u003c/th\u003e\n      \u003cth\u003eMiddleFinger\u003c/th\u003e\n      \u003cth\u003eOk\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003eleft\u003c/th\u003e\n      \u003ctd\u003e207\u003c/td\u003e\n      \u003ctd\u003e201\u003c/td\u003e\n      \u003ctd\u003e213\u003c/td\u003e\n      \u003ctd\u003e203\u003c/td\u003e\n      \u003ctd\u003e204\u003c/td\u003e\n      \u003ctd\u003e206\u003c/td\u003e\n      \u003ctd\u003e208\u003c/td\u003e\n      \u003ctd\u003e202\u003c/td\u003e\n      \u003ctd\u003e241\u003c/td\u003e\n      \u003ctd\u003e221\u003c/td\u003e\n      \u003ctd\u003e205\u003c/td\u003e\n      \u003ctd\u003e235\u003c/td\u003e\n      \u003ctd\u003e203\u003c/td\u003e\n      \u003ctd\u003e202\u003c/td\u003e\n      \u003ctd\u003e207\u003c/td\u003e\n      \u003ctd\u003e208\u003c/td\u003e\n      \u003ctd\u003e205\u003c/td\u003e\n      \u003ctd\u003e206\u003c/td\u003e\n      \u003ctd\u003e201\u003c/td\u003e\n      \u003ctd\u003e203\u003c/td\u003e\n      \u003ctd\u003e202\u003c/td\u003e\n      \u003ctd\u003e205\u003c/td\u003e\n      \u003ctd\u003e209\u003c/td\u003e\n      \u003ctd\u003e204\u003c/td\u003e\n      \u003ctd\u003e204\u003c/td\u003e\n      \u003ctd\u003e209\u003c/td\u003e\n      \u003ctd\u003e205\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003eright\u003c/th\u003e\n      \u003ctd\u003e208\u003c/td\u003e\n      \u003ctd\u003e207\u003c/td\u003e\n      \u003ctd\u003e203\u003c/td\u003e\n      \u003ctd\u003e205\u003c/td\u003e\n      \u003ctd\u003e213\u003c/td\u003e\n      \u003ctd\u003e206\u003c/td\u003e\n      \u003ctd\u003e202\u003c/td\u003e\n      \u003ctd\u003e202\u003c/td\u003e\n      \u003ctd\u003e205\u003c/td\u003e\n      \u003ctd\u003e209\u003c/td\u003e\n      \u003ctd\u003e206\u003c/td\u003e\n      \u003ctd\u003e209\u003c/td\u003e\n      \u003ctd\u003e208\u003c/td\u003e\n      \u003ctd\u003e221\u003c/td\u003e\n      \u003ctd\u003e204\u003c/td\u003e\n      \u003ctd\u003e214\u003c/td\u003e\n      \u003ctd\u003e206\u003c/td\u003e\n      \u003ctd\u003e205\u003c/td\u003e\n      \u003ctd\u003e204\u003c/td\u003e\n      \u003ctd\u003e208\u003c/td\u003e\n      \u003ctd\u003e204\u003c/td\u003e\n      \u003ctd\u003e206\u003c/td\u003e\n      \u003ctd\u003e211\u003c/td\u003e\n      \u003ctd\u003e207\u003c/td\u003e\n      \u003ctd\u003e203\u003c/td\u003e\n      \u003ctd\u003e201\u003c/td\u003e\n      \u003ctd\u003e208\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n```python\ncolors = [\"r\", \"y\", \"g\", \"b\", \"m\"]\n\ndisplay(Markdown(\"## Mean sample visualization\"))\ntb = widgets.TabBar(list(labels))\nfor i, label_i in enumerate(labels):\n    # Only select the first 3 tabs, and render others in the background.\n    with tb.output_to(i, select=(i \u003c 3)):\n        fig, axs = pylab.subplots(1, 2, figsize=(12,6))\n        for ax, hand_visualization in zip(axs, ['left', 'right']):\n            sample_data = dataset_df[(dataset_df.label==label_i) \u0026 (dataset_df.hand==hand_visualization)].drop(['label', 'hand', 'accuracy'], axis=1).to_numpy().mean(axis=0)\n            sample_data_2D = np.stack([sample_data[::2], sample_data[1::2]])\n\n            sample_fingers = [\n                sample_data_2D[:, 0:5],\n                np.insert(sample_data_2D[:, 5:9].T, 0, sample_data_2D[:, 0], axis=0).T,\n                np.insert(sample_data_2D[:, 9:13].T, 0, sample_data_2D[:, 0], axis=0).T,\n                np.insert(sample_data_2D[:, 13:17].T, 0, sample_data_2D[:, 0], axis=0).T,\n                np.insert(sample_data_2D[:, 17:21].T, 0, sample_data_2D[:, 0], axis=0).T,\n            ]\n            for finger, c in zip(sample_fingers, colors):\n                ax.plot(finger[0], finger[1])\n            ax.set_xlim([-1,1])\n            ax.set_ylim([-1,1])\n            ax.set_aspect(\"equal\")\n            ax.set_title('{} hand'.format(hand_visualization), fontdict={'size':18})\n        fig.show()\n```\n\n\n### *Shaka* mean sample visualization\n\n\n![png](./.github/markdown//OpenHand-Models_6_107.png)\n\n\n\n### Create NumPy arrays\n\nNow that we know what the dataset looks like, we can create NumPy arrays to train TensorFlow models. We are only using samples associated with the right hand. However, you can easily use the whole dataset to train a single neural network through hands vertical symmetry.\n\n\n```python\nhand_label = 'right'\ntest_split = 0.15\n\n# Find the minimum number of samples accross categories to uniformly distributed sample sets\ntotal_size_cat = df_hand_labels[hand_label].size().min()\ntest_size_cat  = int(total_size_cat*test_split)\ntrain_size_cat = total_size_cat - test_size_cat\n\nx_train = []\nx_test  = []\ny_train = []\ny_test  = []\n\n# Iterate over each labeled group\nfor label, group in df_hand_labels[hand_label]:\n    # remove irrelevant columns\n    group_array = group.drop(['label', 'hand', 'accuracy'], axis=1).to_numpy()\n    np.random.shuffle(group_array)\n    \n    x_train.append(group_array[:train_size_cat])\n    y_train.append([label]*train_size_cat)\n    x_test.append(group_array[train_size_cat : train_size_cat+test_size_cat])\n    y_test.append([label]*test_size_cat)\n\n# Concatenate sample sets as numpy arrays and shuffle in unison\nshuffler_test = np.random.permutation(test_size_cat*len(labels))\nshuffler_train = np.random.permutation(train_size_cat*len(labels))\nx_train = np.concatenate(x_train, axis=0)[shuffler_train]\nx_test = np.concatenate(x_test, axis=0)[shuffler_test]\ny_train = np.concatenate(y_train, axis=0)[shuffler_train]\ny_test = np.concatenate(y_test, axis=0)[shuffler_test]\n\n# One-hot encoding\ny_train_onehot = keras.utils.to_categorical([list(labels).index(sample) for sample in y_train])\ny_test_onehot  = keras.utils.to_categorical([list(labels).index(sample) for sample in y_test])\n\ndisplay(HTML('''There are {n_cat} categories with a minimum of {min} samples for the {hand} hand,\n                Given a training/test split ratio of {split}% on uniformly distributed sets,\n                the training set has {n_train} samples and the test set has {n_test} samples:'''\n             .format(hand=hand_label, n_cat=len(labels), min=total_size_cat, split=test_split*100, n_train=x_train.shape[0], n_test=x_test.shape[0])))\ndisplay(pd.DataFrame([str(d.shape) for d in (x_train, x_test, y_train, y_test, y_train_onehot, y_test_onehot)],\n                     index = ['x_train', 'x_test', 'y_train', 'y_test', 'y_train_onehot', 'y_test_onehot'],\n                     columns = ['shape']))\n```\n\n\nThere are 27 categories with a minimum of 201 samples for the right hand,\n                Given a training/test split ratio of 15.0% on uniformly distributed sets,\n                the training set has 4617 samples and the test set has 810 samples:\n\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eshape\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003ex_train\u003c/th\u003e\n      \u003ctd\u003e(4617, 42)\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003ex_test\u003c/th\u003e\n      \u003ctd\u003e(810, 42)\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003ey_train\u003c/th\u003e\n      \u003ctd\u003e(4617,)\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003ey_test\u003c/th\u003e\n      \u003ctd\u003e(810,)\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003ey_train_onehot\u003c/th\u003e\n      \u003ctd\u003e(4617, 27)\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003ey_test_onehot\u003c/th\u003e\n      \u003ctd\u003e(810, 27)\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n## Models exploration\n\n\nThis section is optional. The following blocks can be used to compare different model architecture and training processes.\n\n**[Open in Google Colab for details.](https://colab.research.google.com/github/ArthurFDLR/OpenHand-Models/blob/main/OpenHand-Models.ipynb)**\n    \n![png](./OpenHand-Models_files/OpenHand-Models_15_1.png)\n\n\n## Model export\n\nOnce you have a good model, you can save it on your Google Drive. The model is saved using the [folder hierarchy of OpenHand](https://github.com/ArthurFDLR/OpenHand-App#pose-classifier-models).\n\n\n```python\nfrom google.colab import drive\n\nfrom pathlib import Path\n\n\n\ncontent_path = Path('/').absolute() / 'content'\n\ndrive_path = content_path / 'drive'\n\ndrive.mount(str(drive_path))\n``` \n\n\n```python\nmodel = keras.models.Sequential(name = '27Class_3x64',\n                                   layers =\n    [\n        keras.layers.Input(shape=x_train.shape[1]),\n        keras.layers.Dense(64, activation=keras.activations.relu),\n        keras.layers.Dense(64, activation=keras.activations.relu),\n        keras.layers.Dense(64, activation=keras.activations.relu),\n        keras.layers.Dense(len(labels), activation=keras.activations.softmax),\n    ]\n)\n\nmodel.summary()\nmodel.compile(\n    optimizer=keras.optimizers.Adam(),\n    loss='categorical_crossentropy',\n    metrics=['accuracy'],\n)\n\nsave_path = drive_path / 'My Drive'\nfor subfolder in ['OpenHand', 'Models', model.name]:\n    save_path /= subfolder\n    if not (save_path).is_dir():\n        %mkdir \"{save_path}\"\n    \n\nmodel_path = save_path / '{name}_{hand}.h5'.format(name = model.name, hand = hand_label)\n\nmodel.fit(\n    x=x_train,\n    y=y_train_onehot,\n    epochs=10,\n    batch_size=4,\n    validation_split=0.15,\n    shuffle=True,\n    callbacks=[keras.callbacks.ModelCheckpoint(filepath=model_path, verbose=1, save_best_only=True)],\n    verbose = 2,\n)\n\nclass_file = open(save_path / 'class.txt', 'w')\nfor i, label_i in enumerate(labels):\n    class_file.write((',' if i!=0 else '') + label_i)\nclass_file.close()\n```\n\n    Model: \"27Class_3x64\"\n    _________________________________________________________________\n    Layer (type)                 Output Shape              Param #   \n    =================================================================\n    dense_15 (Dense)             (None, 64)                2752      \n    _________________________________________________________________\n    dense_16 (Dense)             (None, 64)                4160      \n    _________________________________________________________________\n    dense_17 (Dense)             (None, 64)                4160      \n    _________________________________________________________________\n    dense_18 (Dense)             (None, 27)                1755      \n    =================================================================\n    Total params: 12,827\n    Trainable params: 12,827\n    Non-trainable params: 0\n    _________________________________________________________________\n    Epoch 1/10\n    981/981 - 2s - loss: 0.7942 - accuracy: 0.7867 - val_loss: 0.1111 - val_accuracy: 0.9582\n\n    ...\n\n    Epoch 10/10\n    981/981 - 2s - loss: 0.0032 - accuracy: 0.9995 - val_loss: 0.0016 - val_accuracy: 1.0000\n    \n    Epoch 00010: val_loss did not improve from 0.00124\n    \n\n\n```python\nmodel_test = keras.models.load_model(model_path)\nmodel_test.evaluate(x=x_test, y=y_test_onehot)\n```\n\n    26/26 [==============================] - 0s 2ms/step - loss: 0.0018 - accuracy: 1.0000\n    [0.0018195720622316003, 1.0]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farthurfdlr%2Fopenhand-models","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farthurfdlr%2Fopenhand-models","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farthurfdlr%2Fopenhand-models/lists"}