{"id":15064128,"url":"https://github.com/frazzledturtle/gesture-recognition-glove","last_synced_at":"2026-02-06T03:02:05.187Z","repository":{"id":256016967,"uuid":"854074427","full_name":"FrazzledTurtle/Gesture-Recognition-Glove","owner":"FrazzledTurtle","description":"Bachelor Thesis","archived":false,"fork":false,"pushed_at":"2025-01-27T19:42:31.000Z","size":14100,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-17T20:53:56.576Z","etag":null,"topics":["bachelor-thesis","embedded-c","embedded-networking","esp32","javascript-website","network-programming"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FrazzledTurtle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":"frazzledturtle","thanks_dev":null,"custom":null}},"created_at":"2024-09-08T10:46:19.000Z","updated_at":"2025-01-27T19:42:35.000Z","dependencies_parsed_at":"2025-07-17T22:37:07.612Z","dependency_job_id":null,"html_url":"https://github.com/FrazzledTurtle/Gesture-Recognition-Glove","commit_stats":null,"previous_names":["frazzledturtle/gesture-recognition-glove"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/FrazzledTurtle/Gesture-Recognition-Glove","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FrazzledTurtle%2FGesture-Recognition-Glove","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FrazzledTurtle%2FGesture-Recognition-Glove/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FrazzledTurtle%2FGesture-Recognition-Glove/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FrazzledTurtle%2FGesture-Recognition-Glove/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FrazzledTurtle","download_url":"https://codeload.github.com/FrazzledTurtle/Gesture-Recognition-Glove/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FrazzledTurtle%2FGesture-Recognition-Glove/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29147377,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-06T02:39:25.012Z","status":"ssl_error","status_checked_at":"2026-02-06T02:37:22.784Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bachelor-thesis","embedded-c","embedded-networking","esp32","javascript-website","network-programming"],"created_at":"2024-09-25T00:11:57.887Z","updated_at":"2026-02-06T03:02:05.138Z","avatar_url":"https://github.com/FrazzledTurtle.png","language":"HTML","funding_links":["https://buymeacoffee.com/frazzledturtle"],"categories":[],"sub_categories":[],"readme":"# Gesture Recognition Glove\n\n*A short intro before starting the article. To anyone who will find themselves reading through this project, please know that, being written around 2020-2021, the algorithm is most likely out-of-date, from the libraries to the dev board used. This is a certain version of what I have written and I've kept everything untouched since then, just as some sort of comparison for myself a long time ago. The algorithm is poorly written and hardware concepts are basic. Hope it helps anyone who's trying to get into embedded systems!*\n\n***The whole prototype will require 5 flex sensors, a development board, an accelerometer,\ngyroscope, and a battery shield.*** During the making of this prototype, a wearable glove will be created\nthat is **responsive, portable, and durable.** It will be able to translate sign language gestures into readable\nform for hearing-impaired people by analysing the bend of the fingers and the position of the hand in\nspace, process it and send messages to the website. The project started by selecting a suitable\ndevelopment board, a suitable power supply corresponding the voltage input of this controller, selecting\nthe right sensors, developing the firmware, designing the PCB and enclosure, and building the prototype.\nThe PCB will rest along with the battery shield onto a custom-made enclosure. This enclosure will rest\non the back of the hand of the user using a strap keep it tight with the hand and a buckle connecting both\nends of the strap. It will have two straps going on the specified holes. One will be resting above the\nthumb and the other strap that connects with a buckle, on the wrist. This will lead for the entire prototype\nto not be too close to the fingers, making it a suitable place to make the connection through the\nconnectors.\n\nThe focus of the demo prototype will be on reading and analysing hand movement. Since ASL’s gestures require an extensive range of motions, including rotation and complex arm movement, the algorithm of this prototype will have to be exceptionally well written. It will need to capture correct and consistent analog to digital data to have a precise definition for each value. To bring the proposed design as close as possible to becoming a viable product, there should be a certain required level of standards related to safety to follow. As with any electronic equipment, there is a risk of shorting wires, which can result in injury or fire. This means the points where wires are connected with the Flex Sensors will be fully insulated and the enclosure will have to cover most of the PCB and power supply. Sometimes between gestures, there are very subtle differences and minor changes, so this similarity can be read and\nanalysed from the glove as the same gesture. The complexity of ASL make it challenging to distinguish\nand capture differences between gestures. This means the sensors should have a desired sensitivity and\nlinearity as well as a well-developed software algorithm that detects these subtle changes. In this case,\nwe were faced with two options on how to proceed and develop the algorithm. One could be using\nmachine learning, where the glove would be trained over time by performing the same gestures, or by\ncreating a dataset function holding if constraints with segments of values. With datasets one gesture\nwould have exact predetermined values for every sensor reading and would be true if those values would\nbe met. Deciding which method to choose, and what components to use, we had to make several research\non similar products done beforehand and learn from them.\n\n## Block Diagram\n\nOur next very important step would be to design the block diagram, which will be crucial in the\nfoundation to the realization of this prototype. The block diagram provides a detailed view and\ninstructions, making it much easier to understand how every component is related to each other. It is\ncomprised of the Power supply, Input information, Data processing unit, Data acquisition and the\nWebsite. Starting from the main source of power, the Power supply block (see figure below). It consists of\nthe battery which supplies the gyroscope, accelerometer, flex sensors and development board. The power\nsupply in our case will give a +5V as input voltage that goes into the voltage regulator of development\nboard which will drop it to 3.3V. We can see that the red line represents the input voltage throughout the\ncircuit. It goes from the microcontroller to the voltage divider of the flex sensor and to that sensor, and\nit also connects with the accelerometer and gyroscope module.\n\nFlex sensors will be connected with the microcontroller through an ADC pin and the\naccelerometer and gyroscope through a I2C pin. The input information block is where the hand gestures\nperformed are picked up from the sensors as analog data and fed into the controller. This part only holds\noutside factors regarding the prototype. The data given to the controller is processed and analysed\nthrough the algorithm and checked with many datasets. Datasets in our case are going to be numerous if\nconstraints that hold specific values for when a certain sensor reaches that value. If one of these\nconstraints if fulfilled, then the output information, which is just a String value, will sent to the opened\nwebpage through the built-in Wi-Fi module of our development board. From the feeding of the data to\nthe microprocessor and till the moment this data was sent, it’s called Data Processing Block. In the last\nstages of the block diagram, the data sent to webpage, arrives, gets pulled from a function in the\nalgorithm and is displayed.\n\nThe website will display the word, phrase, number, or letter depending on what gesture was\nperformed from the user of the glove. The information will be displayed as a translated gesture, which\nis the goal of our prototype. To get access to this website, the user will have to be connected into the\nsame Wi-Fi that the board is connected to. By entering this local IP address into a browser of our choice,\nwe will be directed into the webpage, making the whole process of interaction, simple and efficient. The\nwebsite will be sending and giving requests with the board constantly.\n\n![image](https://user-images.githubusercontent.com/66420672/174337449-ea25139d-c078-4aae-bba5-4b6928d0845c.png)\n\n#### ***ESP32 Devkit V1***\n\nThe ESP32 DevKit V1 is one of the development boards created to evaluate the ESPWROOM-32 module. It is based on the ESP32 microcontroller that boasts Wi-Fi, Bluetooth, Ethernet,\nand Low Power support all in a single chip. ESP32 is already integrated antenna and RF balun, power\namplifier, low-noise amplifiers, filters, and power management module. The entire solution takes up the\nleast amount of printed circuit board area. This board is used with 2.4 GHz dual-mode Wi-Fi and\nBluetooth chips by TSMC 40nm low power technology, power, and RF properties best, which is safe,\nreliable, and scalable to a variety of applications. The internal flash of the ESP32 module is organized\nin a single flash area with pages of 4096 bytes each. There exist two different layouts based on the\npresence of BLE support. Power to the ESP32 DevKit v1 is supplied via the on-board USB Micro B\nconnector or directly via the “VIN” pin. The power source is selected automatically. The device can\noperate on an external supply of 5 to 20 volts. If using more than 12V, the voltage regulator may overheat\nand damage the device. The recommended range is 5 to 12 volts. The ESP32 DevKit V1 comes with a\nserial-to-USB chip on board that allows programming and opening the UART of the ESP32 module. It\ncan be programmed in different programming environments like Arduino IDE, Espressif IDF (IoT\nDevelopment Framework), MicroPython, JavaScript, LUA.\n\n![image](https://user-images.githubusercontent.com/66420672/174338389-c25a7bf2-6ff0-4eaa-a664-2e6fb347675f.png)\n\n##### **Power Consumption**\n\nSince our goal is to build a portable and reliable prototype, we should consider how much does\nthe ESP32 consumes so we can have a better understand of its lifetime. ESP32 has different power modes\nsuch as:\n\n- Active Mode: The chip radio is powered on. The chip can receive, transmit, or listen.Page |36\n- Modem-sleep Mode: The CPU is operational, and the clock is configurable. The WiFi/Bluetooth baseband and radio are disabled.\n- Light-sleep Mode: The CPU is paused. The RTC memory and RTC peripherals, as well as\nthe ULP coprocessor are running. Any wake-up events will wake u the chip.\n- Deep-sleep Mode: Only the RTC memory and RTC peripherals are powered on. Wi-Fi and\nBluetooth connection data are stored in the RTC memory. The ULP coprocessor is\nfunctional.\n- Hibernation Mode: The internal 8 MHz oscillator and ULP coprocessor are disabled. The\nRTC recovery memory is powered down. Only one RTC timer on the slow clock and\ncertain RTC GPIOs are active.\n\nWhile we want to have a website open during the whole time the device is on, and the ESP32\nwill be constantly communicating with the website, the power mode will be stated as ‘Active’. To have a better understanding about the development boards power consumption in an Active\nMode, we will be including a table displaying the RF Power-Consumption Specifications, from the ESP32 DevKit V1 documentation. We can approximate that our power consumption would be 240 mA, considering that we must\nkeep the development board and website communicating constantly to update the data. The formula to\nfind the battery life is as follows:\nBattery Life (h) = Battery Capacity (mAh) / Device consumption (mA)\nFor example, if we were to have a battery with the capacity of 2000 mAh, and our maximum\nconsumption, in ideal conditions, we would have the device running for 8 hours and 19 minutes straight\nwithout interruptions, which is enough for our project.\n\n### Development Board\n\nRegarding the power supply, a suitable battery was searched for to match the input voltage of the\nchosen development board. As said before, our ESP32 requires an input voltage of 5V to 12V, so any\nbattery that has that output is enough for our project. We will be conducting research for rechargeable\nbattery options because we would like for the lifetime of the prototype being turn on to be as much as\npossible without changing the battery. \n\nThree most common rechargeable battery types include NickelCadmium (NiCd), Nickel-Metal-Hydride (NiMH), and Lithium Polymer (LiPo). Each of these is a good\nchoice for projects and they all have their benefits and drawbacks. Nickel-Cadmium batteries have a\nvery high cycle count, and they are typically the cheapest of the three options. However, this type of\nbattery has a relatively low energy density, so they offer less capacity per cell than the other two types.\nOn the flip side, NiCd batteries can often deliver very high currents, which makes them perfect for power\ntools. Note that NiCd batteries contain toxic heavy metals, which makes them more difficult to recycle.Page |38\nNickel-Metal-Hydride batteries are like NiCd cells. However, NiMH batteries offer an improvement in\ncapacity over NiCd batteries. One major drawback of NiMH cells is their high self-discharge rate. Li-Po\nand Li-Ion batteries have a significantly higher energy density compared to typical NiMH and NiCd\ncells. However, this benefit also means that you usually pay more for Li-Po and Li-Ion batteries. Besides\nthe increased price, lithium batteries always require monitoring by a battery management system, as\nthese batteries are sensitive to improper handling. Note that Li-Po batteries come in various shapes and\npackages, of which some can be very flat. This makes Li-Po cells perfect for handheld devices such as\nmodern smartphones. Li-Ion batteries often come in larger sizes and are typically used in applications\nwhere size isn’t critical. Note that there is not a single type of Li-Ion or Li-Po battery. Instead, the\ncategory of lithium batteries comprises a whole range of different chemical formulations that all have\ntheir very own pros, cons, and recommended applications.\n\nIn our case, a Li-Ion was chosen because they can be recharged many times and are quite stable\nand efficient. They tend to have a higher energy density, voltage capacity and lower self-discharge rate\nthan other rechargeable batteries. This makes for better power efficiency as a single cell has longer\ncharge retention than other battery types. For the battery to be included and connected with the rest of\nthe prototype, it should be connected in a certain way. Either through connecting the VIN and GND pins\nof the development board or through USB. As mentioned above, our chosen board has a micro-USB\ntype ‘B’ female. To have a much consistent connection and reliable to connect and disconnect, it was\ndecided to use a battery shield which will have an USB to make the connection with our board. The size\nof this shield should be able to be held on the back of the hand of the user and it should have the option\nof being rechargeable. The power supply’s purpose will be simple. It will be connected through USB\ntype A with the development board with the right input voltage. Since the development board requires\nan input voltage of 5 V and has an operational voltage of 3.3V, it will be supplied the exact amount. The\nbattery shield chosen is the 18650 Battery shield (V3). It was considered for multiple reasons. This shield\nis a cheap and portable power supply that can accommodate an ‘unprotected’ single-cell lithium-ion\nbattery. It also includes a standard USB type ‘A’ female connector, one power on/off switch and a micro-USB\ntype ‘B’ female for recharging. Its key specifications are as follows:\n\n• Battery charging input: 5V/500mA typical via micro-USB port\n\n• Power supply output: Switched 5VDC via USB type ‘A’ port, 3x 5VDC (VDC= volts\nto direct current) (up to 4A) connectors, and 3x3VDC (up to 1A)\n\n• System status indicators: Battery charging and battery charged LEDs\n\n• Battery protection mechanism: Battery overcharge and deep discharge\n\nThe background electronics are a clever blend of three requisite circuits a lithium-ion\nbattery charger circuitry, a battery protection circuitry, and a dc-dc boost converter circuitry. Refer to\nthe annotated image provided below to get a deep insight on the underlying electronics. In the shield,\nthe dc - dc boost converter chip is used to deliver 5VDC (4.99V) through the USB ‘A’ socket and 5V\nmale-header points, generated from the available 3.7V-4.2VDC Li-Ion battery output. The second\nchannel output, marked as 3V but 3.3V on the dot, is supplied by the parallel combination of three\npositive voltage regulator chips. The 3.3VDC supply is derived from the 3.7V-4.2VDC Li-Ion battery\noutput (not from the 5VDC output)\n\n![image](https://user-images.githubusercontent.com/66420672/174339083-54c56ddf-b4e8-47c6-b6ea-0db6693cb14e.png)\n\nThis device is composed of many components. They will be described by declaring their component\ninitials which can be check at its Schematic.\n\n- U9 WD01A = Li-Ion battery protection chip\n- F1 8205A = Dual N-Channel MOSFET\n- U2 TP4056E = Li-Ion battery charger chip\n- U7 FP6298 = 4.5A current mode DC-DC boost converter chip\n- U4, U5, U6 (XC6206) = Positive voltage regulator chip (3.3V)Page |40\n- L1 Green LED = Battery done charging indicator\n- L2 Red LED = Battery charging indicator\n- S1, S2, S4 = Schottky Diodes\n\n![image](https://user-images.githubusercontent.com/66420672/174339127-b9741c29-c8bc-4bcd-b324-41db76487393.png)\n\n### Flex Sensor\n\nCapturing the bend of the fingers could be done in very few ways. Judging from the research\nconducted beforehand on similar projects, flex sensors proved to be a good and efficient way of\nmeasuring the bend of the fingers. Our options regarding the decision on acquiring the right flex sensor\nfor our prototype were limited. Generally, there are 3 main types of flex sensors: Conductive ink-based\nflex sensors, fibre optic flex sensors and capacitive flex sensors. The conductive ink-based flex sensors\nare quite common for projects of this nature. Capacitive and fibre optic flex sensors can be used more inPage |41\nindustrial and professional equipment. Conductive ink-based flex sensors are unipolar sensors, which\nmeans the change of resistance is true only if you bend it in one direction. Bending the sensor on the\nother direction does not change its resistance. It uses conductive ink whose resistance varies when bent.\nThe ink may contain carbon or silver to make it conductive. The spacing between carbon particles are\nlager when bent and close when straight which results in a change of resistance.\nRegarding the purchasing options, there were only two. 2.2 inches and 4.5 inches flex sensors\nwere available. \n\nFive 4.5 inches flex sensors were purchased instead of the 2.2 inches because they better\nmatch the length of the fingers. The main computing device is going to be used to configure input and\noutput pins for each sensor, setup a clock speed, configure communications with the ADC, configure\ncommunications with I2C pins, control sensor data, and send valid data to a Wi-Fi module. Flex Sensors\nwill be manipulated in the algorithm to decide whether the finger will be closed, open or to determine\nexact positions for them.\n\n##### **Power Consumption**\n\nEach Flex Sensor is treated as a variable resistor whose resistance increases which the sensors\nbending. Each sensor constitutes a part of its own voltage divider circuit, whose output is processed\nthrough one of the development board’s analog-to-digital converters. The voltage provided from the\ncontroller will be changing because of how much the corresponding finger is bent, which can lead to\nproblems while using the device. More specifically speaking, if the flex sensor would be changing its\nvalue, the voltage would do as well. To fix this issue, voltage dividers will be placed before the Flex\nSensors. Voltage divider’s purpose in this case is to supply analog voltage values from\ndivider central point where both two resistors are connected. Voltage dividers only divides power\nsupply voltage value in proportion of Resistors values. One resistor has fixed value other value is\nchanging. Imagine two values are equal. Then voltage will be divided by 2.\n\n![image](https://user-images.githubusercontent.com/66420672/174339269-4a46b2b5-439c-489c-8769-c8e2bab4b698.png)\n\nThe flex sensors were measured using a multimeter to determine their values. We agreed to set\nthe resistance when the sensor is straight as 12.3kΏ and for when it’s bend at 180º, it’s 29kΏ. By\nconnecting the sensor to a voltage divider circuit, the resulting voltage can be calculated from the\nformula:\n\n![image](https://user-images.githubusercontent.com/66420672/174339328-6d533eee-0c62-4259-a789-aa933124299a.png)\n\nSo, if we wanted to find out the output voltage, taking the flex sensor when it is straight and the\npower supply voltage of 3.3V, according to the formula, it would be equal to 1.47V and for when we\nare calculating for when flex sensor is bent, we get 0.846V.\n\n![image](https://user-images.githubusercontent.com/66420672/174339389-b67e585f-125a-4b28-93ac-e6d38b4f4093.png)\n\nNow, to find the current consumption of each flex sensor we would use this formula:\n\n![image](https://user-images.githubusercontent.com/66420672/174339507-e483cf93-b7d1-49b0-ac5b-46479b146936.png)\n\nWe can see that the circuit consumes very little current making it an energy efficient circuit,\nconsidering that the board will reach around 240mA as consumption. This makes the use of the 10kΏ a\ngood value to choose for the voltage divider.\nRegarding the value of voltage that will serve as an indicator for us to know if the flex sensor has\nbeen bent, we decided that bending the sensor 90º (see Figure 24). This means that we can calculating\nthe resistance at half-way to our maximum bend value of 180º. This resistance is equal to:\n\n29000Ώ – 12300Ώ = 16700Ώ.\n16700Ώ / 2 = 8350Ώ.\n12300Ώ + 8350 Ώ= 20650Ώ\n\nWe can calculate the voltage of this threshold as follows:\n\n![image](https://user-images.githubusercontent.com/66420672/174339574-ca9fe9ad-84f0-47d0-8d8c-f26934f831f2.png)\n\nFrom our results we can understand that the voltage of 1.07V will be understood as a bend in the\nflex sensor with that value.\n\n### Accelerometer and Gyroscope\n\nRegarding both accelerometer and gyroscope, the idea was based on the solution in Review 1,\nwhere the team was using a MPU6050 module which contained both an accelerometer module and a\ngyroscope module. It combines a 3-axis gyroscope and a 3-axis accelerometer on the same silicon die\ntogether with an onboard Digital Motion Processor (DMP) capable of processing complex 9-axis\nalgorithms. This will help us to measure acceleration, velocity, orientation, displacement, and many\nother motions related parameter of the prototype. The price of this is 5.5€ which made it a very\nreasonable and efficient option for the realization of our prototype. It is comprised of 5 pins, INT, SCL,\nSDA, GND and VIN. \n\nThe INT is the interrupt signal which is an optional pin to connect. It indicates that\ndata is available for MCU to read. SCL stands for Serial Clock and is used for providing clock pulse for\nI2C communication. SDA means Serial Data and is used for transferring data through I2C\ncommunication. VIN will provide the power for this module, can be +3V to +5V depending on the added\npower supply. In our case, the power will supply 5V into the board. GND stand for ground and connected\nthe system to ground. The main features or the MPU6050 are:\n\n• MEMS 3-axis accelerometer and 3-axis gyroscope values combined,\n• Power Supply: 3-5V,Page |44\n• Communication: I2C protocol,\n• Built-in 16-bit ADC provides high accuracy,\n• Built-in DMP provides high computational power,\n• Built-in Temperature sensor.\n\nA gyroscope to capture the orientation and the angular velocity of the hands. Gyroscope will prove\nuseful for certain gestures that will require rotary movement in space. Accelerometer will be used to\nmeasure the acceleration of the hand, which will be useful for gestures of phrases or words that need\nstretched movement of the hand in space.\n\n![image](https://user-images.githubusercontent.com/66420672/174339685-b53cd42b-e52c-4275-a50f-ee5fb95bd89b.png)\n\n## Schematic Diagram\n\nThe first step taken for the realisation on this prototype was to design the schematic by placing\nthe components on a simulated breadboard to have a clearer view of how everything would be connected.\nThis proved to be quite helpful as a step before placing the components a real breadboard and start\nexperimenting. was designed using Altium Designer. This would\nprove very useful, because the schematic would be imported easily for the next step of creating the PCB\nusing the same software. In the schematic, we can see how the components are connected with each\nother. The flex sensors are shows as variable resistors with the names of Flex 1, Flex 2, Flex 3, Flex 4,\nand Flex 5. Each sensors had their resistances measured beforehand to decide their value without any\nforce put into. This proved an approximated value of 12kΏ. Each of these sensors is connected to their\nown voltage divider. These resistances have been assigned a value of 10kΏ. Each flex sensor has been\nconnected to an ADC pin of the ESP32. These pins are GPIO34, GPIO 35, GPIO 33, GPIO 32 and\nGPIO39. We can also see in the schematic design that the MPU6050 module is connected without\ndevelopment board as well.\n\n![image](https://user-images.githubusercontent.com/66420672/174339795-919f377b-5427-48b1-b03b-03b3396c9b92.png)\n\nThe ESP32 has 6 Analog-To-Digital pins assigned as ADC1 and 9 Analog-To-Digital pins\nassigned as ADC2. Since our goal is to send the data gathered to an opened webpage, this means that\nthe flex sensors would be able to connect to ADC1 pins only. From the schematic, we can\nalso see that the MPU6050 module is connected with the board to the specified pins. In the Pinout Figure\nof the ESP32, we can see that GPIO21 and GPIO22 have the following names assigned: I2C SDA and\nI2C SCL respectively. Therefore, they are connected to the pins of the same names of the module. Every\nVIN pin has been connected and so has the GND pins. It is displayed that the input voltage of the circuit\nwill be 3.3V as is the operating voltage of the development board. The 18650 battery in-use are the\n18650 US18650VTC4 Li-Ion 3.7V 2100mAh 30A. \n\nThe ESP32’s lifetime on battery power has not been\ntested, but as an approximation it could last around 4 months without charging the battery. Since the\nboard will be awake, it will need to be on during the whole time. Our main objective is to enable the user\nof the glove to be able to fully communicate with another person through the use of a website, which\nwill serve as a lobby where anyone can connect with. This website will be run on the development board\nat the same time it is reading the sensor data. As long as the user of the glove has a mobile phone and\ninternet mobile data, it will make the communication much easier. They will be taken to a specific IPPage |47\naddress where this webpage is opened. This was an option we explored as a very efficient one, amongst\nother options. We could have developed an easy-to-use mobile application that would’ve performed the\nsame tasks, but it opened the way to new problems. Such as for what reason would someone download\nthis app, taking into consideration the chances a person has to meet someone with a hearing impairment.\nEven if they did, it would’ve made it more difficult to get the message across to download it in order to\ncommunicate. How would someone know the existence of this app? What if in an extreme situation,\nthere was a need for the interaction of a person and a hearing-impaired person? What if the person who\nwill need to get the app does not have enough internet to download it? All these obstacles made the\ndevelopment of an app obsolete, as it would lead to more headaches and annoyances throughout the\ndaily life. We decided to research on this matter as well as it is very important how we will be displaying\nthe translated gesture.\n\n## PCB Design\n\nFrom the schematic design, all the connections, components and names of components have been\nimported to create the PCB of this prototype. The PCB was designed while have the enclosure design in\nmind. This means that to properly design the PCB, the dimensions should’ve have been decided before\ncreating it. Therefore, the dimensions of this PCB are 70x50mm. According to the design of the\nenclosure, the longest part would be placing the 18650-battery shield which would be very near the PCB.\nThe PCB’s dimensions resemble the back of the hand, from the end of the wrist till the beginning of the\nfingers. PCB has two layers, a top and bottom one. It has a top and bottom overlay with the name of the\nauthor and its logo and the logo of the university on the back. 4 mounting holes have been put on the\nfour corners with the sizes of 2.5M screws. Every resistor has their name on the top overlay, just like the\ndevelopment board and MPU6050 module. The PCB has a common ground, which means the ground\npins are not connected through route connections but go through the same layer. This is a very effective\ntool to minimize the routes through the PCB, making less chaotic. Via stitching was used to tie together\nlarge copper areas on different layers to create a strong vertical connection through the board structure.\nThis helps maintain a low impedance and short return loops. The width of the board was decided to be\na standard 1.6 mm. The PCB was designed and ordered through PCBWay, and it arrived surprisingly\nquick, in around 5 days. When ordering our PCB, Gerger Files were generated and the Drill Holes files\nthat are necessary for the manufacturing and development of our PCB.\n\n![image](https://user-images.githubusercontent.com/66420672/174339876-3b0ad631-d8bb-4b01-83c1-1213fb321a04.png)\n\n![image](https://user-images.githubusercontent.com/66420672/174339891-398759ca-d42e-4021-bc85-684f90bfa280.png)\n\nThe resulting PCB that was ordered and shipped. PCB was ordered twice. The first time’s\nresulting PCBs had two main mistakes. The place where the MPU6050 would be soldered into, was too\nclose to the development board and Flex sensor 5 was connected with the wrong the ADC pin of the\nboard. It was connected to a ADC2 pin, the ESP32’s ADC2 pins\ncannot be used while using the Wi-Fi module. Redesigning the PCB, the stitching holes were kept the\nsame. The holes for the MPU6050 have been moved up by 3 mm leaving more space for it and the Flex\nSensor 5 was connected with GPIO 39 of the ESP32.\n\n## Software Framework\n\nAfter designing the PCB, we put our attention to create a very efficient algorithm that would\ngather data, analyse them, process them, check them, open a website, create a webpage, and update the\nvalues. In this chapter, we are going through the Software Framework, which explains the logical idea\nof the whole algorithm. From the extensive research, it was decided that the prototype will be working\nusing a dataset in its algorithm, discarding completely the idea of including machine learning. We will\nbe exploring the ESP32 and its entire capabilities while focusing on speed of transmission of data.\nThe framework starts from booting the microcontroller. As soon as it\nis turned on, it will initialize the Accelerometer and Gyroscope sensors. This means, a function will\nperform a checking statement to verify whether MPU6050 is connected or not. If it’s not, then the\nMPU6050’s LED won’t turn on. In this situation the main board can be restarted by pressing the EN\nButton. If the problem is persistent, then this means there is a problem with the connection between the\nmain board and the sensor module. This explains the significance to this simple function that is critical\nto the communication between the device and the user as it indicates the state of functionality. It is very\nimportant to have visual representation of the status of the device the user is using. This function is one\nof the few functions that were added to serve as debugging functions in case problem would arise.\nWhenever the board is booted, a red LED is turned on to indicate it. \n\nThe next part is calibration of the\nflex sensors. Here the user has 15 seconds from the boot of the board to open and close their fingers to\ncalibrate the flex sensors values. The ESP32s ADC pins gathering of data fluctuate extensively, with the\nvalues changing for the same position of the finger every time the board is booted. This proved that the\nspecified maximum and minimum values of flex sensors resistance would be incorrect. The\nmeasurements would be less than the minimum and more than the maximum than the measurements\nconducted with a multimeter for each flex sensor. Calibrating the flex sensor would give each flex sensor\ntheir individual maximum and minimum, balancing out the values differently each time the board would\nboot. \n\nAfter taking care of every sensor the prototype has, the next step would be to open the website.\nThe board has been given specific credentials to connect to a Wi-Fi in its surrounding. It will scan the\narea and if it finds the same name of the router or switch device, it will connect to it, putting in the\npassword provided as well. In this stage, the board has successfully initialized a Transmission Control\nProtocol (TCP) server. This is a connection-oriented communication protocol that facilitates the\nexchange of messages between computing devices in a network. If the server has been initialized, then\nthe board connects to it. From experimenting, it has happened that the device may not find the specified\nWi-Fi, or there may be a problem with the transmission of the Wi-Fi. This issue leads to the device not\nbeing able to connect to the Wi-Fi and the website cannot be open successfully. In the case where the\nconnection is successful, the board opens the website. This concludes the setup part of the algorithm,\nwhich is displayed to the user with a blue LED turning on, indicating that the glove is ready to be used.\nIn the case explained beforehand, where the website would not connect properly, the user could not see\nthe blue LED after the roughly 15-16 seconds from booting the board. This can be fixed by manually\npressing the restart button EN. The next part is conducted in the loop function of the board. It starts with\nthe gathering the data from each individual sensor. \n\nThere is a specific function for data of accelerometer,\none for gyroscope and one for flex sensors. Each of these functions is called in the loop function. The\nvalues gathered are checked with the constraints of the dataset, which itself is its own function. Datasets\nare if constraints that hold specific values for every sensor and are equal to one specific word, number\nletter or phrase. When one of these if constraints is fulfilled, they send a string value with the right result.\nThis result is sent to the webpage and gathered there where it would be displayed as a translated gesture.\nSo, since the webpage remains open indefinitely until the user restarts, or the device is disconnected\nfrom the Wi-Fi. If new data was not received, then nothing would change in the webpage displaying the\ntranslate gesture. This was managed through a function that makes the current displayed gesture as\n‘current reading’. Since we know that the loop function keeps running indefinitely, the displayed gesture\nwould print every half a second. This function will display the next gesture when another constraint is\nfulfilled, naming it as ‘new gesture’. So, this constant loop will keep reading new values, checking with\nthe datasets, and updating whenever gestures change, reaching our desired goal.\n\n![image](https://user-images.githubusercontent.com/66420672/174340017-71bb2dfa-e139-4036-aa53-c0a97a11902f.png)\n\n## Enclsoure Design\n\nTo design the enclosure, the PCB dimensions were taken into consideration. The idea behind the\nenclosure is to build a sturdy box that would hold both the power supply and development board. It is\ngoing to have an upper side and a lower side. With the\nalready mentioned specifications on what our goal is, the enclosure will have 2 openings on the eastern\nside, for the USB type A of the 18650-battery shield and one for the micro-USB type B of the ESP32.\nTwo smaller openings would go on the southern side, where the smallest would be for the switch that\nthe battery shield has and the bigger one would be for the micro-USB type B which will be used to\ncharge the battery inside. Since our prototype requires a certain level of conveying stages of functionality\nto the user along with the restart button EN being a crucial part when a problem would arise, an opening\nwould be positioned on top of the development board. This opening would cover the two LEDs and\nmake it possible for the user to manually press the restart button EN. One other opening on the top side\nis going to be positioned on top of the flex sensor markings. This will leave space for the connectors to\nbe soldered onto the PCB and they will be as high as the enclosure. This will protect them from outside\nfactors or accidental pushes. Enclosure was designed with a logo to give it a more personal touch and\nserves as a branding mark. It will have two smaller opening it the thickest side, the nearest with the\nbattery shield, one to recharge the battery using a micro-USB type B, and the other opening will be used\nfor the switch, to turn the battery on or off. The opening for charging has the dimensions of 5x5mm and\nthe opening for the switch has the dimensions of 4x4mm. To have the enclosure sit on the back of the hand, we designed it in a way that it could be strapped\nonto the hand. This was done through two holes on both sides where a strap with the width of 2mm\nwould go inside and around it, connecting with a buckle. The idea was inspired from professional digital\nphotographing equipment that are carried around strapped onto the user’s neck. The furthest holes,\nfurthest away from the battery shield, would be connected around with a strap, without the need of a\nbuckle. This would serve only as s supporting strap that will stand above the thumb. The other hole\nwould be connected together using a buckle that would make the connection around the wrist. This way,\nthe enclosure would stay and rest in place without moving while the user is performing gestures.\nMounting holes for 18650 battery shield and PCB support 2.5M screws and the lower lid can be mounted\nusing holes supporting 3M screws. The design was made using Autodesk’s Fusion 360 and SolidWorks.\nThe thickness of the walls has been put as 2mm across the whole device. The real model was 3D printed\nand mounted together with the other components. Results of the design have been displayed.\n\n![image](https://user-images.githubusercontent.com/66420672/174340115-3955b8e7-8d9b-44ea-ae90-db8df0338df5.png)\n\n![image](https://user-images.githubusercontent.com/66420672/174340133-4bd309d0-7a1a-40fd-a1af-3c66757dec6f.png)\n\n## Results and Testing\n\nThe nature of this PCB, required movement of the board in many directions, making it harder to\nsolder components. It was accomplished from the help of ‘Flexible Helping Hands Soldering Station’.\nThis station made it much easier to solder the connectors and the MPU6050 chip, which proved to be\nharder to keep in place to solder. Soldering iron used has been displayed for full disclosure in Appendix\n5. It was equipped with a conical tip for greater precision during most of the soldering process, and a\nbent tip for the development boards pins since they are quite close with each other. The soldered resulting\nPCB has been shown below:\n\n![image](https://user-images.githubusercontent.com/66420672/174340211-f8400e4b-00cb-467c-b771-5bc8e3683e48.png)\n\nRegarding the printing of the enclosure, both upper and lower lid have been 3D printed. Since the upper lid is a\ncomplete opened area, a lot of supporting fibres were added inside it entirely covering the empty spaces.\nThese supports would be cut out after it had been printed entirely. This was added to not have the printing\nfail. The printing of the upper lid took roughly 12 hours and for the lower lid, around 2 hours. Both were\nprinted successfully with the first try, so there was no need for more printing.\n\n![image](https://user-images.githubusercontent.com/66420672/174340300-2f8ad26d-2334-46ec-af18-fe2571690e66.png)\n\nAs it was mentioned during the Enclosure Design, the lower lid has mounting holes for M3\nscrews. M3 nylon screws have been used to mount up the lower part unto the upper part. For the PCB,\nit is needed 2.5M screws. PCB was placed carefully onto the holes, while taking care of the connectors\nof flex sensors to pop out from the opening specified. Battery shield has been mounted in the\ncorresponding place using 2.5M screws, just like the PCB. Results of mounted components has been displayed.\n\n![image](https://user-images.githubusercontent.com/66420672/174340357-2a9598a6-e6a1-403c-8fb9-e271d52ee879.png)\n\nRegarding the glove, flex sensors have been placed into each finger. Two 5mm cuts were made in each\nfinger, with a 1mm to 2mm distance from the first cut, depending on the length of the finger. The idea\nwas to make a path for the sensor so it would be harder for it to fall off the glove. The cloth material\ndefinitely helps with friction, making it impossible for the\nsensors to fall away. These sensors go around and inside these\ncuts, making them like going in and out small pockets, and\nappearing only by a bit outside of the glove, on these pockets.\nEach of them, has been cold soldered with the male header\npins of the PCB connectors. This way we have concluded in creating\nthe enclosure and glove. After connecting the enclosure and the flex sensors on the glove, the strap and buckle were\nprepared. Both sides of connecting straps have been sewed for greater grip on the hand. The final\ncompleted prototype is shown below.\n\n![image](https://user-images.githubusercontent.com/66420672/174340432-57d2c802-1690-4963-9ec4-967df0c1e23d.png)\n\n![image](https://user-images.githubusercontent.com/66420672/174340459-00f5c431-12f1-4bc6-8dc8-2457442d26a8.png)\n\nAfter finishing the mounting and creating the prototype, we continued to test. This means we\nwould start and test the functionality of the algorithm. Many gestures have been implemented into the\nprototype. For the time being, the glove can translate number 1, 2 3 and 4. It can say the phrases ofPage |68\n‘Hello!’ and ‘I love you!’. It can say the words ‘My’, ‘Name’ and it can translate the letters ‘T’, ‘E’, ‘D’\nand ‘I’. The next set of figures will be a representation of our work, displayed with a proof of hand\ngesture, website reading and a representation of the real gesture in ASL.\n\n![image](https://user-images.githubusercontent.com/66420672/174340517-0693193f-eb43-4ffc-a641-42c6d876f303.png)\n\n![image](https://user-images.githubusercontent.com/66420672/174340550-afc28076-b054-45f6-85db-84c03e0a8aa6.png)\n\n## Summary of work results\n\nAfter testing our prototype, we can safely say that we have reached our goal of developing a\nresponsive, portable, and durable device. The website works as intended and as designed. It gathers and\ntranslates the gestures performed in real time. We have displayed many phrases, words, numbers, and\nletters. The entire project was started from an idea. An idea that was stated as a solution to a problem. It\nwas intended to solve the difficulty gap between hearing-impaired people and the hearing world, by\nmaking their daily life safer and easier. It was meant to surpass the designs of previous projects.\nThe process started by building a block diagram, deciding what relationships the many devices would\nhave with each other. It continued into much extensive research about similar works previously made,\nwhich made the idea clearer. This way we could decide which roads to undertake without spending our\ntime exploring many ideas.\n\nWe constructed the schematic design using the components along adapting each of their parameters\nwith each other. That step made the building and designing the PCB much easier by importing the already\ndone assets onto Altium Designer. The PCB was made while having in mind an idea for the enclosure.\nAlgorithm was developed in a quite simple manner, with many void functions, by making the whole\ncode readable to anyone and efficient. This way, we could focus on performance issues, small fixes on\nthe datasets of gestures, and adding more gestures as features. The data was uploaded in a JSON variable\ninto our Asynchronous server. Enclosure was built later, which was the most challenging part of this\nwhole project. It required many trials and errors.\n\nMany problems arise on designing the openings for the battery shield and the PCB. Looking at the\nwork in a more critical point of view, the opening for the on/off switch of the battery shield could be a\ncouple more millimeters bigger on both dimensions, and so could the opening for the charging microUSB type B. Using SolidWorks for the enclosure proved quite useful because the design was done part\nby part. Each component was designed separately to have a better picture of the idea. The testing revealed\nmany smaller defects of the prototype. The wires can be insulated for a more professional look. The flex\nsensors inside the glove could have been sewed for a better grip, leaving room for smaller improvements.\nAfter the implementation, the device meets its intended requirements and objectives, while achieving\nevery goal set at the beginning of this project. The designed device is extremely simple in construction,\nboth in PCB, enclosure, making it a reliable product with a low-cost production.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrazzledturtle%2Fgesture-recognition-glove","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrazzledturtle%2Fgesture-recognition-glove","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrazzledturtle%2Fgesture-recognition-glove/lists"}