https://github.com/frazzledturtle/gesture-recognition-glove
Bachelor Thesis
https://github.com/frazzledturtle/gesture-recognition-glove
bachelor-thesis embedded-c embedded-networking esp32 javascript-website network-programming
Last synced: 5 months ago
JSON representation
Bachelor Thesis
- Host: GitHub
- URL: https://github.com/frazzledturtle/gesture-recognition-glove
- Owner: FrazzledTurtle
- License: mit
- Created: 2024-09-08T10:46:19.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-01-27T19:42:31.000Z (over 1 year ago)
- Last Synced: 2025-07-17T20:53:56.576Z (12 months ago)
- Topics: bachelor-thesis, embedded-c, embedded-networking, esp32, javascript-website, network-programming
- Language: HTML
- Homepage:
- Size: 13.4 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# Gesture Recognition Glove
*A short intro before starting the article. To anyone who will find themselves reading through this project, please know that, being written around 2020-2021, the algorithm is most likely out-of-date, from the libraries to the dev board used. This is a certain version of what I have written and I've kept everything untouched since then, just as some sort of comparison for myself a long time ago. The algorithm is poorly written and hardware concepts are basic. Hope it helps anyone who's trying to get into embedded systems!*
***The whole prototype will require 5 flex sensors, a development board, an accelerometer,
gyroscope, and a battery shield.*** During the making of this prototype, a wearable glove will be created
that is **responsive, portable, and durable.** It will be able to translate sign language gestures into readable
form for hearing-impaired people by analysing the bend of the fingers and the position of the hand in
space, process it and send messages to the website. The project started by selecting a suitable
development board, a suitable power supply corresponding the voltage input of this controller, selecting
the right sensors, developing the firmware, designing the PCB and enclosure, and building the prototype.
The PCB will rest along with the battery shield onto a custom-made enclosure. This enclosure will rest
on the back of the hand of the user using a strap keep it tight with the hand and a buckle connecting both
ends of the strap. It will have two straps going on the specified holes. One will be resting above the
thumb and the other strap that connects with a buckle, on the wrist. This will lead for the entire prototype
to not be too close to the fingers, making it a suitable place to make the connection through the
connectors.
The focus of the demo prototype will be on reading and analysing hand movement. Since ASL’s gestures require an extensive range of motions, including rotation and complex arm movement, the algorithm of this prototype will have to be exceptionally well written. It will need to capture correct and consistent analog to digital data to have a precise definition for each value. To bring the proposed design as close as possible to becoming a viable product, there should be a certain required level of standards related to safety to follow. As with any electronic equipment, there is a risk of shorting wires, which can result in injury or fire. This means the points where wires are connected with the Flex Sensors will be fully insulated and the enclosure will have to cover most of the PCB and power supply. Sometimes between gestures, there are very subtle differences and minor changes, so this similarity can be read and
analysed from the glove as the same gesture. The complexity of ASL make it challenging to distinguish
and capture differences between gestures. This means the sensors should have a desired sensitivity and
linearity as well as a well-developed software algorithm that detects these subtle changes. In this case,
we were faced with two options on how to proceed and develop the algorithm. One could be using
machine learning, where the glove would be trained over time by performing the same gestures, or by
creating a dataset function holding if constraints with segments of values. With datasets one gesture
would have exact predetermined values for every sensor reading and would be true if those values would
be met. Deciding which method to choose, and what components to use, we had to make several research
on similar products done beforehand and learn from them.
## Block Diagram
Our next very important step would be to design the block diagram, which will be crucial in the
foundation to the realization of this prototype. The block diagram provides a detailed view and
instructions, making it much easier to understand how every component is related to each other. It is
comprised of the Power supply, Input information, Data processing unit, Data acquisition and the
Website. Starting from the main source of power, the Power supply block (see figure below). It consists of
the battery which supplies the gyroscope, accelerometer, flex sensors and development board. The power
supply in our case will give a +5V as input voltage that goes into the voltage regulator of development
board which will drop it to 3.3V. We can see that the red line represents the input voltage throughout the
circuit. It goes from the microcontroller to the voltage divider of the flex sensor and to that sensor, and
it also connects with the accelerometer and gyroscope module.
Flex sensors will be connected with the microcontroller through an ADC pin and the
accelerometer and gyroscope through a I2C pin. The input information block is where the hand gestures
performed are picked up from the sensors as analog data and fed into the controller. This part only holds
outside factors regarding the prototype. The data given to the controller is processed and analysed
through the algorithm and checked with many datasets. Datasets in our case are going to be numerous if
constraints that hold specific values for when a certain sensor reaches that value. If one of these
constraints if fulfilled, then the output information, which is just a String value, will sent to the opened
webpage through the built-in Wi-Fi module of our development board. From the feeding of the data to
the microprocessor and till the moment this data was sent, it’s called Data Processing Block. In the last
stages of the block diagram, the data sent to webpage, arrives, gets pulled from a function in the
algorithm and is displayed.
The website will display the word, phrase, number, or letter depending on what gesture was
performed from the user of the glove. The information will be displayed as a translated gesture, which
is the goal of our prototype. To get access to this website, the user will have to be connected into the
same Wi-Fi that the board is connected to. By entering this local IP address into a browser of our choice,
we will be directed into the webpage, making the whole process of interaction, simple and efficient. The
website will be sending and giving requests with the board constantly.
#### ***ESP32 Devkit V1***
The ESP32 DevKit V1 is one of the development boards created to evaluate the ESPWROOM-32 module. It is based on the ESP32 microcontroller that boasts Wi-Fi, Bluetooth, Ethernet,
and Low Power support all in a single chip. ESP32 is already integrated antenna and RF balun, power
amplifier, low-noise amplifiers, filters, and power management module. The entire solution takes up the
least amount of printed circuit board area. This board is used with 2.4 GHz dual-mode Wi-Fi and
Bluetooth chips by TSMC 40nm low power technology, power, and RF properties best, which is safe,
reliable, and scalable to a variety of applications. The internal flash of the ESP32 module is organized
in a single flash area with pages of 4096 bytes each. There exist two different layouts based on the
presence of BLE support. Power to the ESP32 DevKit v1 is supplied via the on-board USB Micro B
connector or directly via the “VIN” pin. The power source is selected automatically. The device can
operate on an external supply of 5 to 20 volts. If using more than 12V, the voltage regulator may overheat
and damage the device. The recommended range is 5 to 12 volts. The ESP32 DevKit V1 comes with a
serial-to-USB chip on board that allows programming and opening the UART of the ESP32 module. It
can be programmed in different programming environments like Arduino IDE, Espressif IDF (IoT
Development Framework), MicroPython, JavaScript, LUA.
##### **Power Consumption**
Since our goal is to build a portable and reliable prototype, we should consider how much does
the ESP32 consumes so we can have a better understand of its lifetime. ESP32 has different power modes
such as:
- Active Mode: The chip radio is powered on. The chip can receive, transmit, or listen.Page |36
- Modem-sleep Mode: The CPU is operational, and the clock is configurable. The WiFi/Bluetooth baseband and radio are disabled.
- Light-sleep Mode: The CPU is paused. The RTC memory and RTC peripherals, as well as
the ULP coprocessor are running. Any wake-up events will wake u the chip.
- Deep-sleep Mode: Only the RTC memory and RTC peripherals are powered on. Wi-Fi and
Bluetooth connection data are stored in the RTC memory. The ULP coprocessor is
functional.
- Hibernation Mode: The internal 8 MHz oscillator and ULP coprocessor are disabled. The
RTC recovery memory is powered down. Only one RTC timer on the slow clock and
certain RTC GPIOs are active.
While we want to have a website open during the whole time the device is on, and the ESP32
will be constantly communicating with the website, the power mode will be stated as ‘Active’. To have a better understanding about the development boards power consumption in an Active
Mode, we will be including a table displaying the RF Power-Consumption Specifications, from the ESP32 DevKit V1 documentation. We can approximate that our power consumption would be 240 mA, considering that we must
keep the development board and website communicating constantly to update the data. The formula to
find the battery life is as follows:
Battery Life (h) = Battery Capacity (mAh) / Device consumption (mA)
For example, if we were to have a battery with the capacity of 2000 mAh, and our maximum
consumption, in ideal conditions, we would have the device running for 8 hours and 19 minutes straight
without interruptions, which is enough for our project.
### Development Board
Regarding the power supply, a suitable battery was searched for to match the input voltage of the
chosen development board. As said before, our ESP32 requires an input voltage of 5V to 12V, so any
battery that has that output is enough for our project. We will be conducting research for rechargeable
battery options because we would like for the lifetime of the prototype being turn on to be as much as
possible without changing the battery.
Three most common rechargeable battery types include NickelCadmium (NiCd), Nickel-Metal-Hydride (NiMH), and Lithium Polymer (LiPo). Each of these is a good
choice for projects and they all have their benefits and drawbacks. Nickel-Cadmium batteries have a
very high cycle count, and they are typically the cheapest of the three options. However, this type of
battery has a relatively low energy density, so they offer less capacity per cell than the other two types.
On the flip side, NiCd batteries can often deliver very high currents, which makes them perfect for power
tools. Note that NiCd batteries contain toxic heavy metals, which makes them more difficult to recycle.Page |38
Nickel-Metal-Hydride batteries are like NiCd cells. However, NiMH batteries offer an improvement in
capacity over NiCd batteries. One major drawback of NiMH cells is their high self-discharge rate. Li-Po
and Li-Ion batteries have a significantly higher energy density compared to typical NiMH and NiCd
cells. However, this benefit also means that you usually pay more for Li-Po and Li-Ion batteries. Besides
the increased price, lithium batteries always require monitoring by a battery management system, as
these batteries are sensitive to improper handling. Note that Li-Po batteries come in various shapes and
packages, of which some can be very flat. This makes Li-Po cells perfect for handheld devices such as
modern smartphones. Li-Ion batteries often come in larger sizes and are typically used in applications
where size isn’t critical. Note that there is not a single type of Li-Ion or Li-Po battery. Instead, the
category of lithium batteries comprises a whole range of different chemical formulations that all have
their very own pros, cons, and recommended applications.
In our case, a Li-Ion was chosen because they can be recharged many times and are quite stable
and efficient. They tend to have a higher energy density, voltage capacity and lower self-discharge rate
than other rechargeable batteries. This makes for better power efficiency as a single cell has longer
charge retention than other battery types. For the battery to be included and connected with the rest of
the prototype, it should be connected in a certain way. Either through connecting the VIN and GND pins
of the development board or through USB. As mentioned above, our chosen board has a micro-USB
type ‘B’ female. To have a much consistent connection and reliable to connect and disconnect, it was
decided to use a battery shield which will have an USB to make the connection with our board. The size
of this shield should be able to be held on the back of the hand of the user and it should have the option
of being rechargeable. The power supply’s purpose will be simple. It will be connected through USB
type A with the development board with the right input voltage. Since the development board requires
an input voltage of 5 V and has an operational voltage of 3.3V, it will be supplied the exact amount. The
battery shield chosen is the 18650 Battery shield (V3). It was considered for multiple reasons. This shield
is a cheap and portable power supply that can accommodate an ‘unprotected’ single-cell lithium-ion
battery. It also includes a standard USB type ‘A’ female connector, one power on/off switch and a micro-USB
type ‘B’ female for recharging. Its key specifications are as follows:
• Battery charging input: 5V/500mA typical via micro-USB port
• Power supply output: Switched 5VDC via USB type ‘A’ port, 3x 5VDC (VDC= volts
to direct current) (up to 4A) connectors, and 3x3VDC (up to 1A)
• System status indicators: Battery charging and battery charged LEDs
• Battery protection mechanism: Battery overcharge and deep discharge
The background electronics are a clever blend of three requisite circuits a lithium-ion
battery charger circuitry, a battery protection circuitry, and a dc-dc boost converter circuitry. Refer to
the annotated image provided below to get a deep insight on the underlying electronics. In the shield,
the dc - dc boost converter chip is used to deliver 5VDC (4.99V) through the USB ‘A’ socket and 5V
male-header points, generated from the available 3.7V-4.2VDC Li-Ion battery output. The second
channel output, marked as 3V but 3.3V on the dot, is supplied by the parallel combination of three
positive voltage regulator chips. The 3.3VDC supply is derived from the 3.7V-4.2VDC Li-Ion battery
output (not from the 5VDC output)
This device is composed of many components. They will be described by declaring their component
initials which can be check at its Schematic.
- U9 WD01A = Li-Ion battery protection chip
- F1 8205A = Dual N-Channel MOSFET
- U2 TP4056E = Li-Ion battery charger chip
- U7 FP6298 = 4.5A current mode DC-DC boost converter chip
- U4, U5, U6 (XC6206) = Positive voltage regulator chip (3.3V)Page |40
- L1 Green LED = Battery done charging indicator
- L2 Red LED = Battery charging indicator
- S1, S2, S4 = Schottky Diodes
### Flex Sensor
Capturing the bend of the fingers could be done in very few ways. Judging from the research
conducted beforehand on similar projects, flex sensors proved to be a good and efficient way of
measuring the bend of the fingers. Our options regarding the decision on acquiring the right flex sensor
for our prototype were limited. Generally, there are 3 main types of flex sensors: Conductive ink-based
flex sensors, fibre optic flex sensors and capacitive flex sensors. The conductive ink-based flex sensors
are quite common for projects of this nature. Capacitive and fibre optic flex sensors can be used more inPage |41
industrial and professional equipment. Conductive ink-based flex sensors are unipolar sensors, which
means the change of resistance is true only if you bend it in one direction. Bending the sensor on the
other direction does not change its resistance. It uses conductive ink whose resistance varies when bent.
The ink may contain carbon or silver to make it conductive. The spacing between carbon particles are
lager when bent and close when straight which results in a change of resistance.
Regarding the purchasing options, there were only two. 2.2 inches and 4.5 inches flex sensors
were available.
Five 4.5 inches flex sensors were purchased instead of the 2.2 inches because they better
match the length of the fingers. The main computing device is going to be used to configure input and
output pins for each sensor, setup a clock speed, configure communications with the ADC, configure
communications with I2C pins, control sensor data, and send valid data to a Wi-Fi module. Flex Sensors
will be manipulated in the algorithm to decide whether the finger will be closed, open or to determine
exact positions for them.
##### **Power Consumption**
Each Flex Sensor is treated as a variable resistor whose resistance increases which the sensors
bending. Each sensor constitutes a part of its own voltage divider circuit, whose output is processed
through one of the development board’s analog-to-digital converters. The voltage provided from the
controller will be changing because of how much the corresponding finger is bent, which can lead to
problems while using the device. More specifically speaking, if the flex sensor would be changing its
value, the voltage would do as well. To fix this issue, voltage dividers will be placed before the Flex
Sensors. Voltage divider’s purpose in this case is to supply analog voltage values from
divider central point where both two resistors are connected. Voltage dividers only divides power
supply voltage value in proportion of Resistors values. One resistor has fixed value other value is
changing. Imagine two values are equal. Then voltage will be divided by 2.
The flex sensors were measured using a multimeter to determine their values. We agreed to set
the resistance when the sensor is straight as 12.3kΏ and for when it’s bend at 180º, it’s 29kΏ. By
connecting the sensor to a voltage divider circuit, the resulting voltage can be calculated from the
formula:
So, if we wanted to find out the output voltage, taking the flex sensor when it is straight and the
power supply voltage of 3.3V, according to the formula, it would be equal to 1.47V and for when we
are calculating for when flex sensor is bent, we get 0.846V.
Now, to find the current consumption of each flex sensor we would use this formula:
We can see that the circuit consumes very little current making it an energy efficient circuit,
considering that the board will reach around 240mA as consumption. This makes the use of the 10kΏ a
good value to choose for the voltage divider.
Regarding the value of voltage that will serve as an indicator for us to know if the flex sensor has
been bent, we decided that bending the sensor 90º (see Figure 24). This means that we can calculating
the resistance at half-way to our maximum bend value of 180º. This resistance is equal to:
29000Ώ – 12300Ώ = 16700Ώ.
16700Ώ / 2 = 8350Ώ.
12300Ώ + 8350 Ώ= 20650Ώ
We can calculate the voltage of this threshold as follows:
From our results we can understand that the voltage of 1.07V will be understood as a bend in the
flex sensor with that value.
### Accelerometer and Gyroscope
Regarding both accelerometer and gyroscope, the idea was based on the solution in Review 1,
where the team was using a MPU6050 module which contained both an accelerometer module and a
gyroscope module. It combines a 3-axis gyroscope and a 3-axis accelerometer on the same silicon die
together with an onboard Digital Motion Processor (DMP) capable of processing complex 9-axis
algorithms. This will help us to measure acceleration, velocity, orientation, displacement, and many
other motions related parameter of the prototype. The price of this is 5.5€ which made it a very
reasonable and efficient option for the realization of our prototype. It is comprised of 5 pins, INT, SCL,
SDA, GND and VIN.
The INT is the interrupt signal which is an optional pin to connect. It indicates that
data is available for MCU to read. SCL stands for Serial Clock and is used for providing clock pulse for
I2C communication. SDA means Serial Data and is used for transferring data through I2C
communication. VIN will provide the power for this module, can be +3V to +5V depending on the added
power supply. In our case, the power will supply 5V into the board. GND stand for ground and connected
the system to ground. The main features or the MPU6050 are:
• MEMS 3-axis accelerometer and 3-axis gyroscope values combined,
• Power Supply: 3-5V,Page |44
• Communication: I2C protocol,
• Built-in 16-bit ADC provides high accuracy,
• Built-in DMP provides high computational power,
• Built-in Temperature sensor.
A gyroscope to capture the orientation and the angular velocity of the hands. Gyroscope will prove
useful for certain gestures that will require rotary movement in space. Accelerometer will be used to
measure the acceleration of the hand, which will be useful for gestures of phrases or words that need
stretched movement of the hand in space.
## Schematic Diagram
The first step taken for the realisation on this prototype was to design the schematic by placing
the components on a simulated breadboard to have a clearer view of how everything would be connected.
This proved to be quite helpful as a step before placing the components a real breadboard and start
experimenting. was designed using Altium Designer. This would
prove very useful, because the schematic would be imported easily for the next step of creating the PCB
using the same software. In the schematic, we can see how the components are connected with each
other. The flex sensors are shows as variable resistors with the names of Flex 1, Flex 2, Flex 3, Flex 4,
and Flex 5. Each sensors had their resistances measured beforehand to decide their value without any
force put into. This proved an approximated value of 12kΏ. Each of these sensors is connected to their
own voltage divider. These resistances have been assigned a value of 10kΏ. Each flex sensor has been
connected to an ADC pin of the ESP32. These pins are GPIO34, GPIO 35, GPIO 33, GPIO 32 and
GPIO39. We can also see in the schematic design that the MPU6050 module is connected without
development board as well.
The ESP32 has 6 Analog-To-Digital pins assigned as ADC1 and 9 Analog-To-Digital pins
assigned as ADC2. Since our goal is to send the data gathered to an opened webpage, this means that
the flex sensors would be able to connect to ADC1 pins only. From the schematic, we can
also see that the MPU6050 module is connected with the board to the specified pins. In the Pinout Figure
of the ESP32, we can see that GPIO21 and GPIO22 have the following names assigned: I2C SDA and
I2C SCL respectively. Therefore, they are connected to the pins of the same names of the module. Every
VIN pin has been connected and so has the GND pins. It is displayed that the input voltage of the circuit
will be 3.3V as is the operating voltage of the development board. The 18650 battery in-use are the
18650 US18650VTC4 Li-Ion 3.7V 2100mAh 30A.
The ESP32’s lifetime on battery power has not been
tested, but as an approximation it could last around 4 months without charging the battery. Since the
board will be awake, it will need to be on during the whole time. Our main objective is to enable the user
of the glove to be able to fully communicate with another person through the use of a website, which
will serve as a lobby where anyone can connect with. This website will be run on the development board
at the same time it is reading the sensor data. As long as the user of the glove has a mobile phone and
internet mobile data, it will make the communication much easier. They will be taken to a specific IPPage |47
address where this webpage is opened. This was an option we explored as a very efficient one, amongst
other options. We could have developed an easy-to-use mobile application that would’ve performed the
same tasks, but it opened the way to new problems. Such as for what reason would someone download
this app, taking into consideration the chances a person has to meet someone with a hearing impairment.
Even if they did, it would’ve made it more difficult to get the message across to download it in order to
communicate. How would someone know the existence of this app? What if in an extreme situation,
there was a need for the interaction of a person and a hearing-impaired person? What if the person who
will need to get the app does not have enough internet to download it? All these obstacles made the
development of an app obsolete, as it would lead to more headaches and annoyances throughout the
daily life. We decided to research on this matter as well as it is very important how we will be displaying
the translated gesture.
## PCB Design
From the schematic design, all the connections, components and names of components have been
imported to create the PCB of this prototype. The PCB was designed while have the enclosure design in
mind. This means that to properly design the PCB, the dimensions should’ve have been decided before
creating it. Therefore, the dimensions of this PCB are 70x50mm. According to the design of the
enclosure, the longest part would be placing the 18650-battery shield which would be very near the PCB.
The PCB’s dimensions resemble the back of the hand, from the end of the wrist till the beginning of the
fingers. PCB has two layers, a top and bottom one. It has a top and bottom overlay with the name of the
author and its logo and the logo of the university on the back. 4 mounting holes have been put on the
four corners with the sizes of 2.5M screws. Every resistor has their name on the top overlay, just like the
development board and MPU6050 module. The PCB has a common ground, which means the ground
pins are not connected through route connections but go through the same layer. This is a very effective
tool to minimize the routes through the PCB, making less chaotic. Via stitching was used to tie together
large copper areas on different layers to create a strong vertical connection through the board structure.
This helps maintain a low impedance and short return loops. The width of the board was decided to be
a standard 1.6 mm. The PCB was designed and ordered through PCBWay, and it arrived surprisingly
quick, in around 5 days. When ordering our PCB, Gerger Files were generated and the Drill Holes files
that are necessary for the manufacturing and development of our PCB.
The resulting PCB that was ordered and shipped. PCB was ordered twice. The first time’s
resulting PCBs had two main mistakes. The place where the MPU6050 would be soldered into, was too
close to the development board and Flex sensor 5 was connected with the wrong the ADC pin of the
board. It was connected to a ADC2 pin, the ESP32’s ADC2 pins
cannot be used while using the Wi-Fi module. Redesigning the PCB, the stitching holes were kept the
same. The holes for the MPU6050 have been moved up by 3 mm leaving more space for it and the Flex
Sensor 5 was connected with GPIO 39 of the ESP32.
## Software Framework
After designing the PCB, we put our attention to create a very efficient algorithm that would
gather data, analyse them, process them, check them, open a website, create a webpage, and update the
values. In this chapter, we are going through the Software Framework, which explains the logical idea
of the whole algorithm. From the extensive research, it was decided that the prototype will be working
using a dataset in its algorithm, discarding completely the idea of including machine learning. We will
be exploring the ESP32 and its entire capabilities while focusing on speed of transmission of data.
The framework starts from booting the microcontroller. As soon as it
is turned on, it will initialize the Accelerometer and Gyroscope sensors. This means, a function will
perform a checking statement to verify whether MPU6050 is connected or not. If it’s not, then the
MPU6050’s LED won’t turn on. In this situation the main board can be restarted by pressing the EN
Button. If the problem is persistent, then this means there is a problem with the connection between the
main board and the sensor module. This explains the significance to this simple function that is critical
to the communication between the device and the user as it indicates the state of functionality. It is very
important to have visual representation of the status of the device the user is using. This function is one
of the few functions that were added to serve as debugging functions in case problem would arise.
Whenever the board is booted, a red LED is turned on to indicate it.
The next part is calibration of the
flex sensors. Here the user has 15 seconds from the boot of the board to open and close their fingers to
calibrate the flex sensors values. The ESP32s ADC pins gathering of data fluctuate extensively, with the
values changing for the same position of the finger every time the board is booted. This proved that the
specified maximum and minimum values of flex sensors resistance would be incorrect. The
measurements would be less than the minimum and more than the maximum than the measurements
conducted with a multimeter for each flex sensor. Calibrating the flex sensor would give each flex sensor
their individual maximum and minimum, balancing out the values differently each time the board would
boot.
After taking care of every sensor the prototype has, the next step would be to open the website.
The board has been given specific credentials to connect to a Wi-Fi in its surrounding. It will scan the
area and if it finds the same name of the router or switch device, it will connect to it, putting in the
password provided as well. In this stage, the board has successfully initialized a Transmission Control
Protocol (TCP) server. This is a connection-oriented communication protocol that facilitates the
exchange of messages between computing devices in a network. If the server has been initialized, then
the board connects to it. From experimenting, it has happened that the device may not find the specified
Wi-Fi, or there may be a problem with the transmission of the Wi-Fi. This issue leads to the device not
being able to connect to the Wi-Fi and the website cannot be open successfully. In the case where the
connection is successful, the board opens the website. This concludes the setup part of the algorithm,
which is displayed to the user with a blue LED turning on, indicating that the glove is ready to be used.
In the case explained beforehand, where the website would not connect properly, the user could not see
the blue LED after the roughly 15-16 seconds from booting the board. This can be fixed by manually
pressing the restart button EN. The next part is conducted in the loop function of the board. It starts with
the gathering the data from each individual sensor.
There is a specific function for data of accelerometer,
one for gyroscope and one for flex sensors. Each of these functions is called in the loop function. The
values gathered are checked with the constraints of the dataset, which itself is its own function. Datasets
are if constraints that hold specific values for every sensor and are equal to one specific word, number
letter or phrase. When one of these if constraints is fulfilled, they send a string value with the right result.
This result is sent to the webpage and gathered there where it would be displayed as a translated gesture.
So, since the webpage remains open indefinitely until the user restarts, or the device is disconnected
from the Wi-Fi. If new data was not received, then nothing would change in the webpage displaying the
translate gesture. This was managed through a function that makes the current displayed gesture as
‘current reading’. Since we know that the loop function keeps running indefinitely, the displayed gesture
would print every half a second. This function will display the next gesture when another constraint is
fulfilled, naming it as ‘new gesture’. So, this constant loop will keep reading new values, checking with
the datasets, and updating whenever gestures change, reaching our desired goal.
## Enclsoure Design
To design the enclosure, the PCB dimensions were taken into consideration. The idea behind the
enclosure is to build a sturdy box that would hold both the power supply and development board. It is
going to have an upper side and a lower side. With the
already mentioned specifications on what our goal is, the enclosure will have 2 openings on the eastern
side, for the USB type A of the 18650-battery shield and one for the micro-USB type B of the ESP32.
Two smaller openings would go on the southern side, where the smallest would be for the switch that
the battery shield has and the bigger one would be for the micro-USB type B which will be used to
charge the battery inside. Since our prototype requires a certain level of conveying stages of functionality
to the user along with the restart button EN being a crucial part when a problem would arise, an opening
would be positioned on top of the development board. This opening would cover the two LEDs and
make it possible for the user to manually press the restart button EN. One other opening on the top side
is going to be positioned on top of the flex sensor markings. This will leave space for the connectors to
be soldered onto the PCB and they will be as high as the enclosure. This will protect them from outside
factors or accidental pushes. Enclosure was designed with a logo to give it a more personal touch and
serves as a branding mark. It will have two smaller opening it the thickest side, the nearest with the
battery shield, one to recharge the battery using a micro-USB type B, and the other opening will be used
for the switch, to turn the battery on or off. The opening for charging has the dimensions of 5x5mm and
the opening for the switch has the dimensions of 4x4mm. To have the enclosure sit on the back of the hand, we designed it in a way that it could be strapped
onto the hand. This was done through two holes on both sides where a strap with the width of 2mm
would go inside and around it, connecting with a buckle. The idea was inspired from professional digital
photographing equipment that are carried around strapped onto the user’s neck. The furthest holes,
furthest away from the battery shield, would be connected around with a strap, without the need of a
buckle. This would serve only as s supporting strap that will stand above the thumb. The other hole
would be connected together using a buckle that would make the connection around the wrist. This way,
the enclosure would stay and rest in place without moving while the user is performing gestures.
Mounting holes for 18650 battery shield and PCB support 2.5M screws and the lower lid can be mounted
using holes supporting 3M screws. The design was made using Autodesk’s Fusion 360 and SolidWorks.
The thickness of the walls has been put as 2mm across the whole device. The real model was 3D printed
and mounted together with the other components. Results of the design have been displayed.
## Results and Testing
The nature of this PCB, required movement of the board in many directions, making it harder to
solder components. It was accomplished from the help of ‘Flexible Helping Hands Soldering Station’.
This station made it much easier to solder the connectors and the MPU6050 chip, which proved to be
harder to keep in place to solder. Soldering iron used has been displayed for full disclosure in Appendix
5. It was equipped with a conical tip for greater precision during most of the soldering process, and a
bent tip for the development boards pins since they are quite close with each other. The soldered resulting
PCB has been shown below:
Regarding the printing of the enclosure, both upper and lower lid have been 3D printed. Since the upper lid is a
complete opened area, a lot of supporting fibres were added inside it entirely covering the empty spaces.
These supports would be cut out after it had been printed entirely. This was added to not have the printing
fail. The printing of the upper lid took roughly 12 hours and for the lower lid, around 2 hours. Both were
printed successfully with the first try, so there was no need for more printing.
As it was mentioned during the Enclosure Design, the lower lid has mounting holes for M3
screws. M3 nylon screws have been used to mount up the lower part unto the upper part. For the PCB,
it is needed 2.5M screws. PCB was placed carefully onto the holes, while taking care of the connectors
of flex sensors to pop out from the opening specified. Battery shield has been mounted in the
corresponding place using 2.5M screws, just like the PCB. Results of mounted components has been displayed.
Regarding the glove, flex sensors have been placed into each finger. Two 5mm cuts were made in each
finger, with a 1mm to 2mm distance from the first cut, depending on the length of the finger. The idea
was to make a path for the sensor so it would be harder for it to fall off the glove. The cloth material
definitely helps with friction, making it impossible for the
sensors to fall away. These sensors go around and inside these
cuts, making them like going in and out small pockets, and
appearing only by a bit outside of the glove, on these pockets.
Each of them, has been cold soldered with the male header
pins of the PCB connectors. This way we have concluded in creating
the enclosure and glove. After connecting the enclosure and the flex sensors on the glove, the strap and buckle were
prepared. Both sides of connecting straps have been sewed for greater grip on the hand. The final
completed prototype is shown below.
After finishing the mounting and creating the prototype, we continued to test. This means we
would start and test the functionality of the algorithm. Many gestures have been implemented into the
prototype. For the time being, the glove can translate number 1, 2 3 and 4. It can say the phrases ofPage |68
‘Hello!’ and ‘I love you!’. It can say the words ‘My’, ‘Name’ and it can translate the letters ‘T’, ‘E’, ‘D’
and ‘I’. The next set of figures will be a representation of our work, displayed with a proof of hand
gesture, website reading and a representation of the real gesture in ASL.
## Summary of work results
After testing our prototype, we can safely say that we have reached our goal of developing a
responsive, portable, and durable device. The website works as intended and as designed. It gathers and
translates the gestures performed in real time. We have displayed many phrases, words, numbers, and
letters. The entire project was started from an idea. An idea that was stated as a solution to a problem. It
was intended to solve the difficulty gap between hearing-impaired people and the hearing world, by
making their daily life safer and easier. It was meant to surpass the designs of previous projects.
The process started by building a block diagram, deciding what relationships the many devices would
have with each other. It continued into much extensive research about similar works previously made,
which made the idea clearer. This way we could decide which roads to undertake without spending our
time exploring many ideas.
We constructed the schematic design using the components along adapting each of their parameters
with each other. That step made the building and designing the PCB much easier by importing the already
done assets onto Altium Designer. The PCB was made while having in mind an idea for the enclosure.
Algorithm was developed in a quite simple manner, with many void functions, by making the whole
code readable to anyone and efficient. This way, we could focus on performance issues, small fixes on
the datasets of gestures, and adding more gestures as features. The data was uploaded in a JSON variable
into our Asynchronous server. Enclosure was built later, which was the most challenging part of this
whole project. It required many trials and errors.
Many problems arise on designing the openings for the battery shield and the PCB. Looking at the
work in a more critical point of view, the opening for the on/off switch of the battery shield could be a
couple more millimeters bigger on both dimensions, and so could the opening for the charging microUSB type B. Using SolidWorks for the enclosure proved quite useful because the design was done part
by part. Each component was designed separately to have a better picture of the idea. The testing revealed
many smaller defects of the prototype. The wires can be insulated for a more professional look. The flex
sensors inside the glove could have been sewed for a better grip, leaving room for smaller improvements.
After the implementation, the device meets its intended requirements and objectives, while achieving
every goal set at the beginning of this project. The designed device is extremely simple in construction,
both in PCB, enclosure, making it a reliable product with a low-cost production.