https://github.com/luluw8071/asl-alphabet-detection
American Sign Language Alphabet Detection in Real Time using OpenCV-Mediapipe with EfficientNetB0 in PyTorch
https://github.com/luluw8071/asl-alphabet-detection
american-sign-language deep-neural-networks efficientnetb0 image-classification python pytorch sign-language-recognition streamlit
Last synced: 17 days ago
JSON representation
American Sign Language Alphabet Detection in Real Time using OpenCV-Mediapipe with EfficientNetB0 in PyTorch
- Host: GitHub
- URL: https://github.com/luluw8071/asl-alphabet-detection
- Owner: LuluW8071
- License: mit
- Created: 2024-03-08T15:49:50.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-11-13T05:16:20.000Z (6 months ago)
- Last Synced: 2025-04-06T05:34:30.013Z (about 1 month ago)
- Topics: american-sign-language, deep-neural-networks, efficientnetb0, image-classification, python, pytorch, sign-language-recognition, streamlit
- Language: Python
- Homepage:
- Size: 29.6 MB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# American Sign Language Detection using EfficientNetB0
This project uses the **EfficientNetB0** CNN architecture for image classification of the American Sign Language (ASL) alphabet and 0-9 digits, with real-time detection support.
## Usage
### 1. Install Dependencies
Install all required libraries from the `requirements.txt` file:
```bash
pip install -r requirements.txt
```
### 2. Download Dataset
Download and extract the ASL dataset using the following command:
```bash
gdown https://drive.google.com/uc\?id\=1b0-MLad_AcVvbocCk7RUB2XH5Xbr7L3x
```
```bash
sudo apt install rar
unrar asl.rar
```
### 3. Train the Model
Before training, set up the `COMET_API_KEY` in a `.env` file inside the `neuralnet` directory to log metrics.
To train the model, run:
```bash
python3 ASL-Alphabet-Detection/neuralnet/train.py
```
> Hyperparameter configurations are available in `train.py`.
### 4. Run the Demo
To run the real-time detection demo:
```bash
python3 detect.py
```
> **Note**: If you don't have a webcam, you can use the DroidCam app to turn your mobile phone into a webcam. Logs will be saved in the `action_handler.log` file.
### 5. Pre-trained Model
You can use the pre-trained model `best_model.pth` located in the `assets/` directory to perform inference.
## Experiment Results
| Loss Curves | Accuracy Curves |
|-------------|-----------------|
|  |  |
> The best model was selected based on the highest test accuracy and was trained for 25 epochs, with the best results at epoch 22.
| Train Loss | Test Loss | Train Accuracy | Test Accuracy |
|------------|-----------|----------------|---------------|
| 0.052 | 0.028 | 0.984 | 0.990 |
> [!NOTE]
> Since the number of class labels was large and the test set was randomly sampled, not all labels were included in the evaluation. As a result, some labels may be missing from the confusion matrix.
|Confusion Matrix|
|-|
||
---
Feel free to report any issues you encounter.
Don't forget to ⭐ the repo!