https://github.com/AkinduID/FazeTrak
EyeRiz is a smart webcam capable of face recognition, gesture-based face locking, tracking, and release the lock, developed For the CS3283 - Embedded Systems Project in Semester 5.
https://github.com/AkinduID/FazeTrak
arduino computer-vision cplusplus face-detection gesture-recognition mediapipe opencv pyqt5 python
Last synced: 6 months ago
JSON representation
EyeRiz is a smart webcam capable of face recognition, gesture-based face locking, tracking, and release the lock, developed For the CS3283 - Embedded Systems Project in Semester 5.
- Host: GitHub
- URL: https://github.com/AkinduID/FazeTrak
- Owner: AkinduID
- Created: 2024-07-14T10:22:42.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-02-10T08:32:56.000Z (over 1 year ago)
- Last Synced: 2025-02-10T09:31:02.480Z (over 1 year ago)
- Topics: arduino, computer-vision, cplusplus, face-detection, gesture-recognition, mediapipe, opencv, pyqt5, python
- Language: Python
- Homepage:
- Size: 62.9 MB
- Stars: 0
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# FazeTrak
**Smart face-tracking webcam system with gesture control and ESP32-C3 servo integration**
---
## 📋 Table of Contents
- [Overview](#overview)
- [Features](#features)
- [Project Structure](#project-structure)
- [Hardware Requirements](#hardware-requirements)
- [Software Requirements](#software-requirements)
- [Installation](#installation)
- [Usage](#usage)
- [System Architecture](#system-architecture)
- [Documentation](#documentation)
- [Contributing](#contributing)
---
## 🎯 Overview
FazeTrak is an intelligent webcam system capable of real-time face detection, gesture-based control, and autonomous tracking. Developed for **CS3283 - Embedded Systems Project (Semester 5)**, the system combines computer vision, embedded systems, and robotics to create a self-centering camera platform.
The system uses:
- **Desktop Application** (PyQt5) - Face/gesture detection and servo control
- **ESP32-C3 Microcontroller** - Servo motor driver and command processing
- **Pan-Tilt Mechanism** - Dual servo motors for 2-axis tracking
- **Virtual Camera Integration** - Stream tracked video to Zoom, OBS, etc.
---
## ✨ Features
- ✅ **Real-time Face Detection** - Powered by MediaPipe
- ✅ **Gesture-based Control** - Lock/unlock tracking with hand gestures
- ✅ **Autonomous Tracking** - Servos automatically center face in frame
- ✅ **Virtual Camera Support** - Stream to OBS, Zoom, Teams
- ✅ **Desktop Application** - Full PyQt5 GUI with live preview
---
## 📁 Project Structure
```
Face-Tracking-WebCam/
├── desktop-app/
│ ├── app/
│ │ ├── main.py # PyQt5 GUI entry point
│ │ ├── video_thread.py # Video capture & face tracking
│ │ ├── servo_controller.py # Serial communication with ESP32-C3
│ │ ├── gesture.py # Hand gesture detection
│ │ parameters
│ ├── requirements.txt
│ └── README.md
├── device-firmware/
│ ├── src/
│ │ └── main.cpp # ESP32-C3 servo control firmware
│ ├── platformio.ini
│ └── README.md
├── assets/
│ ├── esp32_c3_supermini.jpeg
│ ├── circuit.png
│ └── ...
├── README.md
└── .gitignore
```
---
## 🔧 Hardware Requirements
- **Microcontroller**: ESP32-C3 SuperMini
- **Servos**: 2× SG90 Pan-Tilt Servos (180° range)
- **Webcam**: Logitech C270 HD (or compatible USB camera)
- **Servo Bracket**: Pan-Tilt mount
- **USB Cable**: Type-C (ESP32-C3 connection)
### Hardware Connections
| ESP32-C3 Pin | Component |
|--------------|-----------|
| GPIO 4 | Pan Servo Signal |
| GPIO 3 | Tilt Servo Signal |
| GND | Servo GND |
| 5V | Servo VCC (via external PSU) |
---
## 💻 Software Requirements
### Desktop Application
- Python 3.8+
- PyQt5
- OpenCV (cv2)
- MediaPipe
- pyvirtualcam
### ESP32-C3 Firmware
- PlatformIO
- ESP32Servo library
---
## 📦 Installation
### Desktop Application Setup
1. **Clone the repository**
```bash
git clone https://github.com/AkinduID/FazeTrak.git
cd FazeTrak/desktop-app
```
2. **Create virtual environment** (recommended)
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
```
3. **Install dependencies**
```bash
pip install -r requirements.txt
```
### ESP32-C3 Firmware Setup
1. **Install PlatformIO**
```bash
pip install platformio
```
2. **Navigate to firmware directory**
```bash
cd device-firmware
```
3. **Build and upload**
```bash
platformio run -t upload
```
---
## 🚀 Usage
### Starting the Desktop Application
```bash
cd desktop-app
python -m app.main
```
**Controls:**
- **Start Tracking** - Begin face detection and servo control
- **Stop Tracking** - Stop the tracking system
- **Hand Gestures**:
- ✋ **Open Palm** → Release face lock
- ✊ **Closed Fist** → Lock face and start tracking
**Visual Feedback:**
- 🔵 **Blue circle** - Raw face detection (noisy)
- 🟡 **Yellow circle** - Kalman Filter estimate (smooth)
- 🟢 **Green rectangle** - Face bounding box
- 🟢 **Green line** - Servo aim vector
---
## 🏗️ System Architecture
```
┌─────────────────────────────────────┐
│ Desktop Application (PyQt5) │
├─────────────────────────────────────┤
│ ┌───────────────────────────────┐ │
│ │ Video Input (Webcam) │ │
│ └──────────────┬────────────────┘ │
│ │ │
│ ┌──────────────▼────────────────┐ │
│ │ Face Detection (MediaPipe) │ │
│ └──────────────┬────────────────┘ │
│ │ │
│ ┌──────────────▼────────────────┐ │
│ │ Servo Control Algorithm │ │
│ └──────────────┬────────────────┘ │
│ │ │
└─────────────────┼────────────────────┘
│ Serial @ 115200 baud
┌────────▼─────────┐
│ ESP32-C3 │
├──────────────────┤
│ Servo Driver │
│ (PWM Generator) │
└────────┬─────────┘
│
┌────────┴──────────┐
│ │
┌──▼──┐ ┌──▼──┐
│ Pan │ │Tilt │
│Servo│ │Servo│
└─────┘ └─────┘
```
---
## 📚 Documentation
For detailed progress and technical documentation, see:
- [GitHub Wiki](https://github.com/AkinduID/FazeTrak/wiki) - Weekly progress and technical details
- [Desktop App README](desktop-app/README.md)
- [Firmware README](device-firmware/README.md)
---
## 🤝 Contributing
Contributions are welcome! Please follow these steps:
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit your changes (`git commit -m 'Add AmazingFeature'`)
4. Push to the branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request