https://github.com/arshtiwari2004/handsfree-os
HandsFree OS is a gesture-based desktop control system that lets users navigate their computer using hand motions alone. Powered by MediaPipe and AI, it enables touch-free actions like opening apps, scrolling, and switching tabs—designed to enhance accessibility for differently-abled users and multitaskers alike.
https://github.com/arshtiwari2004/handsfree-os
firebase firestore-database mediapipe pyautogui python reactjs tensorflow websocket
Last synced: 3 months ago
JSON representation
HandsFree OS is a gesture-based desktop control system that lets users navigate their computer using hand motions alone. Powered by MediaPipe and AI, it enables touch-free actions like opening apps, scrolling, and switching tabs—designed to enhance accessibility for differently-abled users and multitaskers alike.
- Host: GitHub
- URL: https://github.com/arshtiwari2004/handsfree-os
- Owner: ArshTiwari2004
- Created: 2025-04-22T07:02:15.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-05-07T09:05:23.000Z (about 1 year ago)
- Last Synced: 2025-06-16T07:51:40.633Z (about 1 year ago)
- Topics: firebase, firestore-database, mediapipe, pyautogui, python, reactjs, tensorflow, websocket
- Language: JavaScript
- Homepage: https://hands-free-os.vercel.app/
- Size: 175 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# HandsFree OS - Gesture-Controlled Accessibility Interface
An innovative assistive technology that enables computer control through hand gestures, designed for differently-abled users and hands-free interaction.
## 🌟 Key Features
- **Gesture Recognition**: Control your computer with intuitive hand motions
- **Voice Feedback**: Audio confirmation of recognized gestures
- **Customizable Mappings**: Train your own gestures for specific actions
- **Cross-Platform**: Works on Windows, macOS, and Linux
- **Webcam-Based**: No special hardware required
- **Open Source**: Free and adaptable to individual needs
## 🛠️ Technology Stack
### 💻 Frontend
- **React.js** – Component-based UI development
- **Tailwind CSS** – Utility-first modern styling
- **Firebase Authentication** – Secure and scalable user authentication
- **WebSocket** – Real-time bi-directional communication
---
### 🧠 Backend
- **Python 3.7+** – Core backend scripting
- **MediaPipe** – Real-time hand tracking
- **PyAutoGUI** – Programmatic system control
- **Socket.IO** – Real-time communication bridge
## 🚀 Installation Guide
### Prerequisites
- Node.js 16+
- Python 3.7+
- Webcam
- Firebase project (for authentication)
### 1. Frontend Setup
```bash
cd frontend
npm install
cp .env.example .env # Update with your Firebase config
npm start
```
### 2. Python Backend Setup
```bash
cd gesture-detection
pip install -r requirements.txt
python -m gesture_detection.main
```
### 3. Firebase Setup
```bash
Create a Firebase project at firebase.google.com
Enable Google Authentication
Update .env file with your Firebase config
```
### 4. Running the Application
```bash
# Start Python Service (in one terminal):
cd gesture-detection && python -m gesture_detection.main
# Start React Frontend (in another terminal):
cd frontend && npm start
```
## ✋ Default Gestures
| Gesture | Action |
|---------------|--------------------|
| ✋ Open Palm | Show Desktop |
| 👍 Thumbs Up | Volume Up |
| 👎 Thumbs Down | Volume Down |
| 👌 OK Sign | Play/Pause Media |
| ➡️ Swipe Right | Next Tab |
| ⬅️ Swipe Left | Previous Tab |
## Project Structure
```bash
handsfree-os/
├── frontend/ # React application
├── gesture-detection/ # Python gesture service
│ ├── actions/ # System control modules
│ ├── hand_landmarks/ # Gesture processing
│ ├── utils/ # Helper functions
│ └── main.py # Main service entry
└── docs/ # Documentation
```
## 🤝 Contributing
We welcome contributions! Please follow these steps:
Fork the repository
Create your feature branch (git checkout -b feature/AmazingFeature)
Commit your changes (git commit -m 'Add some amazing feature')
Push to the branch (git push origin feature/AmazingFeature)
Open a Pull Request
## ✉️ Contact
- Project Maintainer: Arsh Tiwari
- Mail : arshtiwari12345@gmail.com
### Project Link: The frontend is currently hosted at [HandsFree OS](https://hands-free-os.vercel.app/).