https://github.com/duckysmacky/signsense
An Android app for translating dactyl sign language into text
https://github.com/duckysmacky/signsense
ai android-app dactyl machine-learning russian sign-language translation
Last synced: 3 months ago
JSON representation
An Android app for translating dactyl sign language into text
- Host: GitHub
- URL: https://github.com/duckysmacky/signsense
- Owner: duckysmacky
- License: gpl-3.0
- Created: 2023-11-20T19:14:36.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2025-12-14T15:08:39.000Z (8 months ago)
- Last Synced: 2026-01-27T18:56:04.404Z (6 months ago)
- Topics: ai, android-app, dactyl, machine-learning, russian, sign-language, translation
- Language: Java
- Homepage:
- Size: 247 MB
- Stars: 3
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# SignSense
> An Android app for translating dactyl sign language into text
*because knowing sign language is pretty handy*
## About
Signsense is a mobile application that translates finger spelling sign language (dactyl). It utilizes a custom trained model to recognize dactyl symbols using phone's rear camera. Identified symbols are later translated into plain text by the app and presented to the user.
It was originally developed as a school project and is very much just a proof-of-concept project and there are currently no plans to continue its development. Right now it supports only one language for translation (Russian Dactyl).
There is still a goal of rewriting the whole app in order to optimise load times, improve detection, add support for ASL, enable simultanious detection with multiple models and just overall improve the app.
## Features
Current features:
- 4 menus (Main menu, camera translator, information and settings)
- Live dactyl detection and translation from rear and front cameras
- Information menu with proper guide and credits
- Settings menu to configure detection, translation and app itself
- 2 app themes (Light and Dark)
- 2 interface languages (English and Russian)
- 1 translation language (Only Russian)
## Usage
- When using camera translator, make sure to stand about 1 meter from the signer for proper hand tracking and have the signer's hands visible within the frame, or else the app will not be able to properly track their hands.
- In case of situations with low light levels, use the flashlight function (press lighting icon to toggle).
- After aiming the camera at the signer, the app will immediately track their hands.
- Once the sign will be recognised, it will be shown in the "letter" window and will be added to the queue of "translated word" window.
- Once the app recognises a break between signs, it will mark the word as completed, clear the "translated word" window and add the word to the bottom of the screen, which shows all recently translated words.
- Upon closing the translator this list will be cleared.
## Installation
> [!NOTE]
> The app was only tested on Android versions 8-12, **minimum** supported version is Android 8.
1. Download the APK file from [here](https://www.bit.ly/signsense) or from [releases](https://github.com/duckysmacky/signsense/releases)
2. Open and **install** the apk file
3. Done! The app is now installed on your phone
**If your browser is blocking you from downloading the APK file on mobile:** enable [download from unknown sources](https://www.applivery.com/docs/mobile-app-distribution/android-unknown-sources/) feature in settings
## Bugs and feature requests
The app **might still crash** and **have several bugs** as this release was a bit rushed because of our project presentation. Make sure to submit all bugs to [issues](https://github.com/duckysmacky/signsense/issues), I will try to fix them as soon as possible. Any other feature requests can also be submited there.
## Credits
- [Ducky](github.com/duckysmacky) - lead developer, backend and frontend
- [shroom](github.com/shroomwastaken) - model training and ai specialist