https://github.com/haydenshively/aslan
uses mosse algorithm and TF object detection to adjust graphics based on user's head position, creating a 3D effect
https://github.com/haydenshively/aslan
3d opencv tensorflow vtk
Last synced: about 2 months ago
JSON representation
uses mosse algorithm and TF object detection to adjust graphics based on user's head position, creating a 3D effect
- Host: GitHub
- URL: https://github.com/haydenshively/aslan
- Owner: haydenshively
- License: mit
- Created: 2017-11-17T02:20:12.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2020-07-20T02:37:26.000Z (almost 6 years ago)
- Last Synced: 2025-03-24T21:42:36.753Z (over 1 year ago)
- Topics: 3d, opencv, tensorflow, vtk
- Language: Python
- Homepage:
- Size: 19.4 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Aslan
Aslan combines Visual Toolkit (VTK) Python bindings with Google’s TensorFlow Object Detection library to create the
illusion of 3D on a 2D screen.
Since I wrote most of this code in early 2018, the SOTA model for fast object tracking was MobileNet(V1)-SSD. To make
Aslan, I retrained Google's model on the FDDB faces dataset. Since my laptop didn't have a GPU, this model was unable to
process images from the webcam feed in real time. To fix that, I designed a queue-based multi-threaded framework that
combines ML with a traditional Mosse tracker. Whenever Mosse loses accuracy, its bounding box is reinitialized from the
MobileNet predictions. This allows Aslan to run in real time, even on older hardware.
Once the face's location is known, I use VTK to re-position the camera (with off-axis projection) such that everything
looks 3-Dimensional to the user. This can be done with any VTK scene; I've provided a few demos, including one which displays .STL
CAD files.
Demo video available here: https://www.haydenshively.info/aslan
## Usage
1. Install requirements from `environment.yml` using Anaconda
2. Run `python main.py`
3. To try out different scenes, modify lines 19 and 20 of `main.py`. If you're using a .STL file and the scale doesn't
look right, try modifying line 176 of `scenes.py` (`stl_actor.Set_Scale(...)`)
Traditionally, VTK for Python is pretty poorly documented, so if you're having issues, let me know directly. You're
unlikely to find answers in forums.
## Future Updates
If this repo ever gets any stars, I'll try to update it with command line arguments for STL file usage.