https://github.com/oveddan/presence

The code for Presence - a kinetic sculpture that detects a viewer's gaze in real-time using a neural network and moves in response to the gaze
https://github.com/oveddan/presence

Last synced: 17 days ago
JSON representation

The code for Presence - a kinetic sculpture that detects a viewer's gaze in real-time using a neural network and moves in response to the gaze

• Host: GitHub
• URL: https://github.com/oveddan/presence
• Owner: oveddan
• Created: 2017-11-06T22:04:45.000Z (about 7 years ago)
• Default Branch: master
• Last Pushed: 2018-02-22T02:10:12.000Z (over 6 years ago)
• Last Synced: 2024-10-11T01:23:53.461Z (about 1 month ago)
• Language: Jupyter Notebook
• Homepage: http://www.danioved.com/blog/posts/presence/
• Size: 59.7 MB
• Stars: 22
• Watchers: 5
• Forks: 2
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Presence

Presence is a kinetic sculpture that detects a viewer's gaze in real-time using a convolutional neural-network and moves in response to the gaze

It uses a webcam and the pre-trained model from [Eye Tracking for Everyone](http://gazecapture.csail.mit.edu/) to detect

where users are gazing, and moves in the direction of the gaze.

## Key files/documentation

**Full documentation coming soon**

* http://www.danioved.com/blog/posts/presence/ - blog posts which document the process 

* [The jupyter notebook](https://github.com/oveddan/presence/blob/master/notebooks/Predicting%20Gaze%20with%20Eye%20Tracking%20for%20Everyone.ipynb) that walks through how features are extracted in opencv and fed through the neural network to obtain a gaze prediction.

* [features.py](/features.py) - the code to extract features from input images using opencv

* [gaze.py](/gaze.py) - the code to feed the features through the pre-trained neural network and get a gaze prediction

* [processing_sketches/GazeFromStream3d](/processing_sketches/GazeFromStream3d) - processing sketch to render a simulation of the columns in 3d, and send servo positions over serial to the kinetic sculpture.

## Requirements

For gaze detection:

* python

* caffe

* opencv

To render simulations of the servo positions:

* processing