https://github.com/cansik/sonic-touch-localization

Touch localization on a tabletop with the use of multiple microphones.
https://github.com/cansik/sonic-touch-localization

Last synced: 3 months ago
JSON representation

Touch localization on a tabletop with the use of multiple microphones.

• Host: GitHub
• URL: https://github.com/cansik/sonic-touch-localization
• Owner: cansik
• Created: 2016-02-22T16:04:39.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2017-04-09T18:26:43.000Z (about 8 years ago)
• Last Synced: 2025-02-05T06:43:41.937Z (4 months ago)
• Language: Java
• Size: 53.6 MB
• Stars: 3
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Sonic Touch Localization

Touch localization on a tabletop with the use of multiple microphones.

## Idea

A basic interaction is the tapping of interface elements with your finger. On a tabletop touch sensors are expensive and complex to install. The recognition via camera (as Kinect) is unreliable due to the low depth resolution. An alternative is the detection by the noise which is generated by the finger contact with the table surface. For this four microphones are attached to the table surface which then determine the position by latency differences of the sonic.

![Vision](images/experiment_setup.png)

*Vision*

In this project the localisation has been done by time difference on arrival (TDOA).

## Hardware

### Pipeline

The hardware pipelien consists of a table, four microphones, one pre amplifier and an audio interface. All these componets have been evaluated and testet before the experiments.

![Hardware Pipeline](images/hardware.png)

*Hardware Pipeline*

### Experiment Setup

This is the final setup with following hardware components:

* 4 Voice Microphones

* Behringer Ultragain Pro 8 Digital Amplifier

* Saffire Pro 14 Audio Interface

![Experiment Setup](images/setup.png)

*Experiment*

## Software

### Audio Pipeline

The audio pipline is based on the EtherGL multimedia framework. This framework allows to create filters and effects which then are injected into the pipeline.

To smooth the signal filters and dcremove have been used. All the audio signals are stored in a ring buffer which then overwrites the old data.

If a feature is detected the analyze software is going to analyze the delay and runs the positioning algorithms.

![UI](images/audio_pipeline.png)

*Audio Pipeline*

### Feature Detection

The feature detection is done by a simple thresholding approach. If the signal is louder than a given threshold, the software recognizes this as a touch on the table.

![Result](images/feature_detection.png)

*Feature Detection*

### Delay Detection

Three methods were used to detect the delay of two singals. The best results have been achieved with cross-correlation of the two signals.

![Cross correlation](images/cross_delay.png)

*Cross Correlation Delay*

### Algorithms

#### Linear

This linear algorithm uses time shift caluclation to determine the **x** and **y** position of the signal. This is done by shifting the center point of the signal by the delay between two microphones.

![Result](images/linear_algorithm.png)

*Linear Algorithm*

The results were only accurate on the horizontal axis. This is because the two microphones **UL** and **LL** aswell as **UR** and **LR** are too close together to measure a good delay.

![Result](images/linear_spider.png)

*Linear Spider*

#### Diagonal

The diagonal algorithm tries to fix this problem and uses the longest possible way over the table. Again with the center shift method it is possible to find the origin of the signal.

![Result](images/diagonal_algorithm.png)

*Diagonal Algorithm*

It performed very well, even on the vertical axis.

![Result](images/diagonal_spider.png)

*Diagonal Spider*

#### DIWL

DIWL tries to find the origina point by inverting the sonic waves and calculate their relative position to the time. This is an approximative algorithm.

![Result](images/diwl_1.png)

*DIWL Freeze*

At the pont where all sonic waves intersect there should be the origin of the signal.

![Result](images/diwl_2.png)

*DIWL Localisation*

DIWL was a bit disappointing because it could not show its strength in the field test.

![Result](images/diwl_spider.png)

*DIWL Spider*

### Result

In the experiment three test clicks on every point of the table (UL, CL, ...) have been done. The overall accuracy shows how well the algorithms with which delay method worked.

![Result](images/accuracy.png)

*Accuracy of algorithms*

Tapping was not as good as snapping because of table vibrations which then jammed the lag detection.

### Proof of Concept

The software of the poc was splitted up into two windows. One for recording and one for analyzing the results.

![UI](images/software.png)

*Software POC*

## About

*Developed by Florian Bruggisser 2016*