https://github.com/thohemp/nitec
NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction (WACV24)
https://github.com/thohemp/nitec
attention computer contact ego-vision eye focus gaze human human-robot-collaboration human-robot-interaction interaction mutual of pytorch robot vision visual wacv
Last synced: 3 months ago
JSON representation
NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction (WACV24)
- Host: GitHub
- URL: https://github.com/thohemp/nitec
- Owner: thohemp
- License: mit
- Created: 2023-11-01T12:09:36.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-07-17T08:00:27.000Z (over 1 year ago)
- Last Synced: 2025-04-01T21:48:13.829Z (7 months ago)
- Topics: attention, computer, contact, ego-vision, eye, focus, gaze, human, human-robot-collaboration, human-robot-interaction, interaction, mutual, of, pytorch, robot, vision, visual, wacv
- Language: Python
- Homepage:
- Size: 418 KB
- Stars: 16
- Watchers: 1
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
#
**NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction (WACV24)**
## **Citing**
If you find our work useful, please cite the paper:
```BibTeX
@INPROCEEDINGS{10484276,
author={Hempel, Thorsten and Jung, Magnus and Abdelrahman, Ahmed A. and Al-Hamadi, Ayoub},
booktitle={2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
title={NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction},
year={2024},
pages={4425-4434},
doi={10.1109/WACV57701.2024.00438}}
```
##
**Paper**
> [Thorsten Hempel, Magnus Jung, Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "NITEC: Versatile Hand-Annotated Eye Contact Dataset for Ego-Vision Interaction", *WACV 2024*.](https://openaccess.thecvf.com/content/WACV2024/papers/Hempel_NITEC_Versatile_Hand-Annotated_Eye_Contact_Dataset_for_Ego-Vision_Interaction_WACV_2024_paper.pdf)
##
**Abstract**
>Eye contact is a crucial non-verbal interaction modality and plays an important role in our everyday social life. While humans are very sensitive to eye contact, the capabilities of machines to capture a person's gaze are still mediocre. We tackle this challenge and present NITEC, a hand-annotated eye contact dataset for ego-vision interaction. NITEC exceeds existing datasets for ego-vision eye contact in size and variety of demographics, social contexts, and lighting conditions, making it a valuable resource for advancing ego-vision-based eye contact research. Our extensive evaluations on NITEC demonstrate strong cross-dataset performance, emphasizing its effectiveness and adaptability in various scenarios, that allows seamless utilization to the fields of computer vision, human-computer interaction, and social robotics. We make our NITEC dataset publicly available to foster reproducibility and further exploration in the field of ego-vision interaction.
#
Quick Usage:
```sh
pip install face_detection@git+https://github.com/elliottzheng/face-detection
pip install nitec
```
Example usage:
```py
from nitec import NITEC_Classifier, visualize
import cv2
nitec_pipeline = NITEC_Classifier(
weights= CWD / 'models' / 'nitec_rs18_e20.pth',
device=torch.device('cuda') # or 'cpu'
)
cap = cv2.VideoCapture(0)
_, frame = cap.read()
# Process frame and visualize
results = nitec_pipeline.predict(frame)
frame = visualize(frame, results, confidence=0.5)
```
#
Train / Test
## NITEC Dataset
Prepare the dataset as explained [ here](data/README.MD).
## Snapshots
Download from here: https://drive.google.com/drive/folders/1zc6NZZ6yA4NJ52Nn0bgky1XpZs9Z0hSJ?usp=sharing
## Train
```py
python train.py \
--gpu 0 \
--num_epochs 50 \
--batch_size 64 \
--lr 0.0001 \
```
## Test
```py
python test.py \
--snapshot models/nitec_rs18_20.pth \
--gpu 0 \
```