https://github.com/shortthirdman/vibesync
Detecting emotions in video calls with Twilio and AWS Rekognition
https://github.com/shortthirdman/vibesync
aws-rekognition nextjs react-bootstrap reactjs twilio twilio-video typescript
Last synced: about 2 months ago
JSON representation
Detecting emotions in video calls with Twilio and AWS Rekognition
- Host: GitHub
- URL: https://github.com/shortthirdman/vibesync
- Owner: shortthirdman
- License: mit
- Created: 2025-01-01T13:04:38.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-12-11T23:24:50.000Z (6 months ago)
- Last Synced: 2025-12-26T17:52:22.437Z (6 months ago)
- Topics: aws-rekognition, nextjs, react-bootstrap, reactjs, twilio, twilio-video, typescript
- Language: TypeScript
- Homepage:
- Size: 227 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# VibeSync
> [Building a Real-Time Video Emotion Detection App with Twilio Video and AWS Rekognition](https://www.twilio.com/en-us/blog/detect-emotions-video-calls-twilio-aws-rekognition)
Imagine taking virtual meetings and video calls to a whole new level by detecting emotions in real time. By combining the capabilities of [Twilio Video](https://www.twilio.com/docs/video) and [AWS Rekognition](https://aws.amazon.com/rekognition/), you can build a system that analyzes facial expressions and offers deep insights into how participants are truly feeling. The `Video Emotion Detector` app brings this vision to life by merging live video calls with advanced facial analysis. It captures video frames from a participant's webcam, processes them through a canvas element, and sends the data to AWS Rekognition for emotion detection. Emotions like happiness, sadness, or anger are then identified and displayed instantly, with the background color dynamically changing to reflect the most dominant emotion. This powerful integration adds an interactive, empathetic dimension to virtual communication, making video calls not just more engaging, but also more emotionally intelligent.
### Prerequisites
- A `Twilio` account with an Account SID, an API Key and an API Secret
- AWS Account with an `AWS Rekognition` User, Access token, and Access token secret.
- Familiarity with `Next.js`
- Familiarity with `Typescript`
- Familiarity with `React`
- Familiarity with `React-Bootstrap`
### Integrating AWS Rekognition for Emotion Detection
AWS Rekognition is a powerful tool for image and video analysis, offering facial recognition and emotion detection features. Once the user’s webcam feed is available through Twilio Video, we can pass the video frames to AWS Rekognition to analyze emotions. AWS Rekognition detects various emotions such as happy, sad, surprised, angry, and more by analyzing facial expressions.