https://github.com/virgiledjimgou/lyna
Lyna is a hybrid AI-powered application that helps car users understand their vehicle’s interior using real-time object detection, enriched explanations (text/audio), and AR-based overlays. This project is built for technicians, rental users, or drivers who want to learn about the cockpit, dashboard symbols.
https://github.com/virgiledjimgou/lyna
3d-engine artificial-intelligence asp-net-core augmented-reality-application csharp digital hmi iot-application mqtt-service treejs uml-diagram virtual vue3 yolov8
Last synced: 9 months ago
JSON representation
Lyna is a hybrid AI-powered application that helps car users understand their vehicle’s interior using real-time object detection, enriched explanations (text/audio), and AR-based overlays. This project is built for technicians, rental users, or drivers who want to learn about the cockpit, dashboard symbols.
- Host: GitHub
- URL: https://github.com/virgiledjimgou/lyna
- Owner: VirgileDjimgou
- Created: 2020-07-09T13:36:58.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2025-07-02T14:59:22.000Z (12 months ago)
- Last Synced: 2025-07-02T15:40:58.872Z (12 months ago)
- Topics: 3d-engine, artificial-intelligence, asp-net-core, augmented-reality-application, csharp, digital, hmi, iot-application, mqtt-service, treejs, uml-diagram, virtual, vue3, yolov8
- Language: Vue
- Homepage:
- Size: 25.1 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# 🚘 Lyna – Augmented Reality Assistant for In-Car Intelligence
**Lyna** is a hybrid AI-powered assistant designed to help users identify and understand the components inside a vehicle using their smartphone camera. By leveraging real-time object detection and an augmented reality overlay, Lyna provides instant feedback about dashboard symbols, control buttons, and cockpit elements — making car interaction intuitive for everyone.
## 🎯 Project Overview
Lyna combines:
- 📸 Real-time detection of interior car elements (steering wheel, gear shift, AC, warning lights, etc.)
- 🧠 AI-powered backend (YOLOv8) to identify visual components from live camera input
- 💬 Enriched information displayed via tooltips and audio hints (text-to-speech)
- 🧩 Modular hybrid architecture with **Flask (AI)**, **ASP.NET Core (API)**, and **Vue 3 + Capacitor (Frontend)**
## 🧠 Architecture Overview
```plaintext
[Vue 3 + Capacitor Mobile App]
↓ (camera frame)
[Flask + YOLOv8 Detection API]
↓ (detected object labels)
[ASP.NET Core API + MongoDB]
↓ (descriptive metadata, audio, icons)
[Augmented Reality Overlay + TTS Output]
```
## 🧩 Core Modules
### 📱 Mobile App (Vue 3 + Capacitor)
- Live camera feed
- Frame capture and API calls
- 2D/3D overlays (labels + tooltips)
- Audio output via Web Speech API or Capacitor plugin
### 🧠 AI Detection (Flask + YOLOv8)
- Receives camera frames (JPEG)
- Runs YOLOv8 inference
- Returns bounding boxes + class labels
### 🧰 Metadata API (ASP.NET Core)
- Exposes detailed metadata for detected objects
- Handles multilingual text and audio support
- Connects to MongoDB or serves from enriched JSON
## 📦 Example Object Metadata
```json
{
"id": "gear_shift",
"name": "Gear Shift Lever",
"descriptionShort": "Used to change gears.",
"descriptionLong": "This lever allows the driver to select driving modes: Drive (D), Neutral (N), Reverse (R), and Park (P). Some models also support manual or
sport mode.",
"models": ["Mazda 3", "Toyota Corolla"],
"icon": "gear.svg",
"audio": {
"fr": "gear_shift_fr.mp3",
"en": "gear_shift_en.mp3"
}
}
```
## 📂 Folder Structure
Lyna-ar/
├── client-app/ # Vue 3 + Capacitor mobile frontend
├── backend-ai/ # Flask + YOLOv8 inference API
├── backend-core/ # ASP.NET Core API (data + metadata)
├── shared-data/ # JSON, icons, audio resources
├── docker-compose.yml # Docker orchestration for all services
└── README.md # Project documentation
## 🚀 MVP Roadmap
| Week | Deliverables |
|------|------------------------------------------------------------------------------|
| 1 | Vue + Capacitor app with live camera preview |
| 2 | Flask backend with YOLOv8 model and /detect endpoint |
| 3 | ASP.NET Core API with enriched vehicle metadata |
| 4 | Integration, audio playback, packaging as APK |