https://github.com/minhphan46/ar_object_detection
📱An React Native app powered by the Viro library that detects and displays images and objects in real-time, presenting lifelike 3D models through augmented reality (AR)🕶️
https://github.com/minhphan46/ar_object_detection
android ar augmented-reality image-detection object-detection react-native typescript viro viro-react
Last synced: 2 months ago
JSON representation
📱An React Native app powered by the Viro library that detects and displays images and objects in real-time, presenting lifelike 3D models through augmented reality (AR)🕶️
- Host: GitHub
- URL: https://github.com/minhphan46/ar_object_detection
- Owner: minhphan46
- Created: 2023-10-20T07:10:32.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-05-19T18:18:11.000Z (about 1 year ago)
- Last Synced: 2025-03-24T12:44:31.437Z (3 months ago)
- Topics: android, ar, augmented-reality, image-detection, object-detection, react-native, typescript, viro, viro-react
- Language: TypeScript
- Homepage:
- Size: 177 MB
- Stars: 13
- Watchers: 1
- Forks: 5
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# AR App with React Native 😎
This project is an AR (Augmented Reality) application developed using React Native, designed to integrate AR into navigation and product detection. The app provides an immersive experience by allowing users to detect products and navigate to them using both 2D maps and 3D AR visualizations.
This is a new [**React Native**](https://reactnative.dev) project, bootstrapped using [`@react-native-community/cli`](https://github.com/react-native-community/cli).\
We use [**Viro library**](https://github.com/viromedia/viro) that detects and displays images and objects in real-time, presenting lifelike 3D models through augmented reality (AR)🕶️
## Technologies
- **React Native**
- **Viro - AR**
- **Mapbox**
- **Redux Toolkit**
- **Mobile Sensors**
# Video Preview
# Getting Started
> **Note**: Make sure you have completed the [React Native - Environment Setup](https://reactnative.dev/docs/environment-setup) instructions till "Creating a new application" step, before proceeding.
## Step 1: Install the dependencies
```bash
# using npm
npm install
# OR using Yarn
yarn install
# if you can't install, try:
npm install --legacy-peer-deps
```
## Step 2: Start your Application
You need to use real device to test or run this app.
```bash
# using npm
npm start
# OR using Yarn
yarn start
```
## Step 3: Open the App on Your Device:
- For iOS: Use Xcode to open the project and run on a simulator or device.
- For Android: Use Android Studio or a connected device.
## Main Features
### Detect Products in AR
- **Product Recognition and Information Display**: The app can scan and recognize products, displaying relevant information in AR.
- **Navigation Using AR**: Users can navigate to the products using both 2D maps and AR 3D paths.
## Detailed Functionality
### Product Recognition and Information Display
- **Product Image Recognition**: Utilizing Viro, the app recognizes product images.
- **Image Dataset Creation**: A dataset of product images is created for recognition.
- **AR Information Card Display**: Once a product is recognized, an information card about the product is displayed in AR.
### Navigation Using AR
- **3D Model Display**: AR models are displayed using Viro.
- **Coordinate Transformation**: Real-world coordinates are converted into AR reference frames.
- **Mapbox Integration in AR**: Mapbox is integrated to provide navigation routes within the AR environment.
### Steps for Shortest Path Navigation to a Product
1. **Data Initialization**: Collect product data and initialize it on Mapbox.
2. **Shortest Path Calculation**: Use Dijkstra's algorithm to find the shortest path from the current location to the product.
3. **Path Visualization**: Draw the navigation path in Viro.
## Challenges
- **Mobile Sensor Optimization**: The current mobile sensors are not fully optimized for AR navigation.
- **Indoor Positioning**: Accurate indoor positioning remains a significant challenge.
## Contributing
Contributions are welcome! Please fork the repository and submit a pull request for any enhancements or bug fixes.