{"id":29628301,"url":"https://github.com/iot-gamer/flutter_flame_finger_tracking_demo","last_synced_at":"2025-07-25T13:02:14.774Z","repository":{"id":305291279,"uuid":"1022491828","full_name":"IoT-gamer/flutter_flame_finger_tracking_demo","owner":"IoT-gamer","description":"A demonstration project showcasing the integration of real-time hand tracking with the Flame game engine in a Flutter application. ","archived":false,"fork":false,"pushed_at":"2025-07-19T07:36:30.000Z","size":1580,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-19T12:09:57.468Z","etag":null,"topics":["computer-vision","flame-engine","flutter","flutter-app","ga","game","handtracking","mediapipe"],"latest_commit_sha":null,"homepage":"","language":"Dart","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IoT-gamer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-19T07:25:45.000Z","updated_at":"2025-07-19T08:49:13.000Z","dependencies_parsed_at":"2025-07-19T12:10:07.308Z","dependency_job_id":"c7d91ef9-a53b-4efb-9e67-df951e53ef67","html_url":"https://github.com/IoT-gamer/flutter_flame_finger_tracking_demo","commit_stats":null,"previous_names":["iot-gamer/flutter_flame_finger_tracking_demo"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/IoT-gamer/flutter_flame_finger_tracking_demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IoT-gamer%2Fflutter_flame_finger_tracking_demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IoT-gamer%2Fflutter_flame_finger_tracking_demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IoT-gamer%2Fflutter_flame_finger_tracking_demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IoT-gamer%2Fflutter_flame_finger_tracking_demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IoT-gamer","download_url":"https://codeload.github.com/IoT-gamer/flutter_flame_finger_tracking_demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IoT-gamer%2Fflutter_flame_finger_tracking_demo/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266270359,"owners_count":23902729,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","flame-engine","flutter","flutter-app","ga","game","handtracking","mediapipe"],"created_at":"2025-07-21T09:00:35.545Z","updated_at":"2025-07-25T13:02:14.755Z","avatar_url":"https://github.com/IoT-gamer.png","language":"Dart","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flutter Flame Hand Tracking Demo\n\n[![MIT License](https://img.shields.io/github/license/IoT-gamer/hand_landmarker)](https://opensource.org/license/MIT)\n\n\u003cimg src=\"docs/assets/demo.gif\" alt=\"drawing\" height=\"200\"/\u003e\n\nA demonstration project showcasing the integration of real-time hand tracking with the Flame game engine in a Flutter application. The application uses the device's front-facing camera to detect hand landmarks and moves a game object based on the position of the user's index finger.\n\n---\n\n## 🚀 Key Features\n\n* **Real-time Hand Tracking**: Utilizes the camera feed to detect and track hand movements.\n* **Flame Engine Integration**: Controls a `Player` component within a Flame game world.\n* **Efficient Processing**: Hand landmark detection is offloaded to a separate isolate using `integral_isolates` to ensure the UI remains smooth and responsive.\n* **State Management**: Employs `flutter_bloc` to manage the state of hand tracking and communicate updates to the game engine.\n* **Cross-Platform (WIP)**: Currently functional on Android. iOS implementation is planned.\n\n## ⚙️ How It Works\n\nThe application follows a clear data flow from camera input to game object movement:\n\n1.  **Camera Service**: The `HandTrackingService` initializes the `camera` plugin and configures it to stream image frames.\n2.  **Isolate Processing**: Each camera frame is passed to a dedicated background isolate.\n3.  **Hand Detection**: Inside the isolate, the `hand_landmarker` package processes the image frame using the MediaPipe Hand Landmarker model to detect hand keypoints.\n4.  **State Emission**: The detected landmark data is sent back to the main isolate, where `HandTrackingCubit` emits a new `HandTrackingSuccess` state containing the hand coordinates.\n5.  **Game Logic**: A `Controller` component within the Flame game listens for state changes from the `HandTrackingCubit`.\n6.  **Player Movement**: Upon receiving a new state with hand data, the `Controller` updates the position of the `Player` component, moving it to correspond with the index finger's location on the screen. A smoothing algorithm (`lerp`) is applied for fluid motion.\n\n## 📂 Project Structure\n\n```bash\nlib/\n├── cubit/\n│   ├── hand_tracking_cubit.dart    # BLoC cubit for managing hand tracking state\n│   └── hand_tracking_state.dart    # States for the cubit (Initial, Loading, Success, Failure)\n├── game/\n│   ├── components/\n│   │   ├── controller.dart         # Flame component that listens to the BLoC and controls the player\n│   │   ├── player.dart             # The player (circle) component in the game\n│   │   └── status.dart             # Text component to show detection status\n│   ├── finger_tracking_game.dart   # The main Flame game class\n│   └── game.dart                   # Main widget that hosts the GameWidget\n├── models/\n│   └── hand_landmark.dart          # Enum for MediaPipe hand landmark points\n├── services/\n│   └── hand_tracking_service.dart  # Service to manage the camera and hand landmarker plugin in an isolate\n└── main.dart                       # Application entry point\n```\n\n## 📦 Core Dependencies\n\n| Dependency          | Version                                                                                             | Description                                                |\n| ------------------- | --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |\n| `flame`             | [![pub package](https://img.shields.io/pub/v/flame.svg)](https://pub.dev/packages/flame)             | A minimalist 2D game engine for Flutter.                   |\n| `flame_bloc`        | [![pub package](https://img.shields.io/pub/v/flame_bloc.svg)](https://pub.dev/packages/flame_bloc)   | Integrates BLoC state management with Flame.               |\n| `hand_landmarker`   | [![pub package](https://img.shields.io/pub/v/hand_landmarker.svg)](https://pub.dev/packages/hand_landmarker) | A Flutter plugin for the MediaPipe Hand Landmarker task.   |\n| `camera`            | [![pub package](https://img.shields.io/pub/v/camera.svg)](https://pub.dev/packages/camera)           | A Flutter plugin for using the device's cameras.           |\n| `flutter_bloc`      | [![pub package](https://img.shields.io/pub/v/flutter_bloc.svg)](https://pub.dev/packages/flutter_bloc) | State management library.                                  |\n| `integral_isolates` | [![pub package](https://img.shields.io/pub/v/integral_isolates.svg)](https://pub.dev/packages/integral_isolates) | Simplifies working with isolates.                          |\n\n## 📱 Platform Support\n\n| Platform | Supported | Tested |\n| :------- | :-------: | :----: |\n| **Android**| ✅        | ✅     |\n| **iOS** | 🚧        | ❌     |\n\n## 🛠️ Setup and Installation\n\n### Prerequisites\n* Flutter SDK\n* An Android device (Minimum SDK version 24).\n\n### Installation\n1.  **Clone the repository:**\n    ```bash\n    git clone [https://github.com/your-username/flutter_flame_finger_tracking_demo.git](https://https://github.com/IoT-gamer/flutter_flame_finger_tracking_demo.git)\n    cd flutter_flame_finger_tracking_demo\n    ```\n\n2.  **Get Flutter packages:**\n    ```bash\n    flutter pub get\n    ```\n\n3.  **Run the application:**\n    ```bash\n    flutter run\n    ```\n\n## 🎮 Usage\n\n1.  Launch the application on a supported device.\n2.  Grant camera permissions when prompted.\n3.  The app will activate the front-facing camera.\n4.  Position your hand in front of the camera. A green circle will appear on the screen.\n5.  Move your index finger to control the movement of the circle.\n\n## ❗ Troubleshooting\n* Ensure you have the necessary permissions to access the camera.\n* If `no hand detected` appears, try adjusting the lighting or moving your hand back from the camera or try slower motion.\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiot-gamer%2Fflutter_flame_finger_tracking_demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiot-gamer%2Fflutter_flame_finger_tracking_demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiot-gamer%2Fflutter_flame_finger_tracking_demo/lists"}