https://github.com/tucan9389/mlproblem-mlframework-for-ios-template-project
https://github.com/tucan9389/mlproblem-mlframework-for-ios-template-project
Last synced: 6 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/tucan9389/mlproblem-mlframework-for-ios-template-project
- Owner: tucan9389
- License: mit
- Created: 2019-07-21T02:56:11.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2019-08-04T05:50:52.000Z (about 6 years ago)
- Last Synced: 2025-02-09T09:13:32.911Z (8 months ago)
- Language: Swift
- Size: 75.2 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# MLProblem-MLFramework for iOS template project
Anyone can download and change the content of this repo.
This project is XXXX XXXXXXXX on iOS with Core ML.
If you are interested in iOS + Machine Learning, visit [here](https://github.com/motlabs/iOS-Proejcts-with-ML-Models) you can see various DEMOs.
| Live Image | Still Image 1 | Still Image 2 |
| ------------------ | ------------------ | ------------------ |
|  |  |  |
## How it works
> Preparing...
## Requirements
- Xcode 10.2+
- iOS 12.0+
- Swift 5
## Model
### Download Link
> Preparing...
### Model Size, Minimum iOS Version, Input/Output Shape
| Model | Size
(MB) | Minimum
iOS Version | Input Shape | Output Shape |
| ----: | :----: | :----: | :----: | :----: |
| cpm | 2.6 | iOS11 | `[1, 192, 192, 3]` | `[1, 96, 96, 14]` |
| hourhglass | 2 | iOS11 | `[1, 192, 192, 3]` | `[1, 48, 48, 14]` |
### Infernece Time (ms)
| Model vs. Device | XS | XS
Max | XR | X | 8 | 8+ | 7 | 7+ | 6S+ | 6+ |
| ----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: |
| cpm | - | 27 | 27 | 32 | 31 | 31 | 39 | 37 | 44 | 115 |
| hourhglass | - | 6 | 7 | 29 | 31 | 32 | 37 | 42 | 48 | 94 |
### Total Time (ms)
| Model vs. Device | XS | XS
Max | XR | X | 8 | 8+ | 7 | 7+ | 6S+ | 6+ |
| ----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: |
| cpm | - | 39 | 40 | 46 | 47 | 45 | 55 | 58 | 56 | 139 |
| hourhglass | - | 15 | 15 | 38 | 40 | 40 | 48 | 55 | 58 | 106 |
### FPS
| Model vs. Device | XS | XS
Max | XR | X | 8 | 8+ | 7 | 7+ | 6S+ | 6+ |
| ----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: |
| cpm | - | 23 | 23 | 20 | 20 | 21 | 17 | 16 | 16 | 6 |
| hourhglass | - | 23 | 23 | 24 | 23 | 23 | 19 | 16 | 15 | 8 |
## Build & Run
### 1. Prerequisites
### 2. Dependencies
### 3. Code
#### 3.1 Import Vision framework
```swift
import Vision
```
#### 3.2 Define properties for Core ML
```swift
// properties on ViewController
typealias EstimationModel = model_cpm // model name(model_cpm) must be equal with mlmodel file name
var request: VNCoreMLRequest!
var visionModel: VNCoreMLModel!
```
#### 3.3 Configure and prepare the model
```swift
override func viewDidLoad() {
super.viewDidLoad()
visionModel = try? VNCoreMLModel(for: EstimationModel().model)
request = VNCoreMLRequest(model: visionModel, completionHandler: visionRequestDidComplete)
request.imageCropAndScaleOption = .scaleFill
}
func visionRequestDidComplete(request: VNRequest, error: Error?) {
/* ------------------------------------------------------ */
/* something postprocessing what you want after inference */
/* ------------------------------------------------------ */
}
```
#### 3.4 Inference 🏃♂️
```swift
// on the inference point
let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer)
try? handler.perform([request])
```
## Performance Test
### 1. Import the model
You can download cpm or hourglass model for Core ML from [tucan9389/pose-estimation-for-mobile](https://github.com/tucan9389/pose-estimation-for-mobile) repo.
### 2. Fix the model name on [`PoseEstimation_CoreMLTests.swift`](PoseEstimation-CoreMLTests/PoseEstimation_CoreMLTests.swift)
### 3. Run the test
Hit the `⌘ + U` or click the `Build for Testing` icon.
## See also
- [motlabs/iOS-Proejcts-with-ML-Models](https://github.com/motlabs/iOS-Proejcts-with-ML-Models)
: The challenge using machine learning model created from tensorflow on iOS
- [tucan9389/PoseEstimation-CoreML](https://github.com/tucan9389/PoseEstimation-CoreML)