{"id":1220,"url":"https://github.com/dokun1/Lumina","last_synced_at":"2025-08-02T04:31:38.019Z","repository":{"id":48596193,"uuid":"89019477","full_name":"dokun1/Lumina","owner":"dokun1","description":"A camera designed in Swift for easily integrating CoreML models - as well as image streaming, QR/Barcode detection, and many other features","archived":false,"fork":false,"pushed_at":"2022-02-05T16:23:08.000Z","size":58186,"stargazers_count":899,"open_issues_count":14,"forks_count":91,"subscribers_count":40,"default_branch":"master","last_synced_at":"2024-11-30T16:52:15.657Z","etag":null,"topics":["camera","coreml","ios","swift"],"latest_commit_sha":null,"homepage":"https://david.okun.io","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dokun1.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-21T20:02:44.000Z","updated_at":"2024-11-03T15:14:28.000Z","dependencies_parsed_at":"2022-08-27T22:42:24.105Z","dependency_job_id":null,"html_url":"https://github.com/dokun1/Lumina","commit_stats":null,"previous_names":[],"tags_count":54,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dokun1%2FLumina","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dokun1%2FLumina/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dokun1%2FLumina/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dokun1%2FLumina/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dokun1","download_url":"https://codeload.github.com/dokun1/Lumina/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228438934,"owners_count":17920017,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["camera","coreml","ios","swift"],"created_at":"2024-01-05T20:15:41.560Z","updated_at":"2024-12-06T08:31:07.915Z","avatar_url":"https://github.com/dokun1.png","language":"Swift","funding_links":[],"categories":["Hardware","Swift"],"sub_categories":["Camera","Other free courses"],"readme":"\u003cp align=\"center\"\u003e\n\t\u003cimg src=\"./Assets/luminaLogo.png\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://api.travis-ci.org/dokun1/Lumina.svg?branch=master\"\u003e\n        \u003cimg src=\"https://api.travis-ci.org/dokun1/Lumina.svg?branch=master\" alt=\"Travis CI Status\"\u003e\n    \u003c/a\u003e\n\t\u003ca href=\"https://choosealicense.com/licenses/mit/\"\u003e\n\t\t\u003cimg src=\"https://img.shields.io/badge/license-MIT-blue.svg\" alt=\"MIT License\"\u003e\n\t\u003c/a\u003e\n\t\u003ca href=\"https://github.com/RichardLitt/standard-readme\"\u003e\n\t\t\u003cimg src=\"https://img.shields.io/badge/standard--readme-OK-green.svg\" alt=\"Standard README Compliant\"\u003e\n\t\u003c/a\u003e\n\t\u003ca href=\"https://img.shields.io/cocoapods/p/Lumina.svg?style=flat\"\u003e\n\t\t\u003cimg src=\"https://img.shields.io/cocoapods/p/Lumina.svg?style=flat\" alt=\"Platforms\"\u003e\n\t\u003c/a\u003e\n\u003c/p\u003e\n\n----------------\n\nWould you like to use a fully-functional camera in an iOS application in seconds? Would you like to do CoreML image recognition in just a few more seconds on the same camera? Lumina is here to help.\n\n\u003cp align=\"center\"\u003e\n\t\u003ca href=\"https://www.youtube.com/watch?v=8eEAvcy708s\" target=\"_blank\"\u003e\n\t\t\u003cimg src=\"https://img.youtube.com/vi/8eEAvcy708s/0.jpg\"\u003e\n\t\u003c/a\u003e\n\u003c/p\u003e\n\n\nCameras are used frequently in iOS applications, and the addition of `CoreML` and `Vision` to iOS 11 has precipitated a rash of applications that perform live object recognition from images - whether from a still image or via a camera feed.\n\nWriting `AVFoundation` code can be fun, if not sometimes interesting. `Lumina` gives you an opportunity to skip having to write `AVFoundation` code, and gives you the tools you need to do anything you need with a camera you've already built.\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg src=\"./Assets/luminaDemo.gif\"\u003e\n\u003c/p\u003e\n\nLumina can:\n\n- capture still images\n- capture videos\n- capture live photos\n- capture depth data for still images from dual camera systems\n- stream video frames to a delegate\n- scan any QR or barcode and output its metadata\n- detect the presence of a face and its location\n- use any CoreML compatible model to stream object predictions from the camera feed\n\n## Table of Contents\n\n- [Requirements](#requirements)\n- [Background](#background)\n- [Install](#install)\n- [Usage](#usage)\n- [Maintainers](#maintainers)\n- [Contribute](#contribute)\n- [License](#license)\n\n## Requirements\n\n- Xcode 12.0+ (by loading Swift 4 Toolchain)\n- iOS 13.0\n- Swift 5.2\n\n## Background\n\n[David Okun](https://twitter.com/dokun24) has experience working with image processing, and he thought it would be a nice thing to have a camera module that allows you to stream images, capture photos and videos, and have a module that lets you plug in a CoreML model, and it streams the object predictions back to you alongside the video frames.\n\n## Contribute\n\nSee [the contribute file](CONTRIBUTING.md)!\n\nPRs accepted.\n\nSmall note: If editing the README, please conform to the [standard-readme](https://github.com/RichardLitt/standard-readme) specification.\n\n## Install\n\nLumina fully supports Swift Package Manager. You can either add the repo url in your Xcode project or in your Package.swift file under dependencies.\n\n## Usage\n\n**NB**: This repository contains a sample application. This application is designed to demonstrate the entire feature set of the library. We recommend trying this application out.\n\n### Initialization\n\nConsider that the main use of `Lumina` is to present a `ViewController`. Here is an example of what to add inside a boilerplate `ViewController`:\n\n```swift\nimport Lumina\n\n```\n\nWe recommend creating a single instance of the camera in your ViewController as early in your lifecycle as possible with:\n\n```swift\nlet camera = LuminaViewController()\n```\n\nPresenting `Lumina` goes like so:\n\n```swift\npresent(camera, animated: true, completion:nil)\n```\n\n**Remember to add a description for `Privacy - Camera Usage Description` and `Privacy - Microphone Usage Description` in your `Info.plist` file, so that system permissions are handled properly.**\n\n### Logging\n\nLumina allows you to set a level of logging for actions happening within the module. The logger in use is [swift-log](https://github.com/apple/swift-log), made by the [Swift Server Working Group](https://github.com/swift-server) team. The deeper your level of logging, the more you'll see in your console.\n\n**NB**: While Lumina is licensed by the MIT license, [swift-log](https://github.com/apple/swift-log) is licensed by [Apache 2.0](https://github.com/apple/swift-log/blob/master/LICENSE.txt). A copy of the license is also included in the source code.\n\nTo set a level of logging, set the static var on `LuminaViewController` like so:\n\n```swift\nLuminaViewController.loggingLevel = .notice\n```\n\nLevels read like so, from least to most logging:\n\n- CRITICAL\n- ERROR\n- WARNING\n- NOTICE\n- INFO\n- DEBUG\n- TRACE\n\n### Functionality\n\nThere are a number of properties you can set before presenting `Lumina`. You can set them before presentation, or during use, like so:\n\n```swift\ncamera.position = .front // could also be .back\ncamera.recordsVideo = true // if this is set, streamFrames and streamingModel are invalid\ncamera.streamFrames = true // could also be false\ncamera.textPrompt = \"This is how to test the text prompt view\" // assigning an empty string will make the view fade away\ncamera.trackMetadata = true // could also be false\ncamera.resolution = .highest // follows an enum\ncamera.captureLivePhotos = true // for this to work, .resolution must be set to .photo\ncamera.captureDepthData = true // for this to work, .resolution must be set to .photo, .medium1280x720, or .vga640x480\ncamera.streamDepthData = true // for this to work, .resolution must be set to .photo, .medium1280x720, or .vga640x480\ncamera.frameRate = 60 // can be any number, defaults to 30 if selection cannot be loaded\ncamera.maxZoomScale = 5.0 // not setting this defaults to the highest zoom scale for any given camera device\n```\n\n### Object Recognition\n\n**NB:** This only works for iOS 11.0 and up.\n\nYou must have a `CoreML` compatible model(s) to try this. Ensure that you drag the model file(s) into your project file, and add it to your current application target.\n\nThe sample in this repository comes with the `MobileNet` and `SqueezeNet` image recognition models, but again, any `CoreML` compatible model will work with this framework. Assign your model(s) to the framework using the convenient class called `LuminaModel` like so:\n\n```swift\ncamera.streamingModels = [LuminaModel(model: MobileNet().model, type: \"MobileNet\"), LuminaModel(model: SqueezeNet().model, type: \"SqueezeNet\")]\n```\n\nYou are now set up to perform live video object recognition.\n\n### Handling output\n\nTo handle any output, such as still images, video frames, or scanned metadata, you will need to make your controller adhere to `LuminaDelegate` and assign it like so:\n\n```swift\ncamera.delegate = self\n```\n\nBecause the functionality of the camera can be updated at runtime, all delegate functions are required.\n\nTo handle the `Cancel` button being pushed, which is likely used to dismiss the camera in most use cases, implement:\n\n```swift\nfunc dismissed(controller: LuminaViewController) {\n    // here you can call controller.dismiss(animated: true, completion:nil)\n}\n```\n\nTo handle a still image being captured with the photo shutter button, implement:\n\n```swift\nfunc captured(stillImage: UIImage, livePhotoAt: URL?, depthData: Any?, from controller: LuminaViewController) {\n        controller.dismiss(animated: true) {\n    // still images always come back through this function, but live photos and depth data are returned here as well for a given still image\n    // depth data must be manually cast to AVDepthData, as AVDepthData is only available in iOS 11.0 or higher.\n}\n```\n\nTo handle a video being captured with the photo shutter button being held down, implement:\n\n```swift\nfunc captured(videoAt: URL, from controller: LuminaViewController) {\n    // here you can load the video file from the URL, which is located in NSTemporaryDirectory()\n}\n```\n\n**NB**: It's import to note that, if you are in video recording mode with Lumina, streaming frames is not possible. In order to enable frame streaming, you must set `.recordsVideo` to false, and `.streamFrames` to true.\n\nTo handle a video frame being streamed from the camera, implement:\n\n```swift\nfunc streamed(videoFrame: UIImage, from controller: LuminaViewController) {\n    // here you can take the image called videoFrame and handle it however you'd like\n}\n```\n\nTo handle depth data being streamed from the camera on iOS 11.0 or higher, implement:\n```swift\nfunc streamed(depthData: Any, from controller: LuminaViewController) {\n    // here you can take the depth data and handle it however you'd like\n    // NB: you must cast the object to AVDepthData manually. It is returned as Any to maintain backwards compatibility with iOS 10.0\n}\n```\n\nTo handle metadata being detected and streamed from the camera, implement: \n\n```swift\nfunc detected(metadata: [Any], from controller: LuminaViewController) {\n    // here you can take the metadata and handle it however you'd like\n    // you must find the right kind of data to downcast from, whether it is of a barcode, qr code, or face detection\n}\n```\n\nTo handle the user tapping the screen (outside of a button), implement:\n\n```swift\nfunc tapped(from controller: LuminaViewController, at: CGPoint) {\n    // here you can take the position of the tap and handle it however you'd like\n    // default behavior for a tap is to focus on tapped point\n}\n```\n\nTo handle a `CoreML` model and its predictions being streamed with each video frame, implement:\n```swift\nfunc streamed(videoFrame: UIImage, with predictions: [LuminaRecognitionResult]?, from controller: LuminaViewController) {\n  guard let predicted = predictions else {\n    return\n  }\n  var resultString = String()\n  for prediction in predicted {\n    guard let values = prediction.predictions else {\n      continue\n    }\n    guard let bestPrediction = values.first else {\n      continue\n    }\n    resultString.append(\"\\(String(describing: prediction.type)): \\(bestPrediction.name)\" + \"\\r\\n\")\n  }\n  controller.textPrompt = resultString\n}\n```\n\nNote that this returns a class type representation associated with the detected results. The example above also makes use of the built-in text prompt mechanism for Lumina.\n\n### Changing the user interface\n\nTo adapt the user interface to your needs, you can set the visibility of the buttons by calling these methods on `LuminaViewController`:\n\n```swift\ncamera.setCancelButton(visible: Bool)\ncamera.setShutterButton(visible: Bool)\ncamera.setSwitchButton(visible: Bool)\ncamera.setTorchButton(visible: Bool)\n```\n\nPer default, all of the buttons are visible.\n\n### Adding your own controls outside the camera view\n\nFor some UI designs, apps may want to embed `LuminaViewController` within a custom View Controler, adding controls adjacent to the camera view rather than putting all the controls inside the camera view. \n\nHere is a code snippet that demonstrates adding a torch buttons and controlling the camera zoom level via the externally accessible API:\n```swift\nclass MyCustomViewController: UIViewController {\n    @IBOutlet weak var flashButton: UIButton!\n    @IBOutlet weak var zoomButton: UIButton!\n    var luminaVC: LuminaViewController? //set in prepare(for segue:) via the embed segue in the storyboard\n    var flashState = false\n    var zoomLevel:Float = 2.0\n    let flashOnImage = UIImage(named: \"Flash_On\") #assumes an image with this name is in your Assets Library\n    let flashOffImage = UIImage(named: \"Flash_Off\") #assumes an image with this name is in your Assets Library\n\n    override public func viewDidLoad() {\n        super.viewDidLoad()\n\n        luminaVC?.delegate = self\n        luminaVC?.trackMetadata = true\n        luminaVC?.position = .back\n        luminaVC?.setTorchButton(visible: false)\n        luminaVC?.setCancelButton(visible: false)\n        luminaVC?.setSwitchButton(visible: false)\n        luminaVC?.setShutterButton(visible: false)\n        luminaVC?.camera?.torchState = flashState ? .on(intensity: 1.0) : .off\n        luminaVC?.currentZoomScale = zoomLevel\n    }\n\n    override public func prepare(for segue: UIStoryboardSegue, sender: Any?) {\n    if segue.identifier == \"Lumina\" { #name this segue in storyboard\n            self.luminaVC = segue.destination as? LuminaViewController\n        }\n    }\n\n    @IBAction func flashTapped(_ sender: Any) {\n        flashState = !flashState\n        luminaVC?.camera?.torchState = flashState ? .on(intensity: 1.0) : .off\n        let image = flashState ? flashOnImage : flashOffImage\n        flashButton.setImage(image, for: .normal)\n    }\n\n    @IBAction func zoomTapped(_ sender: Any) {\n        if zoomLevel == 1.0 {\n            zoomLevel = 2.0\n            zoomButton.setTitle(\"2x\", for: .normal)\n        } else {\n            zoomLevel = 1.0\n            zoomButton.setTitle(\"1x\", for: .normal)\n        }\n        luminaVC?.currentZoomScale = zoomLevel\n    }\n\n```\n\n## Maintainers\n\n- David Okun [![Twitter Follow](https://img.shields.io/twitter/follow/dokun24.svg?style=social\u0026label=Follow)](https://twitter.com/dokun24) [![GitHub followers](https://img.shields.io/github/followers/dokun1.svg?style=social\u0026label=Follow)](https://github.com/dokun1) \n- Richard Littauer [![Twitter Follow](https://img.shields.io/twitter/follow/richlitt.svg?style=social\u0026label=Follow)](https://twitter.com/richlitt) [![GitHub followers](https://img.shields.io/github/followers/RichardLitt.svg?style=social\u0026label=Follow)](https://github.com/RichardLitt)\n- Daniel Conde [![Twitter Follow](https://img.shields.io/twitter/follow/danielconde7.svg?style=social\u0026label=Follow)](https://twitter.com/danielconde7) [![GitHub followers](https://img.shields.io/github/followers/dconde7.svg?style=social\u0026label=Follow)](https://github.com/dconde7)\n- Zach Falgout [![Twitter Follow](https://img.shields.io/twitter/follow/ZFalgout1.svg?style=social\u0026label=Follow)](https://twitter.com/ZFalgout1) [![GitHub followers](https://img.shields.io/github/followers/ZFalgout.svg?style=social\u0026label=Follow)](https://github.com/ZFalgout)  \n- Gerriet Backer [![Twitter Follow](https://img.shields.io/twitter/follow/gerriet.svg?style=social\u0026label=Follow)](https://twitter.com/gerriet) [![GitHub followers](https://img.shields.io/github/followers/gerriet.svg?style=social\u0026label=Follow)](https://github.com/gerriet)\n- Greg Heo [![Twitter Follow](https://img.shields.io/twitter/follow/gregheo.svg?style=social\u0026label=Follow)](https://twitter.com/gregheo) [![GitHub followers](https://img.shields.io/github/followers/gregheo.svg?style=social\u0026label=Follow)](https://github.com/gregheo)\n\n## License\n\n[MIT](LICENSE) © 2019 David Okun\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdokun1%2FLumina","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdokun1%2FLumina","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdokun1%2FLumina/lists"}