{"id":13990249,"url":"https://github.com/alexmx/ios-ui-automation-overview","last_synced_at":"2025-07-22T12:31:45.579Z","repository":{"id":120134800,"uuid":"45062733","full_name":"alexmx/ios-ui-automation-overview","owner":"alexmx","description":"An overview of popular iOS UI testing solutions.","archived":true,"fork":false,"pushed_at":"2016-02-29T17:42:32.000Z","size":17252,"stargazers_count":24,"open_issues_count":0,"forks_count":3,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-08-09T13:17:05.993Z","etag":null,"topics":["appium","automation-test","calabash","testing","ui-automation","ui-testing","xctest"],"latest_commit_sha":null,"homepage":"","language":"Objective-C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alexmx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-10-27T18:51:34.000Z","updated_at":"2024-05-06T03:39:24.000Z","dependencies_parsed_at":"2024-01-18T04:10:28.986Z","dependency_job_id":"f4753015-2ddb-4d69-aa6d-f3ba3025fa05","html_url":"https://github.com/alexmx/ios-ui-automation-overview","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexmx%2Fios-ui-automation-overview","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexmx%2Fios-ui-automation-overview/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexmx%2Fios-ui-automation-overview/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexmx%2Fios-ui-automation-overview/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alexmx","download_url":"https://codeload.github.com/alexmx/ios-ui-automation-overview/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227089601,"owners_count":17729503,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["appium","automation-test","calabash","testing","ui-automation","ui-testing","xctest"],"created_at":"2024-08-09T13:02:29.984Z","updated_at":"2024-11-29T09:31:04.843Z","avatar_url":"https://github.com/alexmx.png","language":"Objective-C","funding_links":[],"categories":["Objective-C"],"sub_categories":[],"readme":"# iOS UI Automation Overview\n[![Twitter: @amaimescu](https://img.shields.io/badge/contact-%40amaimescu-blue.svg)](https://twitter.com/amaimescu)\n[![License](https://img.shields.io/badge/license-MIT-green.svg?style=flat)](https://github.com/alexmx/ios-ui-automation-overview/blob/master/LICENSE)\n\nAn overview of popular iOS UI Automation solutions which will help you to decide which one to use.\n\nCovered solutions:\n* [x] **UI Tests (XCTest);**\n* [x] **UI Automation;**\n* [x] **Appium;**\n* [x] **Calabash;**\n* [ ] **KIF**\n\nIn order to present proper difference between automation solutions, the same app was covered with the same testing scenarios.\nDemo application represents a simple **Contacts** app which allows us to perform basic CRUD operations over the *Contact* entity.\n\n#### Demo app\n\nContact List | Contact Details | Edit Contact | Remove Contact\n------------ | ------------- | ------------- | -------------\n![Contact List](/assets/contact-list.png) | ![Contact Details](/assets/contact-details.png) | ![Edit Contact](/assets/contact-edit.png) | ![Remove Contact](/assets/contact-remove.png)\n\n## Scenarios\n\n**Add contact:**\n\n```\nScenario: User can add a new contact in the contacts list\n\tGiven I see the contacts list screen\n\tWhen I press on \"Add\" button\n\tThen The screen \"Edit Contact Details\" appears\n\tAnd I enter \"Jon\" in the \"first name\" textfield\n\tAnd I enter \"Snow\" in the \"last name\" textfield\n\tWhen I press on \"Done\" button\n\tThen I see the \"Jon Snow\" contact in the contact list\n```\n\n**Update contact:**\n\n```\nScenario: User can update an existing contact in the contacts list\n\tGiven I see the contacts list screen\n\tGiven I see at least one contact in the list\n\tWhen I press on first contact\n\tThen The screen \"Contact Details\" appears\n\tWhen I press on \"Edit\" button\n\tThen The screen \"Edit Contact Details\" appears\n\tAnd I enter \"Robb\" in the \"first name\" textfield\n\tAnd I enter \"Stark\" in the \"last name\" textfield\n\tWhen I press on \"Done\" button\n\tThen The screen \"Contact Details\" appears\n\tAnd I see the \"Robb Stark\" contact in the contact details\n\tWhen I press on \"Back\" button\n\tThen I see the \"Robb Stark\" contact in the contact list\n```\n\n**Remove contact:**\n\n```\nScenario: User can remove an existing contact from the contacts list\n\tGiven I see the contacts list screen\n\tGiven I see at least one contact in the list\n\tWhen I press on \"Edit\" button\n\tAnd I remove the first contact\n\tThen Contacts list is empty\n```\n\n## Installation\nWe will use `bundler` to install dependencies such as `calabash`, `appium`, `fastlane`.\nMake sure that [`bundler`](http://bundler.io/) is installed on your machine:\n\n```bash\n# Check bundler version\nbundler --version\n\n# Install bundler if not installed\nsudo gem install bundler\n```\n\nAfer `bundler` is installed run:\n```bash\nbundle install\n```\nIf you get an error related to `nokogiri` installation, please consider the steps below:\n```bash\n# Install libxml2 library using homebrew\nbrew install libxml2\n\n# Install 'nokogiri' gem manually\nsudo env ARCHFLAGS=\"-arch x86_64\" gem install nokogiri:1.6.6.4 -- --with-xml=/usr/local/Cellar/libxml2/2.9.2\n\n# Continue installation of dependencies\nbundle install\n```\n\nFor **Appium** we will need to install Appium server separately. For this demo we will use Appium server standalone app.\n![Appium Standalone App](/assets/appium.png)\nCheck out [this guide](https://github.com/appium/appium) for more details related to Appium server installation.\n\n## Run Tests\n\nFor all solutions there are two options to run the tests, manual run or using `fastlane` tools.\n[Fastlane](https://github.com/fastlane/fastlane) is a tool which lets you define and run your deployment pipelines for different environments.\n\n### UI Tests (XCTest)\nUI tests were introduced in XCode 7. It is native solution which allows us to write UI tests using `Objective-C` or `Swift`.\nTests are located in `ContactsUITests` folder.\n\n**Run manually:**\n```bash\nxcodebuild -project Contacts.xcodeproj \\\n\t-scheme \"Contacts\" \\\n\t-sdk iphonesimulator \\\n\t-destination 'platform=iOS Simulator,name=iPhone 6,OS=9.1' \\\n\ttest\n```\n\n**Run with `fastlane`:**\n```bash\nfastlane test_xctests\n```\n**Note:** UI tests can be run directly from Xcode: `Product -\u003e Test`\n\n### Appium\nAppium is an open source test automation framework for use with native, hybrid and mobile web apps. It drives iOS and Android apps using the WebDriver protocol. Appium supports client libraries for multiple programming languages `Java`, `Python`, `Ruby`, `JavaScript`, `PHP` and `C#`. Tests were written in `Ruby` and are located in `appium` folder.\n\n**Run manually:**\n```bash\nxcodebuild -project Contacts.xcodeproj \\\n\t-scheme \"Contacts\" \\\n\t-sdk iphonesimulator \\\n\t-derivedDataPath \"build\" \\\n\tbuild\n\ncd appium \u0026\u0026 bundle exec cucumber\n```\n\n**Run with fastlane:**\n```bash\nfastlane test_appium\n```\n**Note:** Appium server should be running while running tests.\n\n### Calabash\nCalabash is an automated testing technology for Android and iOS native and hybrid applications. It is a free-to-use open source project that is developed and maintained by Xamarin. Calabash has two client libraries for `Ruby` and `Java`. Tests were written in `Ruby` and are located in `calabash` folder.\n\n**Run manually:**\n```bash\nxcodebuild -project Contacts.xcodeproj \\\n\t-scheme \"Contacts-cal\" \\\n\t-sdk iphonesimulator \\\n\t-derivedDataPath \"build\" \\\n\tbuild\n\ncd calabash \u0026\u0026 APP=\"../Build/Products/Debug-iphonesimulator/Contacts-cal.app\" bundle exec cucumber\n```\n\n**Run with fastlane:**\n```bash\nfastlane test_calabash\n```\n**Note:** Before running calabash tests disable firewall otherwise the prompt below will appear on every simulator run:\n![Contact List](/assets/calabash-firewall.png)\n\n### UI Automation\nUI Automation is an old native solution which allows us to write UI tests using `JavaScript`.\nTests are located in `ui-automation` folder.\n\n**Run manually:**\n```bash\nxcodebuild -project Contacts.xcodeproj \\\n\t-scheme \"Contacts\" \\\n\t-sdk iphonesimulator \\\n\t-derivedDataPath \"build\" \\\n\tbuild\n\ncd ui-automation \u0026\u0026 ./run-tests.sh \"../build/Products/Debug-iphonesimulator/Contacts-test.app\" \"iPhone 6 (9.1)\"\n```\n\n**Run with fastlane:**\n```bash\nfastlane test_ui_automation\n```\n**Note:** Consider UI Tests (XCTest) solution instead of UI Automation as Apple has officially deprecated UI Automation in Xcode 7.\n\n## Benchmarks\nThe `benchmarks` folder contains some scripts to measure the tests running time for all covered solutions.\nSome results taken on MacBook Air (Late 2014):\n\nUI Tests (XCTest) | Appium | Calabash | UI Automation\n------------ | ------------- | ------------- | -------------\n54s | 3m 53s | 1m 59s | 45s\n\n\n## References\n\n* [Appium Github page](https://github.com/appium/appium);\n* [Calabash iOS Github page](https://github.com/calabash/calabash-ios);\n* [UI Automation API reference guide](https://developer.apple.com/library/ios/documentation/DeveloperTools/Reference/UIAutomationRef/);\n* [UITests presentation at WWDC 2015](https://developer.apple.com/videos/play/wwdc2015-406/).\n\n## License\nThis project is licensed under the terms of the MIT license. See the LICENSE file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexmx%2Fios-ui-automation-overview","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falexmx%2Fios-ui-automation-overview","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexmx%2Fios-ui-automation-overview/lists"}