{"id":19466816,"url":"https://github.com/dena/devfarm","last_synced_at":"2025-04-25T11:30:45.104Z","repository":{"id":49454670,"uuid":"222924450","full_name":"DeNA/devfarm","owner":"DeNA","description":"Tools to control iOS and Android mobile apps across several device farms","archived":false,"fork":false,"pushed_at":"2021-06-17T07:18:00.000Z","size":67343,"stargazers_count":20,"open_issues_count":0,"forks_count":7,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-06-19T01:58:54.603Z","etag":null,"topics":["android","aws","device-farms","ios","testing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DeNA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-11-20T11:41:29.000Z","updated_at":"2024-03-06T19:10:22.000Z","dependencies_parsed_at":"2022-09-11T23:02:27.237Z","dependency_job_id":null,"html_url":"https://github.com/DeNA/devfarm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeNA%2Fdevfarm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeNA%2Fdevfarm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeNA%2Fdevfarm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeNA%2Fdevfarm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DeNA","download_url":"https://codeload.github.com/DeNA/devfarm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223996553,"owners_count":17238327,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android","aws","device-farms","ios","testing"],"created_at":"2024-11-10T18:30:28.330Z","updated_at":"2024-11-10T18:32:25.815Z","avatar_url":"https://github.com/DeNA.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"devfarm\n=======\n\n[![CircleCI](https://circleci.com/gh/DeNA/devfarm/tree/master.svg?style=svg)](https://circleci.com/gh/DeNA/devfarm/tree/master)\n\nTools to control iOS and Android mobile apps across several device farms such as AWS Device Farm.  The purpose of this project is portability among major device farms (including emulator/simulator farms) and be able to do Domain-Specific Tests across device farms.\n\nThe portability is important because there are a lot of device farms, so you must try these device farms indivisually to verify until satisfy your usecases.  This challenge is hard work but you can easily switch to other device cloud if you using this tool.\n\nThis tool supports only launching/watching/halting mobile apps on device farms.  These features can help domain-specific tests that sending/receiving directly domain events (e.g. jump a character, attack to someone, reading other character status, ...). Such tests can be noise-free and more efficient than UI-layer fuzzing or UI-layer schenario tests using like Appium or Calabash.\n\n\n\nUsecases\n--------\n\nIn our usecase, we bundled a domain specific random-walk agent into our apps.  This random-walk agent crash if something go wrong.  This tool could watch app crashes on device farms via CLI, so we could know it via CI services easily.  Additionaly, we could improve the coverage per unit time of the random-walk agent by increasing parallel devices numbers.\n\nOr, if you want to do traditional tests using like JUnit or XCTest for domain specific-tests, you still can bundle the test harness into your apps.\n\n\n\nUsage\n-----\n\n```console\n$ devfarm run-ios --os-version 12.0 --device 'apple iphone xs' --instance-group example --platform aws-device-farm --ipa path/to/app.ipa --args-json '[\"-ARG1\", \"HELLO_DEVFARM\"]' --lifetime-sec 900 \u0026\nplatform                status\naws-device-farm         launching\n\n$ devfarm status --instance-group example\nplatform                device                  os      state   note\naws-device-farm         apple iphone xs         ios     ACTIVE\n\n$ devfarm halt --instance-group example\nplatform                status\naws-device-farm         halting\n\n$ devfarm status --instance-group example\nplatform                device                  os      state           note\naws-device-farm         apple iphone xs         ios     INACTIVATING\n```\n\n\n### List Available Device and OS\n\n```console\n$ devfarm list-devices\nplatform                os              device                                  available       note\naws-device-farm         ios 9           apple iphone xs                         yes\n...\naws-device-farm         android 9       google google pixel 3                    yes\n...\n```\n\n\n### Launch by plan.yml\n\n```console\n$ devfarm run-all --help\nUsage: [options] \u003cplan.yml\u003e\n  -dry-run\n        enables dry-run (WARNING: not stable yet)\n  -verbose\n        enables verbose logs\n```\n\nSyntax of plan.yml:\n\n```yaml\ninstance_groups:\n  \u003cgroup\u003e:\n    # for iOS\n    - platform: \u003cplatform\u003e  # required\n      ios: \u003cversion\u003e        # required\n      device: \u003cdevice\u003e      # required\n      ipa: \u003cfilepath\u003e       # required\n      args: []              # optional\n      lifetime_sec:         # required\n\n    # for Android\n    - platform: \u003cplatform\u003e  # required\n      android: \u003cversion\u003e    # required\n      device: \u003cdecice\u003e      # required\n      apk: \u003cfilepath\u003e       # required\n      app_id: \u003capp_id\u003e      # required\n      intent_extras: []     # optional\n      lifetime_sec:         # required\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\n```console\n$ cat path/to/plan.yml\ninstance_groups:\n  example:\n    - platform: aws-device-farm\n      ios: 12.0\n      device: apple iphone xs\n      ipa: path/to/app.ipa\n      args:\n        - -ARG1\n        - VALUE1\n      lifetime_sec: 900\n    - platform: aws-device-farm\n      android: 9\n      device: google google pixel 3\n      apk: path/to/app.apk\n      app_id: com.example.app\n      intent_extras:\n        - -e\n        - ARG1\n        - VALUE1\n      lifetime_sec: 900\n\n$ devfarm run-all --dry-run path/to/plan.yml\nplatform         status     note\naws-device-farm  launching\naws-device-farm  launching\n```\n\u003c/details\u003e\n\n\n### Instance Group Dependent Commands\n\nInstance Groups is an unit to halt or check status.\nYou can specify names what you want but it must be a non-empty and must not include `[^0-9a-zA-Z_]`.\n\n\n\n#### Launch iOS Apps\n\n```console\n$ devfarm run-ios --help\nUsage:\n  -args-json string\n        arguments that will be passed to the iOS app (via ProcessInfo#arguments) after decoding to plain arguments (default \"[]\")\n  -device string\n        device name listed by 'devfarm list-devices' (required)\n  -dry-run\n        enables dry-run (WARNING: not stable yet)\n  -instance-group string\n        instance group name (required)\n  -ipa string\n        ipa file to launch (required)\n  -os-version string\n        iOS version listed by 'devfarm list-devices' (required)\n  -platform string\n        platform name listed by 'devfarm list-devices' (required)\n  -verbose\n        enables verbose logs\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\n```console\n$ devfarm run-ios \\\n    --instance-group example \\\n    --platform aws-device-farm \\\n    --device 'apple iphone xs' \\\n    --os-version 12.0 \\\n    --ipa 'path/to/app.ipa' \\\n    --args-json '[\"-ARG\", \"VALUE\"]'\nplatform                status\naws-device-farm         launching\n```\n\u003c/details\u003e\n\n\n\n#### Launch Android Apps\n\n```console\n$ devfarm run-android --help\nUsage:\n  -apk string\n        apk file to launch (required)\n  -app-id string\n        application ID (it often called as 'package name') to the app (required)\n  -device string\n        device name listed by 'devfarm list-devices' (required)\n  -dry-run\n        enables dry-run (WARNING: not stable yet)\n  -instance-group string\n        instance group name (required)\n  -intent-extras-json string\n        arguments that will be passed to the Android app (via Intent#getExtras) after decoding to plain arguments (default \"[]\")\n  -os-version string\n        Android version listed by 'devfarm list-devices' (required)\n  -platform string\n        platform name listed by 'devfarm list-devices' (required)\n  -verbose\n        enables verbose logs\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\n```console\n$ devfarm forever-android \\\n    --instance-group example \\\n    --platform aws-device-farm \\\n    --device 'google google pixel 3' \\\n    --os-version 9 \\\n    --apk 'path/to/app.apk' \\\n    --app-id 'com.example.app' \\\n    --intent-extras-json '[\"-e\", \"ARG\", \"VALUE\"]'\nplatform                status\naws-device-farm         launching\n```\n\u003c/details\u003e\n\n\n#### Status\n\n```console\n$ devfarm status --help\nUsage:\n  -dry-run\n        enables dry-run (WARNING: not stable yet)\n  -instance-group string\n        instance group name to filter (optional)\n  -verbose\n        enables verbose logs\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\n```console\n$ devfarm status\nplatform                device                  os      state   note\naws-device-farm         apple iphone xs         ios     ACTIVE\n```\n\nor\n\n```\n$ devfarm status --instance-group example\nplatform                device                  os      state   note\naws-device-farm         apple iphone xs         ios     ACTIVE\n```\n\u003c/details\u003e\n\n\n#### Stop Apps\n\n```console\n$ devfarm halt --help\nUsage:\n  -dry-run\n        enables dry-run (WARNING: not stable yet)\n  -instance-group string\n        instance group name (required)\n  -verbose\n        enables verbose logs\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\n```console\n$ devfarm halt --instance-group example\nplatform                status\naws-device-farm         halting\n```\n\u003c/details\u003e\n\n\n### Utility Commands\n\u003cdetails\u003e\n\u003csummary\u003eCommands\u003c/summary\u003e\n\n#### Version\n\n```console\n$ devfarm version\n0.0.0\n```\n\n\n#### Check Authentication Status\n\n```console\n$ devfarm auth-status --help\nUsage:\n  -dry-run\n        enables dry-run (WARNING: not stable yet)\n  -verbose\n        enables verbose logs\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\n```console\n$ devfarm auth-status\nplatform                auth\naws-device-farm         success\n```\n\u003c/details\u003e\n\n\n#### Validate plan.yml\n\n```console\n$ devfarm validate --help\nUsage: \u003cplan.yml\u003e\n  -verbose\n        enables verbose logs\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\n```console\n$ cat path/to/plan.yml\ninstance_groups:\n  example:\n    - platform: any-platform\n      ios: 12.0\n      device: apple iphone xs\n      ipa: path/to/app.ipa\n      args:\n        - -ARG1\n        - VALUE1\n    - platform: any-platform\n      android: 9\n      device: google google pixel 3\n      apk: path/to/app.apk\n      app_id: com.example.app\n      intent_extras:\n        - -e\n        - ARG1\n        - VALUE1\n\n$ devfarm validate path/to/plan.yml\n$ echo $?\n0\n\n$ devfarm validate --verbose path/to/plan.yml\n{\n  \"instance_groups\": {\n    \"example\": [\n      {\n        \"platform\": \"any-platform\",\n        \"os\": \"ios\",\n        \"ios\": {\n          \"group_name\": \"example\",\n          \"device\": {\n            \"name\": \"apple iphone xs\",\n            \"ios_version\": \"12.0\"\n          },\n          \"ipa\": \"path/to/app.ipa\",\n          \"args\": [\n            \"-ARG1\",\n            \"VALUE1\"\n          ]\n        },\n        \"android\": {\n          \"group_name\": \"\",\n          \"device\": {\n            \"name\": \"\",\n            \"android_version\": \"\"\n          },\n          \"apk\": \"\",\n          \"app_id\": \"\",\n          \"intent_extras\": null\n        }\n      },\n      {\n        \"platform\": \"any-platform\",\n        \"os\": \"android\",\n        \"ios\": {\n          \"group_name\": \"\",\n          \"device\": {\n            \"name\": \"\",\n            \"ios_version\": \"\"\n          },\n          \"ipa\": \"\",\n          \"args\": null\n        },\n        \"android\": {\n          \"group_name\": \"example\",\n          \"device\": {\n            \"name\": \"google google pixel 3\",\n            \"android_version\": \"9\"\n          },\n          \"apk\": \"path/to/app.apk\",\n          \"app_id\": \"com.example.app\",\n          \"intent_extras\": [\n            \"-e\",\n            \"ARG1\",\n            \"VALUE1\"\n          ]\n        }\n      }\n    ]\n  }\n}\n\n$ devfarm validate path/to/broken.yml\ninvalid iOS plan (at 1-th plan of instance group \"example\"):\n    device: must not be empty\n    ipa: must not be empty\ninvalid plan (at 2-th plan of instance group \"example\"):\n    unsupported os: \"unavailable\"\ninvalid Android plan (at 1-th plan of instance group \"other\"):\n    platform: must not be empty\n    device: must not be empty\n    apk: must not be empty\n    app_id: must not be empty\n\n$ echo $?\n1\n```\n\u003c/details\u003e\n\n\n#### See Bundled Assets\n```\n$ devfarm ls-assets\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\n```console\n$ devfarm ls-assets\nassets/aws-device-farm/workflows/0-shared.bash\nassets/aws-device-farm/workflows/1-install.bash\nassets/aws-device-farm/workflows/2-pretest.bash\nassets/aws-device-farm/workflows/3-test.bash\nassets/aws-device-farm/workflows/4-posttest.bash\nassets/devfarmagent/darwin-amd64/devfarmagent\nassets/devfarmagent/devfarmagent.bash\nassets/devfarmagent/linux-amd64/devfarmagent\nassets/ios-deploy-agent/package-lock.json\nassets/ios-deploy-agent/package.json\n```\n\u003c/details\u003e\n\n\n#### See Bundled Asset Content\n```console\n$ devfarm cat-asset \u003casset\u003e\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\n```console\n$ devfarm cat-asset assets/devfarmagent/darwin-amd64/devfarmagent | file -\n/dev/stdin: Mach-O 64-bit executable x86_64\n```\n\u003c/details\u003e\n\u003c/details\u003e\n\n\n\nInstall\n-------\n\n```console\n$ go get -u github.com/dena/devfarm/cmd/devfarm\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eUsing Docker\u003c/summary\u003e\n\n```console\n$ docker pull docker.pkg.github.com/dena/devfarm/devfarm:latest\n\n$ cat .env\nAWS_ACCESS_KEY_ID=***\nAWS_SECRET_ACCESS_KEY=***\n\n$ docker run --rm --env-file ./.env docker.pkg.github.com/dena/devfarm/devfarm:latest auth-status\n\n$ tree -a \"$(pwd)\"\n.\n├── .env\n└── app\n    ├── Example.apk\n    ├── Example.ipa\n    └── planfile.yml\n\n1 directory, 4 files\n\n$ docker run --rm --env-file ./.env -v \"$(pwd)/app:/app\" docker.pkg.github.com/dena/devfarm/devfarm:latest run-all /app/planfile\n```\n\u003c/details\u003e\n\n\n\nRequirements\n------------\n\n- To use AWS Device Farm\n    - [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html)\n\n\nTodo\n----\n\n- ~~Stabilize iOS apps made by Unity on AWS Device Farm (at now, 2/3 runs were failed)~~\n    - We have started to do external monitoring, and we confirmed that it almost achived, but still about 1/30 runs were unstable\n- Detect that the app become background\n    - Detection for iOS have never been tested\n    - Detection for Android is not supported yet, because the version of adb on AWS Device Farm is 1.0.32 (too old)\n- Work `--dry-run` on all commands for testing\n- Support Android Simulator on AWS EC2\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdena%2Fdevfarm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdena%2Fdevfarm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdena%2Fdevfarm/lists"}