https://github.com/testingbot/testingbotctl

A command line interface for the TestingBot platform. Run Maestro, XCUITest and Espresso.
https://github.com/testingbot/testingbotctl
espresso maestro selenium xcuitest
Last synced: 26 days ago
JSON representation
A command line interface for the TestingBot platform. Run Maestro, XCUITest and Espresso.
Host: GitHub
URL: https://github.com/testingbot/testingbotctl
Owner: testingbot
License: mit
Created: 2024-12-20T15:54:32.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2026-04-21T10:34:39.000Z (29 days ago)
Last Synced: 2026-04-21T12:32:17.309Z (29 days ago)
Topics: espresso, maestro, selenium, xcuitest
Language: TypeScript
Homepage: https://testingbot.com
Size: 1.38 MB
Stars: 0
Watchers: 0
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE
- Security: SECURITY.md
Awesome Lists containing this project

README

          


  

    

  

  


  Run mobile tests on real devices in the cloud





  

    

  

  

    

  

  

    

  

  

    

  





  Website

  ·

  Documentation

  ·

  npm



---



  



---

Run **Espresso**, **XCUITest** and **Maestro** tests on real devices in the cloud.

- **Real Devices** — Test on thousands of real iOS and Android devices

- **Emulators & Simulators** — Fast feedback with virtual devices

- **Parallel Execution** — Split tests across multiple devices with sharding

- **CI/CD Ready** — Integrates with GitHub Actions, Jenkins, and more

- **Live Results** — Watch tests run in real-time

- **Artifacts** — Download videos, screenshots, and logs



  Get Started →



---

## Installation

```sh

npm install -g @testingbot/cli

```

**Requirements:** NodeJS 20 or higher

## Authentication

The CLI requires TestingBot API credentials. You can authenticate in several ways:

### Browser Login (Recommended)

```sh

testingbot login

```

This opens your browser for authentication. After logging in, your credentials are saved to `~/.testingbot`.

### Other Methods

- **Command-line options**: `--api-key` and `--api-secret`

- **Environment variables**: `TB_KEY` and `TB_SECRET`

- **Config file**: Create `~/.testingbot` with content `key:secret`

## Commands

### Maestro

Run Maestro UI tests on real devices and emulators/simulators.

```sh

testingbot maestro   [options]

```

**Arguments:**

- `app` - Path to your app file (.apk, .ipa, .app, or .zip)

- `flows` - One or more paths to flow files (.yaml/.yml), directories, .zip files, or glob patterns

**Device Options:**

| Option | Description |

|--------|-------------|

| `--device ` | Device name (e.g., "Pixel 9", "iPhone 16") |

| `--platform ` | Platform: Android or iOS |

| `--deviceVersion ` | OS version (e.g., "14", "17.2") |

| `--real-device` | Use a real device instead of emulator/simulator |

| `--orientation ` | Screen orientation: PORTRAIT or LANDSCAPE |

| `--device-locale ` | Device locale (e.g., "en_US", "de_DE") |

| `--timezone ` | Timezone (e.g., "America/New_York", "Europe/London") |

**Test Configuration:**

| Option | Description |

|--------|-------------|

| `--name ` | Test name for dashboard identification |

| `--build ` | Build identifier for grouping test runs |

| `--include-tags ` | Only run flows with these tags (comma-separated) |

| `--exclude-tags ` | Exclude flows with these tags (comma-separated) |

| `-e, --env ` | Environment variable for flows (can be repeated) |

| `--config ` | Path to a custom Maestro config file (default: config.yaml in project root) |

| `--maestro-version ` | Maestro version to use (e.g., "2.0.10") |

**Network & Location:**

| Option | Description |

|--------|-------------|

| `--throttle-network ` | Network throttling: 4G, 3G, Edge, airplane, or disable |

| `--geo-country-code ` | Geographic IP location (ISO country code, e.g., "US", "DE") |


**Tunnel:**

| Option | Description |

|--------|-------------|

| `-t, --tunnel` | Start a TestingBot tunnel for this test run (cannot be combined with `--async`) |

| `--tunnel-identifier ` | Identifier for the tunnel, allowing multiple tunnels in parallel |

**Output Options:**

| Option | Description |

|--------|-------------|

| `--async` | Start tests and exit without waiting for results |

| `-q, --quiet` | Suppress progress output |

| `--report ` | Download report after completion: html or junit |

| `--report-output-dir ` | Directory to save reports (required with --report) |

| `--download-artifacts [mode]` | Download test artifacts (logs, screenshots, video). Mode: `all` (default) or `failed` |

| `--artifacts-output-dir ` | Directory to save artifacts zip (defaults to current directory) |

**Advanced Options:**

| Option | Description |

|--------|-------------|

| `--shard-split ` | Split flows into N parallel sessions for faster execution |

| `--ignore-checksum-check` | Skip checksum verification and always upload the app |

**CI/CD Integration:**

| Option | Description |

|--------|-------------|

| `--commit-sha ` | Git commit SHA associated with this test run |

| `--pull-request-id ` | Pull request ID this test run originated from |

| `--repo-name ` | Repository name (e.g., GitHub repo slug) |

| `--repo-owner ` | Repository owner (e.g., GitHub organization or username) |

**Examples:**

```sh

# Basic usage

testingbot maestro app.apk ./flows

# Multiple flow directories

testingbot maestro app.apk ./flows/smoke ./flows/regression ./flows/e2e

# With device selection

testingbot maestro app.apk ./flows --device "Pixel 8" --deviceVersion "14"

# Android app on real device with tags

testingbot maestro app.apk ./flows --device "Samsung Galaxy S24" --real-device --include-tags "smoke,regression"

# With environment variables

testingbot maestro app.apk ./flows -e API_URL=https://staging.example.com -e API_KEY=secret

# Download JUnit report

testingbot maestro app.apk ./flows --report junit --report-output-dir ./reports

# Download all artifacts (logs, screenshots, video)

testingbot maestro app.apk ./flows --download-artifacts --build "build-123"

# Download artifacts only for failed tests

testingbot maestro app.apk ./flows --download-artifacts failed --artifacts-output-dir ./artifacts

# Use a custom config file

testingbot maestro app.apk ./flows --config .maestro/ci-config.yaml

# Run in background (async)

testingbot maestro app.apk ./flows --async

# Split flows across 3 shards, grouping all flows over 3 parallel sessions

testingbot maestro app.apk ./flows --shard-split 3

# CI/CD integration with Git metadata

testingbot maestro app.apk ./flows \

  --commit-sha "abc123def" \

  --pull-request-id "42" \

  --repo-owner "myorg" \

  --repo-name "myapp"

```

---

### Espresso

Run Android Espresso tests on real devices and emulators.

```sh

testingbot espresso [appFile] [testAppFile] [options]

```

**Arguments:**

- `appFile` - Path to application APK file

- `testAppFile` - Path to test APK file containing Espresso tests

**Device Options:**

| Option | Description |

|--------|-------------|

| `--app ` | Path to application APK file |

| `--test-app ` | Path to test APK file |

| `--device ` | Device name (e.g., "Pixel 6", "Samsung.*") |

| `--platform-version ` | Android OS version (e.g., "12", "13", "14") |

| `--real-device` | Use a real device instead of an emulator |

| `--tablet-only` | Only allocate tablet devices |

| `--phone-only` | Only allocate phone devices |

| `--locale ` | Device locale (e.g., "en_US", "de_DE") |

| `--timezone ` | Timezone (e.g., "America/New_York", "Europe/London") |

**Test Configuration:**

| Option | Description |

|--------|-------------|

| `--name ` | Test name for dashboard identification |

| `--build ` | Build identifier for grouping test runs |

| `--test-runner ` | Custom test instrumentation runner |

| `--language ` | App language (ISO 639-1 code, e.g., "en", "fr", "de") |

**Test Filtering:**

| Option | Description |

|--------|-------------|

| `--class ` | Run tests in specific classes (comma-separated fully qualified names) |

| `--not-class ` | Exclude tests in specific classes |

| `--package ` | Run tests in specific packages (comma-separated) |

| `--not-package ` | Exclude tests in specific packages |

| `--annotation ` | Run tests with specific annotations (comma-separated) |

| `--not-annotation ` | Exclude tests with specific annotations |

| `--size ` | Run tests by size: small, medium, large (comma-separated) |

**Network & Location:**

| Option | Description |

|--------|-------------|

| `--throttle-network ` | Network throttling: 4G, 3G, Edge, or airplane |

| `--geo-location ` | Geographic IP location (ISO country code, e.g., "US", "DE") |


**Tunnel:**

| Option | Description |

|--------|-------------|

| `-t, --tunnel` | Start a TestingBot tunnel for this test run (cannot be combined with `--async`) |

| `--tunnel-identifier ` | Identifier for the tunnel, allowing multiple tunnels in parallel |

**Output Options:**

| Option | Description |

|--------|-------------|

| `--async` | Start tests and exit without waiting for results |

| `-q, --quiet` | Suppress progress output |

| `--report ` | Download report after completion: html or junit |

| `--report-output-dir ` | Directory to save reports (required with --report) |

**Examples:**

```sh

# Basic usage with positional arguments

testingbot espresso app.apk app-test.apk --device "Pixel 8"

# Using named options

testingbot espresso --app app.apk --test-app app-test.apk --device "Pixel 8"

# Real device with specific Android version

testingbot espresso app.apk app-test.apk \

  --device "Samsung Galaxy S24" \

  --platform-version "14" \

  --real-device

# Run specific test classes

testingbot espresso app.apk app-test.apk \

  --device "Pixel 8" \

  --class "com.example.LoginTest,com.example.HomeTest"

# Run tests with annotations

testingbot espresso app.apk app-test.apk \

  --device "Pixel 8" \

  --annotation "com.example.SmokeTest" \

  --size "small,medium"

# With network throttling and geolocation

testingbot espresso app.apk app-test.apk \

  --device "Pixel 8" \

  --throttle-network "3G" \

  --geo-location "DE" \

  --language "de"

# Download JUnit report

testingbot espresso app.apk app-test.apk \

  --device "Pixel 8" \

  --report junit \

  --report-output-dir ./reports

```

---

### XCUITest

Run iOS XCUITest tests on real devices and simulators.

```sh

testingbot xcuitest [appFile] [testAppFile] [options]

```

**Arguments:**

- `appFile` - Path to application IPA file

- `testAppFile` - Path to test ZIP file containing XCUITests

**Device Options:**

| Option | Description |

|--------|-------------|

| `--app ` | Path to application IPA file |

| `--test-app ` | Path to test ZIP file |

| `--device ` | Device name (e.g., "iPhone 15", "iPad.*") |

| `--platform-version ` | iOS version (e.g., "17.0", "18.2") |

| `--real-device` | Use a real device instead of a simulator |

| `--tablet-only` | Only allocate tablet devices |

| `--phone-only` | Only allocate phone devices |

| `--orientation ` | Screen orientation: PORTRAIT or LANDSCAPE |

| `--locale ` | Device locale (e.g., "DE", "US") |

| `--timezone ` | Timezone (e.g., "America/New_York", "Europe/London") |

**Test Configuration:**

| Option | Description |

|--------|-------------|

| `--name ` | Test name for dashboard identification |

| `--build ` | Build identifier for grouping test runs |

| `--language ` | App language (ISO 639-1 code, e.g., "en", "fr", "de") |

**Network & Location:**

| Option | Description |

|--------|-------------|

| `--throttle-network ` | Network throttling: 4G, 3G, Edge, or airplane |

| `--geo-location ` | Geographic IP location (ISO country code, e.g., "US", "DE") |


**Tunnel:**

| Option | Description |

|--------|-------------|

| `-t, --tunnel` | Start a TestingBot tunnel for this test run (cannot be combined with `--async`) |

| `--tunnel-identifier ` | Identifier for the tunnel, allowing multiple tunnels in parallel |

**Output Options:**

| Option | Description |

|--------|-------------|

| `--async` | Start tests and exit without waiting for results |

| `-q, --quiet` | Suppress progress output |

| `--report ` | Download report after completion: html or junit |

| `--report-output-dir ` | Directory to save reports (required with --report) |

**Examples:**

```sh

# Basic usage with positional arguments

testingbot xcuitest app.ipa app-test.zip --device "iPhone 16"

# Using named options

testingbot xcuitest --app app.ipa --test-app app-test.zip --device "iPhone 16"

# Real device with specific iOS version

testingbot xcuitest app.ipa app-test.zip \

  --device "iPhone 15 Pro" \

  --platform-version "17.2" \

  --real-device

# iPad in landscape mode

testingbot xcuitest app.ipa app-test.zip \

  --device "iPad Pro" \

  --tablet-only \

  --orientation LANDSCAPE

# With localization settings

testingbot xcuitest app.ipa app-test.zip \

  --device "iPhone 16" \

  --locale "DE" \

  --language "de" \

  --timezone "Europe/Berlin"

# With network throttling and geolocation

testingbot xcuitest app.ipa app-test.zip \

  --device "iPhone 16" \

  --throttle-network "3G" \

  --geo-location "DE"

# Download HTML report

testingbot xcuitest app.ipa app-test.zip \

  --device "iPhone 16" \

  --report html \

  --report-output-dir ./reports

# Run in background

testingbot xcuitest app.ipa app-test.zip \

  --device "iPhone 16" \

  --async

```

---

## Common Features

### Real-time Progress

By default, the CLI shows real-time progress updates including:

- Test status updates with actual device names (even when using wildcards)

- Device allocation status

- Live output from Maestro flows

Use `--quiet` to suppress progress output.

### Graceful Shutdown

Press `Ctrl+C` to gracefully stop running tests. The CLI will:

1. Stop all active test runs on TestingBot

2. Clean up resources

3. Exit with appropriate status code

Press `Ctrl+C` twice to force exit immediately.

### Report Downloads

All test frameworks support downloading reports after completion:

```sh

# JUnit XML format (for CI integration)

--report junit --report-output-dir ./reports

# HTML format (for human viewing)

--report html --report-output-dir ./reports

```

### Artifact Downloads (Maestro only)

Download all test artifacts including logs, screenshots, and video recordings:

```sh

testingbot maestro app.apk ./flows --download-artifacts --build "my-build"

```

Artifacts are saved as a zip file named after the `--build` value (or with a timestamp if not provided).

## Exit Codes

- `0` - All tests passed

- `1` - One or more tests failed or an error occurred

## Documentation

For more information, visit [TestingBot Documentation](https://testingbot.com/support).

## License

MIT
ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/testingbot/testingbotctl

Awesome Lists containing this project

README
