{"id":15435207,"url":"https://github.com/drptbl/crows-nest","last_synced_at":"2025-08-30T16:37:30.284Z","repository":{"id":88033397,"uuid":"69367687","full_name":"drptbl/crows-nest","owner":"drptbl","description":"supervisor process to keep Saucelabs Tunnels running in high availability mode","archived":false,"fork":false,"pushed_at":"2016-09-27T14:44:11.000Z","size":11,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-02-27T05:13:28.406Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/drptbl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-09-27T14:52:25.000Z","updated_at":"2019-07-08T18:50:13.000Z","dependencies_parsed_at":"2023-03-13T18:32:27.741Z","dependency_job_id":null,"html_url":"https://github.com/drptbl/crows-nest","commit_stats":{"total_commits":5,"total_committers":2,"mean_commits":2.5,"dds":"0.19999999999999996","last_synced_commit":"b62e35ab855395da71610799558a9a311db73bd5"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/drptbl%2Fcrows-nest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/drptbl%2Fcrows-nest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/drptbl%2Fcrows-nest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/drptbl%2Fcrows-nest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/drptbl","download_url":"https://codeload.github.com/drptbl/crows-nest/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241819717,"owners_count":20025388,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T18:43:05.617Z","updated_at":"2025-03-04T09:23:33.757Z","avatar_url":"https://github.com/drptbl.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg width=\"40%\" height=\"40%\" align=\"right\" src=\"https://cloud.githubusercontent.com/assets/1438478/18613191/055e771a-7d32-11e6-8f1c-9fe3c328696c.jpeg\"\u003e\n\n# crows-nest\n\nA supervisor tool to launch and monitor multiple SauceLabs Sauce Connect tunnel in [high avaiability mode](https://wiki.saucelabs.com/display/DOCS/High+Availability+Sauce+Connect+Setup).\n\nThe tool tracks the availability of each tunnel. If any of the tunnels is unresponsive, the supervisor will attempt to terminate the tunnel gracefully and start a new one. Additionally, the tool provides the ability to perform \"rolling restarts\" all the tunnels periodically to avoid known issues with long-running tunnel processes.\n\nBy launching sauce tunnels in high availability mode with crows-nest, you won't need to: \n\n 1. manually check the availability of each tunnel\n 2. manually launch up new tunnels to replace the dead ones\n 3. manually and periodically restart all tunnels\n\n## Usage\n\n### Configuration\n\n```\n./config.json\n{\n  \"username\": \"\",\n  \"accessKey\": \"\",\n  \"verbose\": false,\n  \"proxy\": null,\n  \"tunnelIdentifier\": \"\",\n  \"waitTunnelShutdown\": true,\n  \"noRemoveCollidingTunnels\": true,\n\n  \"restartCron\": \"*/2 * * * *\"\n}\n```\n\n`./config.json` file has the basic configurations that are required by Sauce Connect. To launch your own tunnels, `username`, `accessKey` and `tunnelIdentifier` are mandatory. \n\nMore configurations please refer to this page [sauce-connect-launcher](https://github.com/bermi/sauce-connect-launcher#advanced-usage).\n\nIn high availability mode all tunnels share the same `tunnelIdentifier`. `tunnelIdentifier` can be any string.  One suggested convention is to use this ID to describe the geographical location where your tunnel terminates.  For example, `east` or `west`.  \n\nYou can set `username` and `accessKey` using one of the following methods:\n \n 1. Specifying the values in `./config.json`\n 2. Setting the environment variable `SAUCE_USERNAME` and `SAUCE_ACCESS_KEY`  \n\nIf the rolling restart feature is enabeld, `restartCron` must be a valid cron value.\n\n### Help\n\n```\n./bin/supervise --help\n```\n\n### Basic usage\n\nStart one sauce tunnel in high availability mode:\n```\n./bin/supervise --tunnels 1\n```\n\nWith rolling restart feature:\n```\nSAUCE_USERNAME=xxx SAUCE_ACCESS_KEY=yyy ./bin/supervise --tunnels 1 --rollingRestart\n```\n\n### Advanced usage\n\nRead sauce tunnel configuration from `./myproject/config.json` and launch 20 sauce tunnels in high availability mode\n```\n./bin/supervise --tunnels 20 --config ./myproject/config.json\n```\n\n### Daemon\n\nWe use [pm2](https://www.npmjs.com/package/pm2) to run the supervisor process as a daemon\n\nTo start:\n```\n./node_modules/.bin/pm2 start ./bin/supervise --kill-timeout 600000 --silent -- --tunnels 10 --rollingRestart\n```\n\nTo stop:\n```\n./node_modules/.bin/pm2 stop supervise\n```\n\nTo view the log:\n```\n./node_modules/.bin/pm2 log --lines 100\n```\n\n## Design\n\n### Architecture\n\n```\n+---------------------------------------------------------------------+         +-----------------+\n|                                                                     |         |                 |\n| Crows-nest                           +---------------------------+  |         | Saucelabs       |\n|                                      |          +----------------|  |         |                 |\n|                                      | Tunnel 1 | Sauce Tunnel  ||  |         | Sauce Tunnel    |\n|                              +-----\u003e |          | Child Process || +--------\u003e | Proxy Cloud     |\n|                              |       |          +----------------|  |         |                 |\n|                              |       +---------------------------+  |         |                 |\n|                              |                                      |         |                 |\n|                              |                                      |         |                 |\n|                              |                                      |         |                 |\n|                              |       +---------------------------+  |         |                 |\n|                              |       |          +----------------|  |         |                 |\n| +-----------------+          |       | Tunnel 2 | Sauce Tunnel  ||  |         |                 |\n| |                 |          +-----\u003e |          | Child Process || +--------\u003e |                 |\n| | Supervisor      +----------+       |          +----------------|  |         |                 |\n| |                 |          |       +---------------------------+  |         |                 |\n| +-----------------+          |                                      |         |                 |\n|                              |                      .               |         |                 |\n|                              |                      .               |         |                 |\n|                              |                      .               |         |                 |\n|                              |                      .               |         |                 |\n|                              |                                      |         |                 |\n|                              |       +---------------------------+  |         |                 |\n|                              |       |          +----------------|  |         |                 |\n|                              |       | Tunnel n | Sauce Tunnel  ||  |         |                 |\n|                              +-----\u003e |          | Child Process || +--------\u003e |                 |\n|                                      |          +----------------|  |         |                 |\n|                                      +---------------------------|  |         |                 |\n+---------------------------------------------------------------------+         +-----------------+\n```\n\n### Components\n\nThere are two major components in Crows-nest, **[Supervisor](https://github.com/TestArmada/crows-nest/blob/master/lib/supervisor.js)** and **[Tunnel](https://github.com/TestArmada/crows-nest/blob/master/lib/tunnel.js)**\n```\n\n +------------+                            +---------+\n | Supervisor |                            | Tunnels |\n +-----+------+                            +----+----+\n       |                                        |\n      +++                                       |\n      | |                                      +++\n      | | Initialize()                         | |\n      | +------------------------------------\u003e | | monitor()\n      | |                                      | +------------+\n      +++                                      | |            |\n       |                                       | | \u003c----------+\n       |                                       +++\n       |                                        |\n       |                                        |\n      +++                                       |\n      | | scheduleRestart()                    +++\n      | +------------------------------------\u003e | | stop()\n      | |                                      | +------------+\n      | |                                      | |            |\n      | |                                      | |            |\n      | |                                      | | \u003c----------+\n      | |                                      | |\n      | |                                      | | start()\n      | |                                      | +------------+\n      | |                                      | |            |\n      | |                                      | |            |\n      | |                                      | | \u003c----------+\n      +++                                      +++\n       |                                        |\n       |                                        |\n       +                                        +\n\n```\n\n#### Tunnel\n\nCrows-nest Tunnel maintains the life cycle of a Saucelabs Sauce Connect Tunnel. It is a one to one mapping to Saucelabs Sauce Connect Tunnel instance. It does following things\n\n1. Start a Sauce Connect Tunnel as child process\n2. Monitor the status of current child process\n  1. Terminate current Sauce Connect Tunnel nicely if the connection drops and start a new one.\n  2. Restart current Sauce Connect Tunnel if scheduled by `rollingRestart`\n3. Stop current Sauce Connect Tunnel\n\n#### Supervisor\n\nCrows-nest Supervisor keeps track of all Crows-nest Tunnels. It does following things\n\n1. Initialize all Tunnels\n2. Start all Tunnels by sending `start` signals to each Tunnel\n3. Restart all tunnels by sending `restart` signals to each Tunnel according to schedule\n4. Stop all Tunnels by sending `stop` signals to each Tunnel\n\n### Randomness\n\nTo avoid network blast (in case all tunnels are scheduled at the same time), some randomness are introduced\n\n1. Each tunnel takes random delay [0, 5000] ms to start\n2. Each tunnel takes random delay [0, 5000] ms to stop\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdrptbl%2Fcrows-nest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdrptbl%2Fcrows-nest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdrptbl%2Fcrows-nest/lists"}