{"id":13581691,"url":"https://github.com/Unbabel/replicant","last_synced_at":"2025-04-06T10:32:50.914Z","repository":{"id":44627498,"uuid":"235334068","full_name":"Unbabel/replicant","owner":"Unbabel","description":"Synthetic application testing made easy, written in Go.","archived":true,"fork":false,"pushed_at":"2020-09-15T13:46:34.000Z","size":3379,"stargazers_count":17,"open_issues_count":6,"forks_count":5,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-11-05T21:44:59.241Z","etag":null,"topics":["golang","monitoring","synthetics","testing"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Unbabel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-21T12:09:57.000Z","updated_at":"2024-06-26T10:17:32.000Z","dependencies_parsed_at":"2022-08-29T04:42:19.729Z","dependency_job_id":null,"html_url":"https://github.com/Unbabel/replicant","commit_stats":null,"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Unbabel%2Freplicant","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Unbabel%2Freplicant/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Unbabel%2Freplicant/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Unbabel%2Freplicant/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Unbabel","download_url":"https://codeload.github.com/Unbabel/replicant/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247470461,"owners_count":20944146,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","monitoring","synthetics","testing"],"created_at":"2024-08-01T15:02:10.922Z","updated_at":"2025-04-06T10:32:45.902Z","avatar_url":"https://github.com/Unbabel.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# ![Replicant](https://raw.githubusercontent.com/Unbabel/replicant/master/doc/logo.png)\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/Unbabel/replicant?style=flat-square)](https://goreportcard.com/report/github.com/Unbabel/replicant)\n[![GoDoc](https://img.shields.io/badge/api-reference-blue.svg?style=flat-square)](https://godoc.org/github.com/Unbabel/replicant)\n[![Docker Cloud Automated build](https://img.shields.io/docker/cloud/automated/unbabel/replicant?style=flat-square)](https://hub.docker.com/r/unbabel/replicant)\n\nReplicant is a synthetic testing service named after the bioengineered androids from Blade Runner. (all synthetics came from Blade Runner :)\n\nIt allows web application testing using chromedp, and api application testing using Go or Javascript. Provides a test manager, execution scheduler, api and facilities for emitting result data to external systems.\n\n## Status\n\n***Under heavy development and API changes are expected. Please file an issue if anything breaks.***\n\n## Requirements\n\n* Go \u003e 1.15\n\n## Runing replicant\n\nThe replicant binary packs all functionality needed to run the server, executor and run local execution of tests for development or CI/CD purposes.\n\n### Locally for test development purposes\n\n```bash\n/path/to/replicant run --file api-test.yaml\n```\n\nIf running locally from with the replicant binary a local chrome web browser with the development protocol can be specified:\n\n```bash\n/path/to/replicant run --chrome-remote-url http://127.0.0.1:9222  --file web-test.yaml\n```\n\nTo have the local chrome browser started with the developer protocol enabled:\n\n```bash\n/path/to/chrome --remote-debugging-port=9222\n```\n\n### Configuration options\n\nPlease see:\n\n```bash\n/path/to/replicant --help\n```\n\n### Replicant server and executor locally with docker\n\nThe unbabel/replicant docker image packs everything needed to run and manage tests for both web apps and APIs.\nSee the example `docker-compose.yaml` for more information.\n\n```bash\ndocker stack deploy -c $PWD/docker-compose.yaml replicant\n```\n\nThis will deploy the replicant server and 2 replicant executor nodes for web tests.\n\n### Web application testing\n\nWeb application testing support is based on the FQL (Ferret Query Language), [documentation](https://github.com/MontFerret/ferret).\n\n#### Test definition (can be also in json format)\n\n```yaml\nPOST http://127.0.0.1:8080/api/v1/run\ncontent-type: application/yaml\n\nname: duckduckgo-web-search\ndriver: web\nschedule: '@every 60s'\ntimeout: 50s\nretry_count: 2\ninputs:\n  url: \"https://duckduckgo.com\"\n  user_agent: \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36\"\n  timeout: 5000000\n  text: \"blade runner\"\nmetadata:\n  transaction: website-search\n  application: duckduckgo\n  environment: production\n  component: website\nscript: |\n  LET doc = DOCUMENT('{{ index . \"url\" }}', { driver: \"cdp\", userAgent: \"{{ index . \"user_agent\" }}\"})\n  INPUT(doc, '#search_form_input_homepage', \"{{ index . \"text\" }}\")\n  CLICK(doc, '#search_button_homepage')\n  WAIT_NAVIGATION(doc)\n  LET result = ELEMENT(doc, '#r1-0 \u003e div \u003e div.result__snippet.js-result-snippet').innerText\n  RETURN {\n    failed: result == \"\",\n    message: \"search result\",\n    data: result,\n  }\n```\n\n#### Response\n\n```json\n{\n  \"data\": [\n    {\n      \"uuid\": \"01DSSR5GH2BPX4G5FFCEVPEBKK\",\n      \"name\": \"duckduckgo-web-search\",\n      \"driver\": \"web\",\n      \"failed\": true,\n      \"message\": \"\",\n      \"data\": \"\",\n      \"time\": \"2019-11-16T09:19:39.554976Z\",\n      \"metadata\": {\n        \"application\": \"duckduckgo\",\n        \"component\": \"website\",\n        \"environment\": \"production\",\n        \"transaction\": \"website-search\"\n      },\n      \"retry_count\": 0,\n      \"with_callback\": false,\n      \"duration_seconds\": 6.967938203,\n      \"error\": \"operation timed out: WAIT_NAVIGATION(doc) at 4:0\"\n    }\n  ]\n}\n```\n\n### API testing\n\n##### Using the javascript driver\nThe following API is exposed by the javascript driver in order to perform HTTP calls and logging:\n* `replicant.Log(string)` log messages from the javascript test on the replicant server log.\n\n\n* `replicant.NewResult()` create a new response object to be returned as a result of the test, which should be modified accordingly to reflect the test result. The response must be returned as a serialized JSON object by calling its bounded method `Response.JSON`, E.g. `return response.JSON()`.\n\nResult type attributes:\n```js\n{\n\t\tData: \"\",\n\t\tMessage: \"\",\n\t\tFailed: false,\n}\n```\n\n* `replicant.http.NewRequest()` creates a new HTTP request object for performing HTTP calls.\n\nHttpRequest attributes:\n```js\n{\n\t\tURL: \"\",\n\t\tMethod: \"\",\n\t\tBody: \"\",\n\t\tHeader: {},\n\t\tParams: {},\n\t\tFormData: {},\n\t\tSSLSkipVerify: false,\n```\n\n* `replicant.http.Do(HttpRequest) performs a HTTP request and returns its response.\n\nHttpResponse attributes:\n```js\n{\n\tStatus: \"\"\n\tStatusCode: 200\n\tProtocol: \"\"\n\tBody: \"\"\n\tHeader: {}\n\tError: \"\"\n}\n```\n\n#### Test definition (can be also in JSON format)\n\n```yaml\nPOST http://127.0.0.1:8080/api/v1/run\ncontent-type: application/yaml\n\nname: duckduckgo-api-search\ndriver: javascript\nschedule: '@every 60s'\ntimeout: 60s\nretry_count: 2\ninputs:\n  url: \"https://api.duckduckgo.com\"\n  text: \"blade runner\"\nmetadata:\n  transaction: api-search\n  application: duckduckgo\n  environment: production\n  component: api\nscript: |\n  function Run(ctx) {\n    req = replicant.http.NewRequest()\n    req.URL = \"{{ index . \"url\" }}\"\n    req.Params.q = \"{{ index . \"text\" }}\"\n    req.Params.format = \"json\"\n    req.Params.no_redirect = \"1\"\n    resp = replicant.http.Do(req)\n    data = JSON.parse(resp.Body)\n    rr = replicant.NewResponse()\n    switch(data.RelatedTopics \u0026\u0026 data.RelatedTopics.length \u003e 0) {\n      case true:\n        rr.Data = data.RelatedTopics[0].Text\n        rr.Message = resp.Status\n        rr.Failed = false\n        break\n      case false:\n        rr.Data = JSON.stringify(data)\n        rr.Message = resp.Status\n        rr.Failed = true\n        break\n    }\n    return rr.JSON()\n  }\n```\n\n\n##### Using the Go driver\nStandard Go code can be used to create tests using following rules:\n* The package name must be `transaction`\n* The test function must implement the following signature: `func Run(ctx context.Context) (message string, data string, err error)`.\n\n***Keep in mind that unlike the javascript driver which doesn't expose any I/O or lower level functionality for accessing the underlying OS, the Go driver currently exposes all of the Go standard library. Only use this driver if you are absolutely sure of what you are doing. This is planned to change in the future.***\n\n#### Test definition (can be also in JSON format)\n```yaml\nPOST http://127.0.0.1:8080/api/v1/run\ncontent-type: application/yaml\n\nname: duckduckgo-api-search\ndriver: go\nschedule: '@every 60s'\ntimeout: 60s\nretry_count: 2\ninputs:\n  url: \"https://api.duckduckgo.com/\"\n  text: \"blade runner\"\nmetadata:\n  transaction: api-search\n  application: duckduckgo\n  environment: production\n  component: api\nscript: |\n  package transaction\n  import \"bytes\"\n  import \"context\"\n  import \"fmt\"\n  import \"net/http\"\n  import \"io/ioutil\"\n  import \"net/http\"\n  import \"regexp\"\n  func Run(ctx context.Context) (m string, d string, err error) {\n    req, err := http.NewRequest(http.MethodGet, \"{{ index . \"url\" }}\", nil)\n      if err != nil {\n        return \"request build failed\", \"\", err\n    }\n    req.Header.Add(\"Accept-Charset\",\"utf-8\")\n    q := req.URL.Query()\n    q.Add(\"q\", \"{{ index . \"text\" }}\")\n    q.Add(\"format\", \"json\")\n    q.Add(\"pretty\", \"1\")\n    q.Add(\"no_redirect\", \"1\")\n    req.URL.RawQuery = q.Encode()\n    client := \u0026http.Client{}\n    resp, err := client.Do(req)\n    if err != nil {\n      return \"failed to send request\", \"\", err\n    }\n    buf, err := ioutil.ReadAll(resp.Body)\n    if err != nil {\n      return \"failed to read response\", \"\", err\n    }\n    rx, err := regexp.Compile(`\"Text\"\\s*:\\s*\"(.*?)\"`)\n    if err != nil {\n      return \"failed to compile regexp\", \"\", err\n    }\n    s := rx.FindSubmatch(buf)\n    if len(s) \u003c 2 {\n      return \"failed to find data\", \"\", fmt.Errorf(\"no match\")\n    }\n    return \"search result\", fmt.Sprintf(\"%s\", s[1]), nil\n  }\n```\n\n#### Response\n\n```json\n{\n  \"data\": [\n    {\n      \"uuid\": \"01DSSR7ST5Q1Y2Y7HDSQDNS7Y7\",\n      \"name\": \"duckduckgo-api-search\",\n      \"driver\": \"go\",\n      \"failed\": false,\n      \"message\": \"search result\",\n      \"data\": \"Blade Runner A 1982 American neo-noir science fiction film directed by Ridley Scott, written by Hampton...\",\n      \"time\": \"2019-11-16T09:20:54.597852Z\",\n      \"metadata\": {\n        \"application\": \"duckduckgo\",\n        \"component\": \"api\",\n        \"environment\": \"production\",\n        \"transaction\": \"api-search\"\n      },\n      \"retry_count\": 0,\n      \"with_callback\": false,\n      \"duration_seconds\": 0.486582328,\n      \"error\": \"\"\n    }\n  ]\n}\n```\n\n## API\n\n| Method | Resource              | Action                                                  |\n|--------|-----------------------|---------------------------------------------------------|\n| POST   | /v1/transaction       | Add a managed transaction                               |\n| GET    | /v1/transaction       | Get all managed transaction definitions                 |\n| GET    | /v1/transaction/:name | Get a managed transaction definition by name            |\n| DELETE | /v1/transaction/:name | Remove a managed transaction                            |\n| POST   | /v1/run               | Run an ad-hoc transaction                               |\n| POST   | /v1/run/:name         | Run a managed transaction by name                       |\n| GET    | /v1/result            | Get all managed transaction last execution results      |\n| GET    | /v1/result/:name      | Get the latest result for a managed transaction by name |\n| GET    | /metrics              | Get metrics (prometheus emitter must be enabled)        |\n| GET    | /debug/pprof          | Get available runtime profile data (debug enabled)      |\n| GET    | /debug/pprof/:profile | Get profile data (for pprof, debug enabled)             |\n\n## TODO\n\n* Tests\n* Developer and user documentation\n* Add support for more conventional persistent stores\n* Vault integration for secrets (inputs)\n* Architecture and API documentation\n* Javascript driver transaction support\n\n\n## Acknowledgements\n\n* [Yaegi is Another Elegant Go Interpreter](https://github.com/containous/yaegi)\n* [Ferret Declarative web scraping](https://github.com/MontFerret/ferret)\n* [otto is a JavaScript parser and interpreter written natively in Go](https://github.com/robertkrimen/otto)\n\n## Contact\n\nBruno Moura [brunotm@gmail.com](mailto:brunotm@gmail.com)\n\n## License\n\nReplicant source code is available under the Apache Version 2.0 [License](https://github.com/Unbabel/replicant/blob/master/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUnbabel%2Freplicant","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FUnbabel%2Freplicant","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUnbabel%2Freplicant/lists"}