{"id":19451723,"url":"https://github.com/pxyup/fitter","last_synced_at":"2025-04-05T17:04:33.858Z","repository":{"id":78602331,"uuid":"597833877","full_name":"PxyUp/fitter","owner":"PxyUp","description":"New way for collect information from the API's/Websites","archived":false,"fork":false,"pushed_at":"2025-03-18T00:18:48.000Z","size":46524,"stargazers_count":122,"open_issues_count":3,"forks_count":9,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-29T16:04:04.749Z","etag":null,"topics":["api","go","golang","json","parsing","scraper","scrawler"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PxyUp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-05T19:18:13.000Z","updated_at":"2025-03-22T10:27:40.000Z","dependencies_parsed_at":"2024-02-07T20:46:27.680Z","dependency_job_id":"648057c9-e65a-456a-87e3-2942b331fcb6","html_url":"https://github.com/PxyUp/fitter","commit_stats":{"total_commits":169,"total_committers":3,"mean_commits":"56.333333333333336","dds":"0.017751479289940808","last_synced_commit":"f988e2d67b4e08e91a6419a8190a774fff35e869"},"previous_names":[],"tags_count":73,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PxyUp%2Ffitter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PxyUp%2Ffitter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PxyUp%2Ffitter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PxyUp%2Ffitter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PxyUp","download_url":"https://codeload.github.com/PxyUp/fitter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247369953,"owners_count":20927928,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","go","golang","json","parsing","scraper","scrawler"],"created_at":"2024-11-10T16:42:45.603Z","updated_at":"2025-04-05T17:04:33.833Z","avatar_url":"https://github.com/PxyUp.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [Fitter + Fitter CLI](https://github.com/PxyUp/fitter)\n\nFitter - new way for collect information from the API's/Websites\n\nFitter CLI - small cli command which provide result from Fitter for test/debug/home usage\n\nFitter Lib - library which provide functional of fitter CLI as a library\n\n![](https://github.com/PxyUp/fitter/blob/master/demo.gif)\n\n\n# Way to collect information\n\n1. **Server** - parsing response from some API's or http request(usage of http.Client)\n2. **Browser** - emulate real browser using chromium + docker + playwright/cypress and get DOM information\n3. **Static** - parsing static string as data\n\n# Format which can be parsed\n\n1. **JSON** - parsing JSON to get specific information\n2. **XML** - parsing xml tree to get specific information\n3. **HTML** - parsing dom tree to get specific information\n4. **XPath** - parsing dom tree to get specific information but by xpath\n\n# Use like a library\n\n```bash\ngo get github.com/PxyUp/fitter\n```\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"github.com/PxyUp/fitter/lib\"\n\t\"github.com/PxyUp/fitter/pkg/config\"\n\t\"log\"\n\t\"net/http\"\n)\n\nfunc main() {\n\tres, err := lib.Parse(\u0026config.Item{\n\t\tConnectorConfig: \u0026config.ConnectorConfig{\n\t\t\tResponseType:  config.Json,\n\t\t\tUrl:           \"https://random-data-api.com/api/appliance/random_appliance\",\n\t\t\tServerConfig: \u0026config.ServerConnectorConfig{\n\t\t\t\tMethod: http.MethodGet,\n\t\t\t},\n\t\t},\n\t\tModel: \u0026config.Model{\n\t\t\tObjectConfig: \u0026config.ObjectConfig{\n\t\t\t\tFields: map[string]*config.Field{\n\t\t\t\t\t\"my_id\": {\n\t\t\t\t\t\tBaseField: \u0026config.BaseField{\n\t\t\t\t\t\t\tType: config.Int,\n\t\t\t\t\t\t\tPath: \"id\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"generated_id\": {\n\t\t\t\t\t\tBaseField: \u0026config.BaseField{\n\t\t\t\t\t\t\tGenerated: \u0026config.GeneratedFieldConfig{\n\t\t\t\t\t\t\t\tUUID: \u0026config.UUIDGeneratedFieldConfig{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"generated_array\": {\n\t\t\t\t\t\tArrayConfig: \u0026config.ArrayConfig{\n\t\t\t\t\t\t\tRootPath: \"@this|@keys\",\n\t\t\t\t\t\t\tItemConfig: \u0026config.ObjectConfig{\n\t\t\t\t\t\t\t\tField: \u0026config.BaseField{\n\t\t\t\t\t\t\t\t\tType: config.String,\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}, nil, nil, nil, nil)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(res.ToJson())\n}\n\n\n```\n\nOutput:\n\n```json\n{\n  \"generated_array\": [\"id\",\"uid\",\"brand\",\"equipment\"],\n  \"my_id\": 6000,\n  \"generated_id\": \"26b08b73-2f2e-444d-bcf2-dac77ac3130e\"\n}\n```\n\n# How to use Fitter\n\n[Download latest version from the release page](https://github.com/PxyUp/fitter/releases)\n\nor locally:\n```bash\ngo run cmd/fitter/main.go --path=./examples/config_api.json\n```\n\n### Arguments\n1. **--path** - string[\"\"] - path for the configuration of the Fitter\n2. **--url** - string[\"\"] - url for the configuration of the Fitter\n3. **--verbose** - bool[false] - enable logging\n4. **--plugins** - string[\"\"] - [path for plugins for Fitter](https://github.com/PxyUp/fitter/blob/master/examples/plugin/README.md)\n5. **--log-level** - enum[\"info\", \"error\", \"debug\", \"fatal\"] - set log level(only if verbose set to true)\n\n# How to use Fitter_CLI\n\n[Download latest version from the release page](https://github.com/PxyUp/fitter/releases)\n\nor locally:\n```bash\ngo run cmd/cli/main.go --path=./examples/cli/config_cli.json\n```\n\n### Arguments\n1. **--path** - string[\"\"] - path for the configuration of the Fitter_CLI\n2. **--url** - string[\"\"] - url for the configuration of the Fitter_CLI\n3. **--copy** - bool[false] - copy information into clipboard\n4. **--pretty** - bool[true] - make readable result(also affect on copy)\n5. **--verbose** - bool[false] - enable logging\n6. **--omit-error-pretty** - bool[false] -  Provide pure value if pretty is invalid\n7. **--plugins** - string[\"\"] - [path for plugins for Fitter](https://github.com/PxyUp/fitter/blob/master/examples/plugin/README.md)\n8. **--log-level** - enum[\"info\", \"error\", \"debug\", \"fatal\"] - set log level(only if verbose set to true)\n9. **--input** - string[\"\"] - specify input value for [formatting](#placeholder-list). Examples: `--input=\\\"\"124\"\\\"` `--input=124` `--input='{\"test\": 5}'`\n\n```bash\n./fitter_cli_${VERSION} --path=./examples/cli/config_cli.json --copy=true\n```\n\nExamples:\n1. **Server version** [HackerNews + Quotes + Guardian News](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_cli.json) - using API + HTML + XPath parsing\n2. **Chromium version** [Guardian News + Quotes](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_browser.json) - using HTML parsing + browser emulation\n3. **Docker version** [Docker version: Guardian News + Quotes](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_docker.json) - using HTML parsing + browser from Docker image\n4. **Playwright version** [Playwright version: Guardian News + Quotes](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_playwright.json) - using HTML parsing + browser from Playwright framework\n5. **Playwright version** [Playwright version: England Cities + Weather](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_weather.json) - using HTML + XPath parsing + browser from Playwright framework\n6. **JSON version** [Generate pagination](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_static_connector.json) - using static connector for generate pagination array\n7. **Server version** [Get current time](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_current_time.json) - get time from url and format it\n\n\n\n# Configuration\n\n## Connector\nIt is the way how you fetch the data\n\n```go\ntype ConnectorConfig struct {\n    ResponseType ParserType `json:\"response_type\" yaml:\"response_type\"`\n    Url          string     `json:\"url\" yaml:\"url\"`\n    Attempts     uint32     `json:\"attempts\" yaml:\"attempts\"`\n    \n    NullOnError bool `yaml:\"null_on_error\" json:\"null_on_error\"`\n    \n    StaticConfig          *StaticConnectorConfig      `json:\"static_config\" yaml:\"static_config\"`\n    IntSequenceConfig     *IntSequenceConnectorConfig `json:\"int_sequence_config\" yaml:\"int_sequence_config\"`\n    ServerConfig          *ServerConnectorConfig      `json:\"server_config\" yaml:\"server_config\"`\n    BrowserConfig         *BrowserConnectorConfig     `yaml:\"browser_config\" json:\"browser_config\"`\n    PluginConnectorConfig *PluginConnectorConfig      `json:\"plugin_connector_config\" yaml:\"plugin_connector_config\"`\n    ReferenceConfig       *ReferenceConnectorConfig   `yaml:\"reference_config\" json:\"reference_config\"`\n    FileConfig            *FileConnectorConfig        `json:\"file_config\" yaml:\"file_config\"`\n}\n```\n\n- NullOnError[false] - if set to true then all errors a ignored\n- ResponseType - enum[\"HTML\", \"json\",\"xpath\"] - in which format data comes from the connector\n- Attempts - how many attempts to use for fetch data by connector\n- Url - define which address to request. Important: can be with [inject of the parent value as a string](#placeholder-list)\n`https://api.open-meteo.com/v1/forecast?latitude={{{latitude}}}\u0026longitude={{{longitude}}}\u0026hourly=temperature_2m\u0026forecast_days=1`\n\nConfig can be one of:\n- [ServerConfig](#serverconnectorconfig)\n- [BrowserConfig](#browserconnectorconfig)\n- [StaticConfig](#staticconnectorconfig)\n- [PluginConnectorConfig](#pluginconnectorconfig)\n- [ReferenceConfig](#referenceconnectorconfig)\n- [IntSequenceConfig](#intsequenceconnectorconfig)\n- [FileConfig](#fileconnectorconfig)\n\nExample:\n```json\n{\n  \"response_type\": \"xpath\",\n  \"attempts\": 3,\n  \"url\": \"https://openweathermap.org/find?q={PL}\",\n  \"browser_config\": {\n    \"playwright\": {\n      \"timeout\": 30,\n      \"wait\": 30,\n      \"install\": false,\n      \"browser\": \"Chromium\"\n    }\n  }\n}\n```\n\n### PluginConnectorConfig\nConnector can be defined via plugin system. For use that you need apply next flags to Fitter/Cli(location of the plugins):\n```bash\n... --plugins=./examples/plugin\n```\n\n--plugins - looking for all files with \".so\" extension in provided folder(subdirs excluded)\n\n\n\n```go\ntype PluginConnectorConfig struct {\n\tName   string          `json:\"name\" yaml:\"name\"`\n\tConfig json.RawMessage `json:\"config\" yaml:\"config\"`\n}\n```\n\n```json\n{\n    \"name\": \"connector\",\n    \"config\": {\n      \"name\": \"Elon\"\n    }\n}\n```\n\n- Name - name of the plugin\n- Config - json config of the plugin\n\n\n#### How to build plugin\n\nBuild plugin\n```bash\ngo build -buildmode=plugin -gcflags=\"all=-N -l\" -o examples/plugin/connector.so examples/plugin/hardcoder/connector.go\n```\n\nMake sure you export **Plugin** variable which implements **pl.ConnectorPlugin** interface\n\nExample for CLI:\n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_plugin.json#L5\n\nPlugin example:\n```go\npackage main\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"github.com/PxyUp/fitter/pkg/config\"\n\t\"github.com/PxyUp/fitter/pkg/logger\"\n\t\"github.com/PxyUp/fitter/pkg/builder\"\n\tpl \"github.com/PxyUp/fitter/pkg/plugins/plugin\"\n)\n\nvar (\n\t_ pl.ConnectorPlugin = \u0026plugin{}\n\n\tPlugin plugin\n)\n\ntype plugin struct {\n\tlog  logger.Logger\n\tName string `json:\"name\" yaml:\"name\"`\n}\n\nfunc (pl *plugin) Get(parsedValue builder.Interfacable, index *uint32, input builder.Interfacable) ([]byte, error) {\n\treturn []byte(fmt.Sprintf(`{\"name\": \"%s\"}`, pl.Name)), nil\n}\n\nfunc (pl *plugin) SetConfig(cfg *config.PluginConnectorConfig, logger logger.Logger) {\n\tpl.log = logger\n\n\tif cfg.Config != nil {\n\t\terr := json.Unmarshal(cfg.Config, pl)\n\t\tif err != nil {\n\t\t\tpl.log.Errorw(\"cant unmarshal plugin configuration\", \"error\", err.Error())\n\t\t\treturn\n\t\t}\n\t}\n}\n```\n\n### ReferenceConnectorConfig\nConnector which allow get prefetched data from [references](#references)\n\n```go\ntype ReferenceConnectorConfig struct {\n\tName string `yaml:\"name\" json:\"name\"`\n}\n```\n\nExample\n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_ref.json#L66\n\n- Name - reference name from [references](#references) map\n\n### IntSequenceConnectorConfig\nImproved version of static connector which generate int sequence as result\n\n```go\ntype IntSequenceConnectorConfig struct {\n\tStart int `json:\"start\" yaml:\"start\"`\n\tEnd   int `json:\"end\" yaml:\"end\"`\n\tStep  int `json:\"step\" yaml:\"step\"`\n}\n```\n- Start[0] - start point for generation(**included**)\n- End[0] - end point for generation(**excluded** from final result like range in any lang)\n- Step[1] - interval for sequence\n\nExample\n```json\n{\n    \"start\": 0,\n    \"end\": 2 \n    // Generate [0, 1]\n}\n```\n\n\n[Config example](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_seq.json)\n\n\n### FileConnectorConfig\nConnector type which fetch data from provided file\n\n```go\ntype FileConnectorConfig struct {\n    Path          string `yaml:\"path\" json:\"path\"`\n    UseFormatting bool   `yaml:\"use_formatting\" json:\"use_formatting\"`\n}\n```\n\n- Path - file path. Support [formatting](#placeholder-list)\n- UseFormatting[false] - use [formatting](#placeholder-list) file content or not\n\n### StaticConnectorConfig\nConnector type which fetch data from provided string\n```go\ntype StaticConnectorConfig struct {\n    Value string `json:\"value\" yaml:\"value\"`\n    Raw   json.RawMessage `json:\"raw\" yaml:\"raw\"`\n}\n```\n\n- Value - static string as data, can be html, json\n- Raw - accept raw json. [Example](https://github.com/PxyUp/fitter/blob/master/examples/config_static.json). Also support [formatting](#placeholder-list)\n\nExample:\n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_static_connector.json#L5\n```json\n{\n  \"value\": \"[1,2,3,4,5]\"\n}\n```\n\n### ServerConnectorConfig\nConnector type which fetch data using golang http.Client(server side request like curl)\n\n```go\ntype ServerConnectorConfig struct {\n    Method      string            `json:\"method\" yaml:\"method\"`\n    Headers     map[string]string `yaml:\"headers\" json:\"headers\"`\n    Timeout     uint32            `yaml:\"timeout\" json:\"timeout\"`\n    JsonRawBody json.RawMessage   `json:\"json_raw_body\" yaml:\"json_raw_body\"`\n    Body        string            `yaml:\"body\" json:\"body\"`\n    \n    Proxy *ProxyConfig `yaml:\"proxy\" json:\"proxy\"`\n}\n```\n\n- Method - supported all http methods: GET, POST, PUT, DELETE, PATCH, OPTIONS, HEAD\n- Headers - predefine headers for using during request [can be injected into key/value](#placeholder-list)\n- Timeout[sec] - default 60sec timeout or used provided\n- Body - body of the request, parsed value [can be injected](#placeholder-list)\n- JsonRawBody - body of the request in json format; value [can be injected](#placeholder-list)\n- Proxy - setup proxy for request [config](#proxy-config)\n\nExample:\n```json\n{\n  \"method\": \"GET\",\n  \"proxy\": {\n    \"server\": \"http://localhost:8080\",\n    \"username\": \"pyx\"\n  }\n}\n```\n\nRight now default timeout it is 10 sec.\n\n##### Proxy config\n\n```go\ntype ProxyConfig struct {\n    // Proxy to be used for all requests. HTTP and SOCKS proxies are supported, for example\n    // `http://myproxy.com:3128` or `socks5://myproxy.com:3128`. Short form `myproxy.com:3128`\n    // is considered an HTTP proxy.\n    Server string `json:\"server\" yaml:\"server\"`\n    // Optional username to use if HTTP proxy requires authentication.\n    Username string `json:\"username\" yaml:\"username\"`\n    // Optional password to use if HTTP proxy requires authentication.\n    Password string `json:\"password\" yaml:\"password\"`\n}\n```\n\n- Server - address with schema of proxy server. Also support [formatting](#placeholder-list)\n- Username - username for proxy(can be empty). Also support [formatting](#placeholder-list)\n- Password - password for proxy(can be empty). Also support [formatting](#placeholder-list)\n\n```json\n{\n  \"server\": \"http://localhost:8080\",\n  \"username\": \"pyx\"\n}\n```\n\n##### Environment variables\n1. **FITTER_HTTP_WORKER** - int[1000] - default concurrent HTTP workers\n\n### BrowserConnectorConfig\nConnector type which emulate fetching of data via browser\n\n```go\ntype BrowserConnectorConfig struct {\n\tChromium   *ChromiumConfig   `json:\"chromium\" yaml:\"chromium\"`\n\tDocker     *DockerConfig     `json:\"docker\" yaml:\"docker\"`\n\tPlaywright *PlaywrightConfig `json:\"playwright\" yaml:\"playwright\"`\n}\n```\n\nConfig can be one of:\n- [Chromium](#chromium) - use local installed Chromium for fetch data\n- [Docker](#docker) - use docker as service for spin up container for fetch data\n- [Playwright](#playwright) - use playwright framework for fetch data\n\nExample:\n```json\n{\n    \"docker\": {\n      \"wait\": 10000,\n      \"image\": \"docker.io/zenika/alpine-chrome:with-node\",\n      \"entry_point\": \"chromium-browser\",\n      \"purge\": true\n    }\n}\n```\n\n#### Chromium\nUse locally installed Chromium for fetch the data\n\n```go\ntype ChromiumConfig struct {\n\tPath    string   `yaml:\"path\" json:\"path\"`\n\tTimeout uint32   `yaml:\"timeout\" json:\"timeout\"`\n\tWait    uint32   `yaml:\"wait\" json:\"wait\"`\n\tFlags   []string `yaml:\"flags\" json:\"flags\"`\n}\n```\n\n- Path - path to binary of Chromium\n- Timeout[sec] - timeout for execution of the chromium\n- Wait[msec] - timeout of page loading\n- Flags - flags for Chromium default: \"--headless\", \"--proxy-auto-detect\", \"--temp-profile\", \"--incognito\", \"--disable-logging\", \"--disable-extensions\", \"--no-sandbox\"\n\nExample:\n```json\n{\n  \"path\": \"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome\",\n  \"wait\": 10000\n}\n```\n\n#### Docker\nUse Docker for spin up container for fetch data\n\n```go\ntype DockerConfig struct {\n\tImage       string   `yaml:\"image\" json:\"image\"`\n\tEntryPoint  string   `json:\"entry_point\" yaml:\"entry_point\"`\n\tTimeout     uint32   `yaml:\"timeout\" json:\"timeout\"`\n\tWait        uint32   `yaml:\"wait\" json:\"wait\"`\n\tFlags       []string `yaml:\"flags\" json:\"flags\"`\n\tPurge       bool     `json:\"purge\" yaml:\"purge\"`\n\tNoPull      bool     `yaml:\"no_pull\" json:\"no_pull\"`\n\tPullTimeout uint32   `yaml:\"pull_timeout\" json:\"pull_timeout\"`\n}\n```\n\n\n**Docker default image**: docker.io/zenika/alpine-chrome\n\n- Image - image for the docker registry(provide with registry host)\n- EntryPoint - cmd which will be run inside container\n- Timeout[sec] - timeout for run container(without pulling image)\n- Wait[msec] - timeout of page loading (works just for Chromium based containers)\n- Flags - cmd arguments for run containers, default for Chromium based: \"--no-sandbox\",\"--headless\", \"--proxy-auto-detect\", \"--temp-profile\", \"--incognito\", \"--disable-logging\", \"--disable-gpu\"\n- Purge - should we remove container after work done(like docker rm)\n- NoPull - prevent pulling of the image\n- PullTimeout - define timeout for pull contains\n\n##### Environment variables\n1. **DOCKER_HOST** - string - (EnvOverrideHost) to set the URL to the docker server.\n2. **DOCKER_API_VERSION** - string - (EnvOverrideAPIVersion) to set the version of the API to use, leave empty for latest.\n3. **DOCKER_CERT_PATH** - string - (EnvOverrideCertPath) to specify the directory from which to load the TLS certificates (ca.pem, cert.pem, key.pem).\n4. **DOCKER_TLS_VERIFY** - bool - (EnvTLSVerify) to enable or disable TLS verification (off by default)\n\n\nExample:\n```json\n{\n  \"wait\": 10000,\n  \"image\": \"docker.io/zenika/alpine-chrome:with-node\",\n  \"entry_point\": \"chromium-browser\",\n  \"purge\": true\n}\n```\n\n#### Playwright\nRun browsers via playwright framework\n\n```go\ntype PlaywrightConfig struct {\n    Browser      PlaywrightBrowser          `json:\"browser\" yaml:\"browser\"`\n    Install      bool                       `yaml:\"install\" json:\"install\"`\n    Timeout      uint32                     `yaml:\"timeout\" json:\"timeout\"`\n    Wait         uint32                     `yaml:\"wait\" json:\"wait\"`\n    TypeOfWait   *playwright.WaitUntilState `json:\"type_of_wait\" yaml:\"type_of_wait\"`\n    PreRunScript string                     `json:\"pre_run_script\" yaml:\"pre_run_script\"`\n    Stealth      bool                       `json:\"stealth\" yaml:\"stealth\"`\n    \n    Proxy *ProxyConfig `yaml:\"proxy\" json:\"proxy\"`\n}\n```\n\n- Browser - enum[\"Chromium\", \"FireFox\", \"WebKit\"] - which browser to use\n- Install - should we install browser\n- Timeout[sec] - timeout to run playwright\n- Wait[sec] - timeout of page loading\n- TypeOfWait - enum[\"load\", \"domcontentloaded\", \"networkidle\", \"commit\"] which state of page we waiting, default is \"load\"\n- PreRunScript[\"\"] - script which will be executed before reading content of the page. Also support placeholder [{PL}](#placeholder-list)\n- Stealth[false] - add script for trying passing bot defends\n- Proxy - setup proxy for request [config](#proxy-config)\n\nExample\n```json\n{\n  \"timeout\": 30,\n  \"wait\": 30,\n  \"install\": false,\n  \"browser\": \"Chromium\"\n}\n```\n\n## Model\nWith model we define result of the scrapping\n\n```go\ntype Model struct {\n    ObjectConfig *ObjectConfig `yaml:\"object_config\" json:\"object_config\"`\n    ArrayConfig  *ArrayConfig  `json:\"array_config\" yaml:\"array_config\"`\n    BaseField    *BaseField    `json:\"base_field\" yaml:\"base_field\"`\n    IsArray      bool          `json:\"is_array\" yaml:\"is_array\"`\n}\n```\n\nConfig can be one of:\n- [ObjectConfig](#objectconfig) - configuration of object format\n- [ArrayConfig](#arrayconfig) - configuration of array format\n- [BaseField](#basefield) - configuration of single/generated field \n- IsArray - bool[false] - force indicate that field is array(usable in case of [model field](#model-field) with [base field](#basefield))\n\nExample:\n```json\n{\n  \"object_config\": {}\n}\n```\n\n### ObjectConfig\nConfiguration of the object and fields\n\n```go\ntype ObjectConfig struct {\n    Fields      map[string]*Field `json:\"fields\" yaml:\"fields\"`\n    Field       *BaseField        `json:\"field\" yaml:\"field\"`\n    ArrayConfig *ArrayConfig      `json:\"array_config\" yaml:\"array_config\"`\n}\n```\n\nConfig can be one of:\n- [Fields](#field) - map of each field definition; key - field name, value - configuration\n- [Field](#basefield) - used for element of array; fields which will be deserialized like basic type like \"string\", \"int\" and etc (used here for case array of basic types)\n- [ArrayConfig](#arrayconfig) - used for element of array; deserialization array of array\n\nExample:\n```json\n{\n  \"fields\": {\n    \"title\": {\n      \"base_field\": {\n        \"type\": \"string\",\n        \"path\": \"type\"\n      }\n    }\n  }\n}\n```\n\n### ArrayConfig\nConfiguration of the array and fields\n\n```go\ntype ArrayConfig struct {\n    RootPath    string        `json:\"root_path\" yaml:\"root_path\"`\n    Reverse     bool          `yaml:\"reverse\" json:\"reverse\"`\n    \n    ItemConfig  *ObjectConfig `json:\"item_config\" yaml:\"item_config\"`\n    LengthLimit uint32        `json:\"length_limit\" yaml:\"length_limit\"`\n    \n    StaticConfig *StaticArrayConfig `json:\"static_array\"  yaml:\"static_array\"`\n}\n```\n\n- RootPath - selector for find root element of the array or repeated element in case of html parsing, size of array will be amount of children element under the root\n- Reverse - bool[false] - indicate that need use reverse iteration(n to 1)\n- LengthLimit - for define size of array only for generated(not working for static)\n\nConfig can be one of:\n- [ItemConfig](#objectconfig) - configuration of each element of the array \n- [StaticConfig](#static-array-config) - configuration of the static array\n\nExample:\n```json\n{\n  \"root_path\": \"#content dt.quote \u003e a\",\n  \"item_config\": {\n    \"field\": {\n      \"type\": \"string\"\n    }\n  }\n}\n```\n\n#### Field\nCommon of the field\n\n\n```go\ntype Field struct {\n\tBaseField    *BaseField    `json:\"base_field\" yaml:\"base_field\"`\n\tObjectConfig *ObjectConfig `json:\"object_config\" yaml:\"object_config\"`\n\tArrayConfig  *ArrayConfig  `json:\"array_config\" yaml:\"array_config\"`\n\n\tFirstOf []*Field `json:\"first_of\" yaml:\"first_of\"`\n}\n```\n\nConfig can be one of: \n- [BaseField](#basefield) - fields which will be deserialized like basic type like \"string\", \"int\" and etc\n- [ObjectConfig](#objectconfig) - in case our field in nested object\n- [ArrayConfig](#arrayconfig) - in case our field in array\n- [FirstOf](#field) - first not empty resolved field will be selected\n\nExample:\n```json\n{\n  \"base_field\": {\n    \"type\": \"string\",\n    \"path\": \"div.current-temp span.heading\"\n  }\n}\n```\n\n#### BaseField\nIn case we want get some static information or generate new one\n\n```go\ntype BaseField struct {\n\tType FieldType `yaml:\"type\" json:\"type\"`\n\tPath string    `yaml:\"path\" json:\"path\"`\n\n\tHTMLAttribute string `json:\"html_attribute\" yaml:\"html_attribute\"`\n\n\tGenerated *GeneratedFieldConfig `yaml:\"generated\" json:\"generated\"`\n\n\tFirstOf []*BaseField `json:\"first_of\" yaml:\"first_of\"`\n}\n```\n\n- FieldType - enum[\"null\", \"boolean\", \"string\", \"int\", \"int64\", \"float\", \"float64\", \"array\", \"object\", \"html\", \"raw_string\"] - static field for parse. **Important**: type html will only works from connector which return HTML (HTMLAttribute - have no effect in this case). [Example](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_ref.json#L25) \n- Path - selector(relative in case it is array child) for parsing\n- HTMLAttribute - extra value which have effect only in HTML parsing via **goquery**. Here you can specify which attribute need to be parsed.\n\n**Important**: by default \"string\" type trimmed and all special chars is replaced, if you need plain string use \"raw_string\"\n\nConfig can be one of or empty:\n- [Generated](#generatedfieldconfig) - field can be generated one which custom configuration\n- [FirstOf](#basefield) - first not empty resolved field will be selected\n\nExamples\n```json\n{\n  \"generated\": {\n    \"uuid\": {}\n  }\n}\n```\n\n```json\n{\n  \"type\": \"string\",\n  \"path\": \"text()\"\n}\n```\n\n#### GeneratedFieldConfig\nProvide functionality of generating field on the flight\n\n```go\ntype GeneratedFieldConfig struct {\n    UUID             *UUIDGeneratedFieldConfig   `yaml:\"uuid\" json:\"uuid\"`\n    Static           *StaticGeneratedFieldConfig `yaml:\"static\" json:\"static\"`\n    Formatted        *FormattedFieldConfig       `json:\"formatted\" yaml:\"formatted\"`\n    Plugin           *PluginFieldConfig          `yaml:\"plugin\" json:\"plugin\"`\n    Calculated       *CalculatedConfig           `yaml:\"calculated\" json:\"calculated\"`\n    File             *FileFieldConfig            `yaml:\"file\" json:\"file\"`\n    Model            *ModelField                 `yaml:\"model\" json:\"model\"`\n    FileStorageField *FileStorageField           `json:\"file_storage\" yaml:\"file_storage\"`\n}\n```\n\nConfig can be one of:\n- [UUID](#uuid) - generate random UUID V4\n- [Static](#static) - generate static field\n- [Formatted](#formatted-field-config) - format field\n- [Model](#model-field) - model generated from the other connector and model\n- [Plugin](#plugin-field) - plugin field\n- [Calculated](#calculated-field) - calculated field\n- [File](#file-field) - file field (for download file from server)\n- [FileStorage](#file-storage-field) - file field which can be saved to local file\n\nExamples:\n```json\n{\n    \"uuid\": {}\n}\n```\n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_cli.json#L58\n```json\n{\n    \"model\": {\n      \"type\": \"array\",\n      \"model\": {\n        \"array_config\": {\n          \"root_path\": \"#content dt.quote \u003e a\",\n          \"item_config\": {\n            \"field\": {\n              \"type\": \"string\"\n            }\n          }\n        }\n      },\n      \"connector_config\": {\n        \"response_type\": \"HTML\",\n        \"attempts\": 3,\n        \"browser_config\": {\n          \"url\": \"http://www.quotationspage.com/random.php\",\n          \"chromium\": {\n            \"path\": \"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome\",\n            \"wait\": 10000\n          }\n        }\n      }\n    }\n}\n```\n\n#### UUID\nGenerate random UUID V4 on the flight, can be used for generate uniq id\n\n```go\ntype UUIDGeneratedFieldConfig struct {\n\tRegexp string `yaml:\"regexp\" json:\"regexp\"`\n}\n```\n\n- Regexp - provide matcher which can be used for get part of generated uuid\n\n#### Static\nGenerate static field\n\n```go\ntype StaticGeneratedFieldConfig struct {\n    Type  FieldType       `yaml:\"type\" json:\"type\"`\n    Value string          `json:\"value\" yaml:\"value\"`\n    Raw   json.RawMessage `json:\"raw\" yaml:\"raw\"`\n}\n```\n\n- Type - enum[\"null\", \"boolean\", \"string\", \"int\",\"int64\",\"float\",\"float64\", \"array\", \"object\"] - type of the field\n- Value - string value of the field\n- Raw - pure json value of the field\n\nExample\n```json\n{\n  \"type\": \"int\",\n  \"value\": \"65\"\n}\n```\n\n```json\n{\n  \"type\": \"array\",\n  \"value\": \"[65,45]\"\n}\n```\n\n```json\n{\n  \"type\": \"array\",\n  \"raw\": [65,45]\n}\n```\n\n#### Formatted Field Config\nGenerate formatted field which will pass value from parent [base field](#basefield)\n\n```go\ntype FormattedFieldConfig struct {\n\tTemplate string `yaml:\"template\" json:\"template\"`\n}\n```\n\n- Template - template in with placeholder [{PL}](#placeholder-list) where [parent](#basefield) value will be injected like string\n \nExample:\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_cli.json#L98\n```json\n{\n  \"template\": \"https://news.ycombinator.com/item?id={PL}\"\n}\n```\n\n#### File Storage Field\n\nField can be used for store field result as local file\n\n```go\ntype FileStorageField struct {\n    Content string          `json:\"content\" yaml:\"content\"`\n    Raw     json.RawMessage `yaml:\"raw\" yaml:\"raw\"`\n    \n    FileName string `json:\"file_name\" yaml:\"file_name\"`\n    Path     string `json:\"path\" yaml:\"path\"`\n    Append   bool   `json:\"append\" yaml:\"append\"`\n}\n```\n\n- Content - template string for content. Important: can be with [inject of the parent value as a string](#placeholder-list)\n- Raw - raw json content of field. Important: can be with [inject of the parent value as a string](#placeholder-list)\n- FileName - local file name for storing file. By default, it is try get FileName from header, after that from url. Important: can be with [inject of the parent value as a string](#placeholder-list).\n- Path - local file parent directory for storing file. Default path it is process directory. Important: can be with [inject of the parent value as a string](#placeholder-list)\n- Append[false] - append to file or not\n\n\n```json\n{\n  \"content\": \"{{{id}}}, {{{message}}}\\n\",\n  \"append\": true,\n  \"file_name\": \"{{{id}}}.csv\",\n  \"path\": \"/Users/pxyup/fitter/examples/cli/test/csv\"\n}\n```\n\n\n#### File Field\n\nField can be used for download file from server locally\n\n```go\ntype FileFieldConfig struct {\n\tConfig *ServerConnectorConfig `yaml:\"config\" json:\"config\"`\n\n\tUrl      string `yaml:\"url\" json:\"url\"`\n\tFileName string `json:\"file_name\" yaml:\"file_name\"`\n\tPath     string `json:\"path\" yaml:\"path\"`\n}\n```\n\n- Config - [ServerConfig](#serverconnectorconfig) use default fitter http.Client for send request\n- Url - url of the image. Important: URL in the connector can be with [inject of the parent value as a string](#placeholder-list)\n- FileName - local file name for storing file. By default, it is try get FileName from header, after that from url. Important: can be with [inject of the parent value as a string](#placeholder-list).\n- Path - local file parent directory for storing file. Default path it is process directory. Important: can be with [inject of the parent value as a string](#placeholder-list)\n\nResult of the field will be local file path as string\n\n\n```json\n{\n  \"url\": \"https://images.shcdn.de/resized/w680/p/dekostoff-gobelinstoff-panel-oriental-cat-46-x-46_P19-KP_2.jpg\",\n  \"path\": \"/Users/pxyup/fitter/bin\",\n  \"config\": {\n    \"method\": \"GET\"\n  }\n}\n```\n\nWith propagated URL ([inject of the parent value as a string](#placeholder-list))\n\n```json\n{\n  \"url\": \"https://picsum.photos{PL}\",\n  \"path\": \"/Users/pxyup/fitter/bin\",\n  \"config\": {\n    \"method\": \"GET\"\n  }\n}\n```\n\nConfig example: \n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_image.json\n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_image_multiple.json\n\n\n#### Calculated field\n\nField can generate different types depends from expression\n\n```go\ntype CalculatedConfig struct {\n\tType       FieldType `yaml:\"type\" json:\"type\"`\n\tExpression string    `yaml:\"expression\" json:\"expression\"`\n}\n```\n\n- Type - resulting type of expression\\\n- Expression - expression for calculation (we use [this lib](https://github.com/expr-lang/expr) for calculated expression)\n\n##### Predefined values\n\n**FNull** - alias for builder.Nullvalue\n\n**FNil** - alias for nil\n\n**isNull(value T)** - function for check is value is FNull\n\n**fRes** - it is raw(with proper type) result from the parsing [base field](#basefield)\n\n**fIndex** - it is index in parent array(only if parent was array field)\n\n**fResJson** - it is JSON string representation of the raw result\n\n**fResRaw** - result in bytes format\n\n**FNewLine** - new line separator\n\n```json\n{\n  \"type\": \"bool\",\n  \"expression\": \"fRes \u003e 500\"\n}\n```\n\n#### Plugin field\n\nField can be some external plugin for fitter\n\n[More](https://github.com/PxyUp/fitter/blob/master/examples/plugin/README.md)\n\n```go\ntype PluginFieldConfig struct {\n\tName string `json:\"name\" yaml:\"name\"`\n\tConfig json.RawMessage `json:\"config\" yaml:\"config\"`\n}\n```\n\n- Name - name of the plugin(without extension just name)\n- Config - json config of the plugin\n\n#### Model Field\nField type which can be generated on the flight by news [model](#model) and [connector](#connector)\n\n```go\ntype ModelField struct {\n\t// Type of parsing\n\tConnectorConfig *ConnectorConfig `yaml:\"connector_config\" json:\"connector_config\"`\n\t// Model of the response\n\tModel *Model `yaml:\"model\" json:\"model\"`\n\n\tType FieldType `yaml:\"type\" json:\"type\"`\n\tPath string             `yaml:\"path\" json:\"path\"`\n\n\tExpression string    `yaml:\"expression\" json:\"expression\"`\n}\n```\n\n- [ConnectorConfig](#connector) - which connector to use. Important: URL in the connector can be with [inject of the parent value as a string](#placeholder-list)\n- [Model](#model) - configuration of the underhood model\n- Type - enum[\"null\", \"boolean\", \"string\", \"int\", \"int64\", \"float\", \"float64\", \"array\", \"object\"] - type of generated field\n- Path - in case we cant extract some information from generated field we can use json selector for extract\n- Expression - string which can be used for post processing of the Model (**ignoring path field**)\n\nExamples:\n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_cli.json#L60\n```json\n{\n  \"type\": \"array\",\n  \"model\": {\n    \"array_config\": {\n      \"root_path\": \"#content dt.quote \u003e a\",\n      \"item_config\": {\n        \"field\": {\n          \"type\": \"string\"\n        }\n      }\n    }\n  }\n}\n```\n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_weather.json#L37\n```json\n{\n    \"type\": \"string\",\n    \"path\": \"temp.temp\",\n    \"model\": {\n       \"object_config\": {\n        \"fields\": {\n          \"temp\": {\n            \"base_field\": {\n              \"type\": \"string\",\n              \"path\": \"//div[@id='forecast_list_ul']//td/b/a/@href\",\n              \"generated\": {\n                \"model\": {\n                  \"type\": \"string\",\n                  \"model\": {\n                    \"object_config\": {\n                      \"fields\": {\n                        \"temp\": {\n                          \"base_field\": {\n                            \"type\": \"string\",\n                            \"path\": \"div.current-temp span.heading\"\n                          }\n                        }\n                      }\n                    }\n                  },\n                  \"connector_config\": {\n                    \"response_type\": \"HTML\",\n                    \"attempts\": 4,\n                    \"url\": \"https://openweathermap.org{PL}\",\n                    \"browser_config\": {\n                      \"playwright\": {\n                        \"timeout\": 30,\n                        \"wait\": 30,\n                        \"install\": false,\n                        \"browser\": \"FireFox\",\n                        \"type_of_wait\": \"networkidle\"\n                      }\n                    }\n                  }\n                }\n              }\n            }\n          }\n        }\n      }\n    },\n    \"connector_config\": {\n      \"response_type\": \"xpath\",\n      \"attempts\": 3,\n      \"url\": \"https://openweathermap.org/find?q={PL}\",\n      \"browser_config\": {\n        \"playwright\": {\n          \"timeout\": 30,\n          \"wait\": 30,\n          \"install\": false,\n          \"browser\": \"Chromium\"\n        }\n      }\n    }\n}\n```\n\n#### Static Array Config\nProvide static(fixed length) array generation\n\n```go\ntype StaticArrayConfig struct {\n    Items map[uint32]*Field `yaml:\"items\" json:\"items\"`\n    Length uint32            `yaml:\"length\" json:\"length\"`\n}\n```\n- [Items](#field) - map[uint32]*[Field](#field) - key is index in array, value is field definition\n- Length - if set(1+) can be used for define custom length of array\n\nExamples:\n```json\n{\n  \"0\": {\n    \"base_field\": {\n      \"type\": \"string\",\n      \"path\": \"div.current-temp span.heading\"\n    }\n  }\n}\n```\n\n```json\n{\n  \"length\": 4,\n  \"0\": {\n    \"base_field\": {\n      \"type\": \"string\",\n      \"path\": \"div.current-temp span.heading\"\n    }\n  }\n}\n```\n\n```json\n{\n  \"length\": 4,\n  \"2\": {\n    \"base_field\": {\n      \"type\": \"string\",\n      \"path\": \"div.current-temp span.heading\"\n    }\n  }\n}\n```\n\n#### Placeholder list\n1. {PL} - for inject value\n2. {INDEX} - for inject index in parent array\n3. {HUMAN_INDEX} - for inject index in parent array in human way\n4. {{{json_path}}} - will get information from propagated \"object\"/\"array\" field\n5. {{{RefName=SomeName}}} - get [reference](#references) value by name. [Example](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_ref.json#L67)\n6. {{{RefName=SomeName json.path}}} - get [reference](#references) value by name and extract value by json path. [Example](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_ref.json#L67)\n7. {{{FromEnv=ENV_KEY}}} - get value from environment variable\n8. {{{FromExp=fRes + 5 + fIndex}}} - get value from the [expression](https://github.com/expr-lang/expr). [Predefined values](#predefined-values)  \n9. {{{FromInput=.}}} or {{{FromInput=json.path}}} - get value from input of trigger or library\n10. {{{FromFile=./test_file.log}}} - get value from file by path. Content of file also can contain placeholders\n11. {{{FromURL=http://localhost:8081}}} - get response from url \n\nExamples:\n```text\n{{{FromExp=\"{{{FromEnv=TEST_VAL}}}\" + \"hello\"}}}\n```\n```text\nCurrent time is: {PL} with token from TokenRef={{{RefName=TokenRef}}} and TokenObjectRef={{{RefName=TokenObjectRef token}}}\n```\n```text\nCurrent time is: {PL} with token from TokenRef={{{RefName=TokenRef}}} and TokenObjectRef={{{RefName=TokenObjectRef token}}}\n```\n```text\nTokenRef={{{RefName=TokenRef}}} and TokenObjectRef={{{RefName=TokenObjectRef token}}} Object={{{value}}} {PL} Env={{{FromEnv=TEST_VAL}}} {INDEX} {HUMAN_INDEX}\n```\n\n\n## References\nSpecial map which **prefetched**(before any processing) and can be user for [connector](#referenceconnectorconfig) or for [placeholder](#placeholder-list)\n\nCan be used for:\n1. Cache jwt token and use them in headers\n2. Cache values\n3. Etc\n\n### Reference\n\n```go\ntype Reference struct {\n    *ModelField\n    \n    Expire uint32 `yaml:\"expire\" json:\"expire\"`\n}\n```\n\n- [ModelField](#model-field) - is embedded struct, you can use same fields\n- Expire[sec] - duration when reference is expired after fetching. Not set =\u003e **forever cached**. Set to 0 =\u003e **every time re-fetch**. Set to n \u003e 0 =\u003e **cached for n second** \n\nFor [Fitter](#how-to-use-fitter)\n```go\ntype RefMap map[string]*Reference\n\ntype Config struct {\n    // Other Config Fields\n\n    Limits     *Limits `yaml:\"limits\" json:\"limits\"`\n    References RefMap  `json:\"references\" yaml:\"references\"`\n}\n```\n\nFor [Fitter Cli](#how-to-use-fittercli)\n\n```go\ntype RefMap map[string]*Reference\n\ntype CliItem struct {\n    // Other Config Fields\n\n    Limits     *Limits `yaml:\"limits\" json:\"limits\"`\n    References RefMap  `json:\"references\" yaml:\"references\"`\n}\n```\n\n\n- References - map[string]*Reference - object where is key if ReferenceName (can be user for [connector](#referenceconnectorconfig) or [placeholder](#placeholder-list)) and value is [Reference](#reference)\n- [Limits](#limits)\n\nExample\n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_ref.json#L2\n```json\n{\n  \"references\": {\n    \"TokenRef\": {\n      \"expire\": 10,\n      \"connector_config\": {\n        \"response_type\": \"json\",\n        \"static_config\": {\n          \"value\": \"\\\"plain token\\\"\"\n        }\n      },\n      \"model\": {\n        \"base_field\": {\n          \"type\": \"string\"\n        }\n      }\n    },\n    \"TokenObjectRef\": {\n      \"connector_config\": {\n        \"response_type\": \"json\",\n        \"static_config\": {\n          \"value\": \"{\\\"token\\\":\\\"token from object\\\"}\"\n        }\n      },\n      \"model\": {\n        \"object_config\": {\n          \"fields\": {\n            \"token\": {\n              \"base_field\": {\n                \"type\": \"string\",\n                \"path\": \"token\"\n              }\n            }\n          }\n        }\n      }\n    }\n  }\n}\n```\n\n[Example](https://github.com/PxyUp/fitter/blob/master/examples/cli/config_ref.json)\n\n## Limits\nProvide limitation for prevent DDOS, big usage of memory\n\n```go\ntype Limits struct {\n\tHostRequestLimiter HostRequestLimiter `yaml:\"host_request_limiter\" json:\"host_request_limiter\"`\n\tChromiumInstance   uint32             `yaml:\"chromium_instance\" json:\"chromium_instance\"`\n\tDockerContainers   uint32             `yaml:\"docker_containers\" json:\"docker_containers\"`\n\tPlaywrightInstance uint32             `yaml:\"playwright_instance\" json:\"playwright_instance\"`\n}\n```\n\n- HostRequestLimiter - map[string]int64 - limitation per host name, key is host, value is amount of parallel request(usage for [server connector](#serverconnectorconfig))\n- ChromiumInstance - amount of parallel [chromium](#chromium) instance\n- DockerContainers - amount of parallel [docker](#docker) instance\n- PlaywrightInstance - amount of parallel [playwright](#playwright) instance\n\nhttps://github.com/PxyUp/fitter/blob/master/examples/cli/config_cli.json#L2\n```json\n{\n  \"limits\": {\n    \"host_request_limiter\": {\n      \"hacker-news.firebaseio.com\": 5\n    },\n    \"chromium_instance\": 3,\n    \"docker_containers\": 3,\n    \"playwright_instance\": 3\n  }\n}\n```\n\n# Roadmap\n\n1. Add browser scenario for preparing, after parsing\n2. Add scrolling support for scenario\n3. Add pagination support for scenario\n4. Add notification methods for Fitter: Webhook/Queue\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpxyup%2Ffitter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpxyup%2Ffitter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpxyup%2Ffitter/lists"}