{"id":16389378,"url":"https://github.com/marcelog/egetter","last_synced_at":"2025-02-22T16:05:56.936Z","repository":{"id":16222716,"uuid":"18969997","full_name":"marcelog/egetter","owner":"marcelog","description":"Wrapper around ibrowse, simple api to use random proxies and user agents and easily create clients for HTTP apis","archived":false,"fork":false,"pushed_at":"2014-07-05T19:28:48.000Z","size":792,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-03T10:28:50.498Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcelog.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-04-20T17:44:30.000Z","updated_at":"2014-08-18T07:53:12.000Z","dependencies_parsed_at":"2022-09-24T11:41:02.385Z","dependency_job_id":null,"html_url":"https://github.com/marcelog/egetter","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelog%2Fegetter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelog%2Fegetter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelog%2Fegetter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelog%2Fegetter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcelog","download_url":"https://codeload.github.com/marcelog/egetter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240199180,"owners_count":19763820,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T04:32:42.769Z","updated_at":"2025-02-22T16:05:56.918Z","avatar_url":"https://github.com/marcelog.png","language":"Erlang","readme":"## About\nA simple erlang application to manage http requests. Will optionally choose a proxy and a\nuser agent at random from the given lists. It's also very useful to create clients\nfor http apis, such as ElasticSearch.\n\n## How to use it\n\n * Start the main supervisor [egetter_sup](https://github.com/marcelog/egetter/blob/master/src/egetter_sup.erl) from your own supervisor\ntree.\n\n * Configure the application, here's a sample:\n```\n{egetter, [\n  {user_agents, \"/etc/egetter/priv/agents.txt\"},\n  {proxies, \"/etc/egetter/proxies.txt\"}\n]}\n```\n\n * Both are **optional**. Egetter will **not try** to use a proxy or choose a user-agent\n if the corresponding option is not set.\n\n * You will need a list of proxies and a list of user agents. There are examples\n in the [priv directory](https://github.com/marcelog/egetter/tree/master/priv)\n \n * Call **egetter:req/1** passing in a list of options. You will get a **result()**, which is one of:\n   * {ibrowse_error, term()}: on \"hard\" errors returned by ibrowse.\n   * {error, Status::pos_integer(), Headers::[{string(), string()}], Body::binary()}: When the request fails (i.e: status code not 2xx/3xx).\n   * {ok, Status::pos_integer(), Headers::[{string(), string()}], Body::binary()}: When the result is 2xx or 3xx.\n\n * Available options:\n   * {url, string()} Either specify this option **OR all of**:\n     * {host, string()}\n     * {port, pos_integer()},\n     * {scheme, string()} e.g: \"http\"\n     * {path_components, [string()]}\n   * {timeout, pos_integer()}\n   * {headers, [{string()|atom(), string()}]}\n   * {body, binary()}\n   * {method, get | post | put | delete | head | options}\n   * {follow_redirect, true | false} (defaults to false)\n   * {ibrowse_options, [{atom(), term()}]}\n   * {use_proxy, true | false} (defaults to false)\n   * {query_string, query_string()} e.g: [{\"key1\", \"value1\"}], keys and values are automatically urlencoded.\n\n## Examples\n\n```erlang\negetter:req([{url, \"http://www.google.com/\"}, {follow_redirect, true}, {query_string, [{\"key\", \" value\"}]}]).\n```\n\n```erlang\negetter:req([{host, \"google.com\"}, {port, 80}, {scheme, \"http\"}, {path_components, [\"some\", \"path\"]}]}]).\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcelog%2Fegetter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcelog%2Fegetter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcelog%2Fegetter/lists"}