{"id":14067521,"url":"https://github.com/hrbrmstr/curlconverter","last_synced_at":"2025-08-13T03:20:03.604Z","repository":{"id":47806052,"uuid":"50591210","full_name":"hrbrmstr/curlconverter","owner":"hrbrmstr","description":":curly_loop: :arrow_right: :heavy_minus_sign: Translate cURL command lines into parameters for use with httr or actual httr calls (R)","archived":false,"fork":false,"pushed_at":"2021-08-12T22:25:04.000Z","size":488,"stargazers_count":90,"open_issues_count":13,"forks_count":12,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-02T00:41:32.493Z","etag":null,"topics":["curl","httr","r","r-cyber","rstats"],"latest_commit_sha":null,"homepage":"http://rud.is/b/2016/02/10/craft-httr-calls-cleverly-with-curlconverter/","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hrbrmstr.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-01-28T15:11:58.000Z","updated_at":"2025-03-22T11:20:50.000Z","dependencies_parsed_at":"2022-08-24T04:31:18.102Z","dependency_job_id":null,"html_url":"https://github.com/hrbrmstr/curlconverter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hrbrmstr/curlconverter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hrbrmstr%2Fcurlconverter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hrbrmstr%2Fcurlconverter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hrbrmstr%2Fcurlconverter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hrbrmstr%2Fcurlconverter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hrbrmstr","download_url":"https://codeload.github.com/hrbrmstr/curlconverter/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hrbrmstr%2Fcurlconverter/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270173854,"owners_count":24539656,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-13T02:00:09.904Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["curl","httr","r","r-cyber","rstats"],"created_at":"2024-08-13T07:05:38.481Z","updated_at":"2025-08-13T03:20:03.571Z","avatar_url":"https://github.com/hrbrmstr.png","language":"R","funding_links":[],"categories":["R"],"sub_categories":[],"readme":"---\noutput: rmarkdown::github_document\neditor_options: \n  chunk_output_type: console\n---\n\n[![Build Status](https://travis-ci.org/hrbrmstr/curlconverter.svg?branch=master)](https://travis-ci.org/hrbrmstr/curlconverter)\n[![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/hrbrmstrcurlconverter?branch=master\u0026svg=true)](https://ci.appveyor.com/project/hrbrmstr/curlconverter)\n[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/curlconverter)](https://cran.r-project.org/package=curlconverter)\n[![](http://cranlogs.r-pkg.org/badges/curlconverter)](https://cran.r-project.org/package=curlconverter)\n[![Coverage Status](https://img.shields.io/codecov/c/github/\u003cUSERNAME\u003e/\u003cREPO\u003e/master.svg)](https://codecov.io/github/\u003cUSERNAME\u003e/\u003cREPO\u003e?branch=master)\n\n# curlconverter\n\nTools to Transform 'cURL' Command-Line Calls to 'httr' Requests\n\n## Description\n\nDeciphering web/'REST' 'API' and 'XHR' calls can be tricky, which is one reason why internet browsers provide \"Copy as cURL\" functionality within their \"Developer Tools\"pane(s). These 'cURL' command-lines can be difficult to wrangle into an 'httr' 'GET' or 'POST' request, but you can now \"straighten\" these 'cURLs' either from data copied to the system clipboard or by passing in a vector of 'cURL' command-lines and getting back a list of parameter elements which can be used to form 'httr' requests. These lists can be passed to another function to automagically make 'httr' functions.\n\n### WIP\n\nThis is the path back to CRAN and to 1.0.0.\n\nThe V8 dependency has been removed and the package now uses `docopt`. This will make it easier to install on many systems and enable rapid addition of support for new `cURL` command-line parameters.\n\nThe \"HAR\" functions are not working well with the new methods but will be for full release.\n\n## What's Inside The Tin\n\nThe following functions are implemented:\n\n- `straighten`: convert one or more _\"Copy as cURL\"_ command lines into useful data\n- `parse_query`: parse URL query parameters into a named list\n- `make_req`: turn parsed cURL command lines into a `httr` request functions (i.e. returns working R functions)\n\n\nThe following functions are implemented:\n\n## Installation\n\n```{r eval=FALSE}\ndevtools::install_github(\"hrbrmstr/curlconverter\")\n```\n\n```{r message=FALSE, warning=FALSE, error=FALSE, include=FALSE}\noptions(width=120)\n```\n\n## Usage\n\n```{r message=FALSE, warning=FALSE, error=FALSE}\nlibrary(curlconverter)\nlibrary(jsonlite)\nlibrary(httr)\n\n# current verison\npackageVersion(\"curlconverter\")\n```\n\nSimple example using a call to \u003chttps://httpbin.org/headers\u003e:\n\n```{r}\nhttpbinrhcurl \u003c- \"curl 'https://httpbin.org/headers' -H 'pragma: no-cache' -H 'accept-encoding: gzip, deflate, sdch' -H 'accept-language: en-US,en;q=0.8' -H 'upgrade-insecure-requests: 1' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.39 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'cache-control: no-cache' -H 'referer: https://httpbin.org/' --compressed\"\n\nstraight \u003c- straighten(httpbinrhcurl)\nres \u003c- make_req(straight)\n\n# or \n\nstraighten(httpbinrhcurl) %\u003e% \n  make_req() -\u003e res\n\ntoJSON(content(res[[1]](), as=\"parsed\"), auto_unbox = TRUE, pretty=TRUE)\n```\n\nSlightly more complex one:\n\n```{r}\ntoJSON(straighten(\"curl 'http://financials.morningstar.com/ajax/ReportProcess4HtmlAjax.html?\u0026t=XNAS:MSFT\u0026region=usa\u0026culture=en-US\u0026cur=\u0026reportType=is\u0026period=12\u0026dataType=A\u0026order=asc\u0026columnYear=5\u0026curYearPart=1st5year\u0026rounding=3\u0026view=raw\u0026r=973302\u0026callback=jsonp1454021128757\u0026_=1454021129337' -H 'Cookie: JSESSIONID=5E43C98903E865D72AA3C2DCEF317848; sfhabit=asc%7Craw%7C3%7C12%7CA%7C5%7Cv0.14; ScrollY=0' -H 'DNT: 1' -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36' -H 'Accept: text/javascript, application/javascript, */*' -H 'Referer: http://financials.morningstar.com/income-statement/is.html?t=MSFT\u0026region=usa\u0026culture=en-US' -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'Cache-Control: max-age=0' --compressed\"), auto_unbox = TRUE, pretty=TRUE)\n```\n\nThere are some built-in test files you can play with:\n\n```{r}\n(curl_line \u003c- readLines(system.file(\"extdata/curl5.txt\", package=\"curlconverter\"), warn=FALSE))\n\ntoJSON(straighten(curl_line, quiet=TRUE), auto_unbox = TRUE, pretty=TRUE)\n(curl_line \u003c- readLines(system.file(\"extdata/curl8.txt\", package=\"curlconverter\"), warn=FALSE))\n\n# example with query parameters in the body\nreq \u003c- straighten(curl_line, quiet=FALSE)\n\n# ugh\n(req[[1]]$data)\n\n#yay!\ntoJSON(parse_query(req[[1]]$data), auto_unbox = TRUE, pretty=TRUE)\n```\n\nSpinning straw into gold\n\n```{r}\ncurl_line \u003c- c('curl \"http://anasim.iet.unipi.it/moniqa/php/from_js.php\" -H \"Origin: http://anasim.iet.unipi.it\" -H \"Accept-Encoding: gzip, deflate\" -H \"Accept-Language: it-IT,it;q=0.8,en-US;q=0.6,en;q=0.4\" -H \"User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36\" -H \"Content-Type: application/x-www-form-urlencoded; charset=UTF-8\" -H \"Accept: */*\" -H \"Referer: http://anasim.iet.unipi.it/moniqa/\" -H \"X-Requested-With: XMLHttpRequest\" -H \"Connection: keep-alive\" --data \"deviceid=65\u0026function_name=extract_measurements\" --compressed')\n\nstraighten(curl_line) %\u003e% \n  make_req() -\u003e get_data\n\ncat(capture.output(get_data[[1]]), sep=\"\\n\")\n```\n\nThat also sends this to the console and clipboard:\n\n```{r eval=FALSE}\nhttr::VERB(\n  verb = \"GET\", url = \"http://anasim.iet.unipi.it/moniqa/php/from_js.php\",\n  httr::add_headers(\n    Origin = \"http://anasim.iet.unipi.it\",\n    `Accept-Encoding` = \"gzip, deflate\",\n    `Accept-Language` = \"it-IT,it;q=0.8,en-US;q=0.6,en;q=0.4\",\n    `User-Agent` = \"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36\",\n    Accept = \"*/*\", Referer = \"http://anasim.iet.unipi.it/moniqa/\",\n    `X-Requested-With` = \"XMLHttpRequest\",\n    Connection = \"keep-alive\"\n  ),\n  body = list(\n    deviceid = \"65\",\n    function_name = \"extract_measurements\"\n  ),\n  encode = \"form\"\n)\n```\n\n### Code of Conduct\n\nPlease note that this project is released with a [Contributor Code of Conduct](CONDUCT.md). \n\nBy participating in this project you agree to abide by its terms.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhrbrmstr%2Fcurlconverter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhrbrmstr%2Fcurlconverter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhrbrmstr%2Fcurlconverter/lists"}