{"id":14068713,"url":"https://github.com/vh-d/RETL","last_synced_at":"2025-07-30T04:31:56.177Z","repository":{"id":82718163,"uuid":"167702187","full_name":"vh-d/RETL","owner":"vh-d","description":"R package for ETL ","archived":false,"fork":false,"pushed_at":"2020-06-27T15:36:26.000Z","size":42,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-08-13T07:15:13.406Z","etag":null,"topics":["etl","etl-framework","transformations"],"latest_commit_sha":null,"homepage":null,"language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vh-d.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-01-26T15:11:23.000Z","updated_at":"2020-06-27T15:36:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"63271296-af1d-46b6-b056-726a63b6a014","html_url":"https://github.com/vh-d/RETL","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vh-d%2FRETL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vh-d%2FRETL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vh-d%2FRETL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vh-d%2FRETL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vh-d","download_url":"https://codeload.github.com/vh-d/RETL/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228082362,"owners_count":17866607,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["etl","etl-framework","transformations"],"created_at":"2024-08-13T07:06:21.724Z","updated_at":"2024-12-04T09:31:16.454Z","avatar_url":"https://github.com/vh-d.png","language":"R","funding_links":[],"categories":["R"],"sub_categories":[],"readme":"---\noutput: github_document\n---\n\n\u003c!-- README.md is generated from README.Rmd. Please edit that file --\u003e\n\n```{r setup, include = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  fig.path = \"man/figures/README-\",\n  out.width = \"100%\"\n)\n```\n\n# RETL\n\n`RETL` is an R package that provides tools for writing ETL jobs in R. It stands on R's wide range of APIs to various types of data sources.\n\n\nIt is intended to be used together with the *[Rflow](https://github.com/vh-d/Rflow)* and *[RETLflow](https://github.com/vh-d/RETLflow)* packages as universal API to data stored in databases, files, excel sheets. RETL relies heavily on the `data.table` package for fast data transofrmations.\n\n\n## Installation\n\nRETL can be installed from [GitHub](https://github.com/vh-d/RETL) by running:\n\n``` r\ndevtools::install_github(\"vh-d/RETL\")\n```\n\n\n## Examples\n\n``` r\nlibrary(RETL)\nlibrary(magrittr)\n\n# establish connections\nmy_db    \u003c- DBI::dbConnect(RSQLite::SQLite(), \"path/to/my.db\")\nyour_csv \u003c- \"path/to/your.csv\"\nyour_db  \u003c- dbConnect(RMariaDB::MariaDB(), group = \"your-db\")\n```\n\n\n### Pipes\n\n``` r\n# simple extract and load\netl_read(from = your_csv) %\u003e% etl_write(to = my_db, name = \"customers\")\n\n# extract -\u003e transform -\u003e load\netl_read(from = my_db, name = \"orders\") %\u003e% # db query: EXTRACT from a database\n  dtq(, order_year := year(order_date)) %\u003e% # data.table query: TRANSFORM (adding a new column)\n  etl_write(to = your_db, name = \"customers\") # LOAD to a db\n```\n\n### Other tools\n\n```r\nset_index(table = \"customers\", c(\"id\", \"order_year\"), your_db)\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvh-d%2FRETL","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvh-d%2FRETL","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvh-d%2FRETL/lists"}