{"id":27768013,"url":"https://github.com/koddr/csv2api","last_synced_at":"2025-10-27T11:08:43.092Z","repository":{"id":175814006,"uuid":"654476649","full_name":"koddr/csv2api","owner":"koddr","description":"🏄‍♂️ The parser reads the CSV file with the raw data, filters the records, identifies fields to be changed, and sends a request to update the data to the specified endpoint of your REST API.","archived":false,"fork":false,"pushed_at":"2025-03-14T11:35:46.000Z","size":102,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-29T19:58:31.671Z","etag":null,"topics":["api","api-rest","csv","csv-api","csv-parser","go","go-app","golang","golang-application","parser","parsing"],"latest_commit_sha":null,"homepage":"https://github.com/koddr/csv2api","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/koddr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"liberapay":"koddr"}},"created_at":"2023-06-16T08:02:37.000Z","updated_at":"2025-03-14T11:35:43.000Z","dependencies_parsed_at":"2024-02-03T23:32:52.134Z","dependency_job_id":"273b35c6-ab51-457c-a4fc-11fab68e240b","html_url":"https://github.com/koddr/csv2api","commit_stats":null,"previous_names":["koddr/csv2api"],"tags_count":1,"template":false,"template_full_name":"koddr/template-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koddr%2Fcsv2api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koddr%2Fcsv2api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koddr%2Fcsv2api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koddr%2Fcsv2api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/koddr","download_url":"https://codeload.github.com/koddr/csv2api/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251574671,"owners_count":21611387,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","api-rest","csv","csv-api","csv-parser","go","go-app","golang","golang-application","parser","parsing"],"created_at":"2025-04-29T19:58:40.319Z","updated_at":"2025-10-27T11:08:43.020Z","avatar_url":"https://github.com/koddr.png","language":"Go","funding_links":["https://liberapay.com/koddr"],"categories":[],"sub_categories":[],"readme":"# csv2api – Parse CSV with filtering and sending to API\n\n[![Go version][go_version_img]][go_dev_url]\n[![Go report][go_report_img]][go_report_url]\n[![Wiki][wiki_img]][wiki_url]\n[![License][license_img]][license_url]\n\nThe **csv2api** parser reads the CSV file with the raw data, filters the \nrecords, identifies fields to be changed, and sends a request to update the \ndata to the specified endpoint of your REST API. \n\nAll actions take place according to the settings in the configuration file.\n\nFeatures:\n\n- 100% **free** and **open source**.\n- Works with any **size** of input CSV file.\n- Use **any** configuration file format: JSON, YAML, TOML, or HCL (Terraform).\n- Ability to keep a configuration file (_and, in the future, input data \n  file_) on a **remote server** with HTTP access, it will be read as if it \n  was in a folder on your local machine. \n- Configure **any** request body for the REST API endpoint directly in the \n  configuration file (in a clear declarative style).\n- Extensive options for **filtering** incoming data from a CSV file.\n- Provides extensive capabilities for constructing **multiple filters** to \n  accurately perform actions on selected fields.\n\n## ⚡️ Quick start\n\nFirst, [download][go_download] and install **Go**. Version `1.20` (or higher)\nis required.\n\nInstallation is done by using the [`go install`][go_install] command:\n\n```console\ngo install github.com/koddr/csv2api@latest\n```\n\n\u003e 💡 Note: See the repository's [Release page][repo_releases_url], if you want\n\u003e to download a ready-made `deb`, `rpm`, `apk` or `Arch Linux` package.\n\nGNU/Linux and macOS users available way to install via [Homebrew][brew_url]:\n\n```console\n# Tap a new formula:\nbrew tap koddr/tap\n\n# Installation:\nbrew install koddr/tap/csv2api\n```\n\nNext, run `csv2api` with `-i` option to generate initial `config.yml` and \n`data.csv` files in the current dir:\n\n```console\ncsv2api -i\n```\n\nPrepare your config and input data files:\n\n- In your `config.yml`:\n  - Make sure that the first column name in the `columns_order` section is a \n    primary key (PK) for your process.\n  - Set up your API (base URL, token, endpoints, etc) in the `api` section.\n  - Set up the filter for your fields in the `filter_columns` section.\n  - Set up fields to be updated in the `update_fields` section.\n- In your input `data.csv`:\n  - Make sure that the first line of your CSV file contains the correct field names.\n\n\u003e 💡 Note: See the repository's [Wiki][wiki_url] page to understand the\n\u003e structure of the config and input data files.\n\nAnd now, run `csv2api` with options:\n\n```console\ncsv2api \\\n  -c /path/to/config.yml \\\n  -d /path/to/data.csv \\\n  -e CONFIG\n```\n\nDone! 🎉 Your transactions have been performed:\n\n``` console\nHello and welcome to csv2api! 👋\n                                \n– According to the settings in './config.yml', 5 transactions were filtered out of 10 to start the process.\n– Only 3 transactions got into the final set of actions to be taken... Please wait!\n                                                                                                                                \n ✓ Field 'tags' with values '[paid]' in the transaction '2' has been successfully updated (HTTP 200)\n ✓ Field 'tags' with values '[paid]' in the transaction '8' has been successfully updated (HTTP 200)\n ✓ Field 'tags' with values '[paid]' in the transaction '10' has been successfully updated (HTTP 200)\n                                                                                                \n– Saving filtered transactions to CSV file './filtered-1686993960.csv' in the current dir... OK!\n                                \nAll done! 🎉 Time elapsed: 0.11s\n```\n\n### 🐳 Docker-way to quick start\n\nIf you don't want to physically install `csv2api` to your system, you feel\nfree to using our [official Docker image][docker_image_url] and run it from\nisolated container:\n\n```console\ndocker run --rm -it -v ${PWD}:${PWD} -w ${PWD} koddr/csv2api:latest [OPTIONS]\n```\n\n## 🧩 Options\n\n| Option | Description                                                                        | Is required? | Default value |\n|--------|------------------------------------------------------------------------------------|--------------|---------------|\n| `-i`   | set to generate initial config (`config.yaml`) and example data (`data.csv`) files | no           | `false`       |\n| `-c`   | set path to your config file                                                       | yes          | `\"\"`          |\n| `-d`   | set path to your CSV file with input data                                          | yes          | `\"\"`          |\n| `-e`   | set prefix used in your environment variables                                      | no           | `CONFIG`      |\n\n## ✨ Solving case\n\nIn my work, I often have to work with **large amounts** of raw data in CSV format. \n\nUsually it goes like this:\n\n1. Unload a file with data from one system.\n2. Clean up this file from duplicates and unnecessary columns.\n3. Make some changes in some columns of some rows.\n4. Mapping the processed lines from CSV file to the database structure fields.\n5. Write a function to bypass the CSV file and form the request body.\n6. Write an HTTP client that will send requests to the REST API endpoint.\n7. Send prepared request body to the REST API endpoint in other system \n   for specified DB records.\n\n\u003e And I'm not talking about the fact that the final REST API (where to send a \nrequest with the processed data) **do not** always have the same parameters for \nthe request body.\n\nTo ease this whole process, I created this parser that takes absolutely any \ndata file as input, does the conversions and filtering, and is set up in one \nsingle configuration file. \n\nJust prepare the data, set the configuration to your liking, run `csv2api` \nand wait a bit! Yes, it's that simple.\n\n## 🏆 A win-win cooperation\n\nAnd now, I invite you to participate in this project! Let's work **together** to\ncreate the **most useful** tool for developers on the web today.\n\n- [Issues][repo_issues_url]: ask questions and submit your features.\n- [Pull requests][repo_pull_request_url]: send your improvements to the current.\n\nYour PRs \u0026 issues are welcome! Thank you 😘\n\n## ⚠️ License\n\n[`csv2api`][repo_url] is free and open-source software licensed \nunder the [Apache 2.0 License][license_url], created and supported with 🩵 \nfor people and robots by [Vic Shóstak][author].\n\n[go_download]: https://golang.org/dl/\n[go_install]: https://golang.org/cmd/go/#hdr-Compile_and_install_packages_and_dependencies\n[go_version_img]: https://img.shields.io/badge/Go-1.20+-00ADD8?style=for-the-badge\u0026logo=go\n[go_report_img]: https://img.shields.io/badge/Go_report-A+-success?style=for-the-badge\u0026logo=none\n[go_report_url]: https://goreportcard.com/report/github.com/koddr/csv2api\n[go_code_coverage_img]: https://img.shields.io/badge/code_coverage-0%25-success?style=for-the-badge\u0026logo=none\n[go_dev_url]: https://pkg.go.dev/github.com/koddr/csv2api\n[docker_image_url]: https://hub.docker.com/repository/docker/koddr/csv2api\n[brew_url]: https://brew.sh\n[wiki_img]: https://img.shields.io/badge/docs-wiki_page-blue?style=for-the-badge\u0026logo=none\n[wiki_url]: https://github.com/koddr/csv2api/wiki\n[license_img]: https://img.shields.io/badge/license-Apache_2.0-red?style=for-the-badge\u0026logo=none\n[license_url]: https://github.com/koddr/csv2api/blob/main/LICENSE\n[repo_url]: https://github.com/koddr/csv2api\n[repo_releases_url]: https://github.com/koddr/csv2api/releases\n[repo_issues_url]: https://github.com/koddr/csv2api/issues\n[repo_pull_request_url]: https://github.com/koddr/csv2api/pulls\n[author]: https://github.com/koddr\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoddr%2Fcsv2api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkoddr%2Fcsv2api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoddr%2Fcsv2api/lists"}