{"id":15665956,"url":"https://github.com/upsuper/csv-transformer","last_synced_at":"2025-03-03T06:21:38.183Z","repository":{"id":66196201,"uuid":"301276411","full_name":"upsuper/csv-transformer","owner":"upsuper","description":"Command line tool to rearrange CSV files in certain ways","archived":false,"fork":false,"pushed_at":"2020-10-05T02:50:59.000Z","size":15,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-12T22:36:17.467Z","etag":null,"topics":["csv","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/upsuper.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-05T02:44:24.000Z","updated_at":"2023-06-21T18:48:53.000Z","dependencies_parsed_at":"2023-03-18T03:45:33.472Z","dependency_job_id":null,"html_url":"https://github.com/upsuper/csv-transformer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upsuper%2Fcsv-transformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upsuper%2Fcsv-transformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upsuper%2Fcsv-transformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upsuper%2Fcsv-transformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/upsuper","download_url":"https://codeload.github.com/upsuper/csv-transformer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241617573,"owners_count":19991707,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","rust"],"created_at":"2024-10-03T13:58:06.399Z","updated_at":"2025-03-03T06:21:38.160Z","avatar_url":"https://github.com/upsuper.png","language":"Rust","readme":"# csv-transformer\n\nCommand line tool to rearrange CSV files in certain ways.\n\nThis was developed to help transforming\nSimplified Chinese survey result of [2020 State of Rust survey][survey] to match the global version.\n\n[survey]: https://blog.rust-lang.org/2020/09/10/survey-launch.html\n\n## Usage\n\nFirstly, with a CSV file, use the following command to extract all the columns:\n```bash\ncsv-transformer extract original.csv \u003e transform.yaml\n```\n\nThis will transform a CSV file like\n```csv\nQuestion 1,Question 2,Question 3\nAnswer B,Answer X,\nAnswer A,Answer X,Random text\nAnswer A,Answer Y,\n```\ninto\n```yaml\n- \"A: Question 1\"\n- \"B: Question 2\"\n- \"C: Question 3\"\n```\n\nThe strings formatted `X: Header text` is called a column reference,\nand the letters before the first colon is the column index.\n\nYou can then edit the YAML file to reflect the transformation you want.\nPlease refer to the [transformations](#Transformations) section for available transformations.\nThis is an example of transformation file we used for the survey:\n[transform.yaml](https://gist.github.com/upsuper/3e90f78d84b84c9741d585a1d462b1b5).\n\nAfter you edit it, you can use the following command to generate the result:\n```bash\ncsv-transformer transform original.csv transform.yaml \u003e result.csv\n```\n\n### Transformations\n\nEach item in the YAML file represents a rule\nto generate one or more columns in the transformation result in its order.\nIf it's kept untouched (just a column reference),\nthe column would be preserved as is.\nOtherwise, it can be one of the following transformations.\n\n#### Rename\n\nA rename transformation basically just changes the header text.\n\nExample:\n```yaml\n- transform: rename\n  header: \"New Header\"\n  column: \"A: Old Header\"\n```\ntransforms\n\n| Old Header |\n| ---------- |\n| Value 1    |\n| Value 2    |\n\nto\n\n| New Header |\n| ---------- |\n| Value 1    |\n| Value 2    |\n\n#### Timestamp\n\nA timestamp transformation reformats the date and time value\nwith the format of [`strftime`](https://docs.rs/chrono/0.4.19/chrono/format/strftime/index.html).\n\nExample:\n```yaml\n- transform: timestamp\n  column: \"A: Timestamp\"\n  from: \"%d-%b-%Y %H:%M:%S\"\n  to: \"%d/%m/%Y %H:%M:%S\"\n```\ntransforms\n\n| Timestamp            |\n| -------------------- |\n| 26-Sep-2020 01:12:42 |\n| 25-Sep-2020 23:23:52 |\n\nto\n\n| Timestamp           |\n| ------------------- |\n| 26/09/2020 01:12:42 |\n| 25/09/2020 23:23:52 |\n\nOptionally, you can also provide a `header` field to rename the column at the same time.\n\n#### Join\n\nA join transformation concatenates values from multiple columns into a single column.\n\nExample:\n```yaml\n- transform: join\n  header: \"Question?\"\n  columns:\n  - \"A: Question? Rust\"\n  - \"B: Question? C++\"\n  - \"C: Question? Python\"\n```\ntransforms\n\n| Question? Rust | Question? C++ | Question? Python |\n| -------------- | ------------- | ---------------- |\n| Rust           |               | Python           |\n|                | C++           |                  |\n| Rust           | C++           | Python           |\n\nto\n\n| Question?         |\n| ----------------- |\n| Rust, Python      |\n| C++               |\n| Rust, C++, Python |\n\nOptionally, you can provide a `sep` field to change the default separator `, ` to something else.\n\nIt's also possible to slightly format the values from columns before joining\nvia replacing the column reference item with a object.\n\nExample:\n```yaml\n- transform: join\n  header: \"Conference anywhere?\"\n  columns:\n  - column: \"A: Conference in China?\"\n    format: \"China - {}\"\n  - column: \"B: Conference outside China?\"\n    format: \"outside China - {}\"\n```\ntransforms\n\n| Conference in China? | Conference outside China? |\n| -------------------- | ------------------------- |\n| Yes                  | No                        |\n| Maybe                |                           |\n|                      | No                        |\n\nto\n\n| Conference anywhere?            |\n| ------------------------------- |\n| China - Yes, outside China - No |\n| China - Maybe                   |\n| outside China - No              |\n\n#### Transpose\n\nA transpose transformation transposes values and their header across several columns.\n\nExample:\n```yaml\n- transform: transpose\n  sources:\n    \"A: Question? 1st\": 1st\n    \"B: Question? 2nd\": 2nd\n    \"C: Question? 3rd\": 3rd\n  columns:\n    \"Question? Go\": Go\n    \"Question? C++\": C++\n    \"Question? Rust\": Rust\n    \"Question? Python\": Python\n```\ntransforms\n\n| Question? 1st | Question? 2nd | Question? 3rd |\n| ------------- | ------------- | ------------- |\n| Rust          | C++           | Go            |\n| Python        | Go            | Rust          |\n\nto\n\n| Question? Go | Question? C++ | Question? Rust | Question? Python |\n| ------------ | ------------- | -------------- | ---------------- |\n| 3rd          | 2nd           | 1st            |                  |\n| 2nd          |               | 3rd            | 1st              |\n\nIf a value is present in multiple source columns,\nthe first matching one would be picked.\n\nAn error would be raised if a non-empty value in the source columns can't be mapped to a target column.\n\n## Development\n\nAll transformations are in `src/transform` directory, and new transformations can be added there.\n\n## License\n\nCopyright (C) 2020 Xidorn Quan\n\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see \u003chttps://www.gnu.org/licenses/\u003e.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fupsuper%2Fcsv-transformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fupsuper%2Fcsv-transformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fupsuper%2Fcsv-transformer/lists"}