{"id":30224317,"url":"https://github.com/decaded/csv-chaos-tamer","last_synced_at":"2026-04-17T06:33:54.039Z","repository":{"id":309418627,"uuid":"1036121057","full_name":"Decaded/CSV-Chaos-Tamer","owner":"Decaded","description":"A Node.js script that processes multiple differently formatted CSV files into clean, normalized JSON, built specifically to prepare data for celestial.decaded.dev.","archived":false,"fork":false,"pushed_at":"2026-02-26T01:16:37.000Z","size":63449,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-02-26T06:32:05.614Z","etag":null,"topics":["celestial","csv","json","parser"],"latest_commit_sha":null,"homepage":"https://celestial.decaded.dev","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Decaded.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":"Decaded","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2025-08-11T15:32:05.000Z","updated_at":"2026-02-26T01:16:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"0fbcb982-5dd5-4633-86e1-bd530f33e13e","html_url":"https://github.com/Decaded/CSV-Chaos-Tamer","commit_stats":null,"previous_names":["decaded/csv-chaos-tamer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Decaded/CSV-Chaos-Tamer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Decaded%2FCSV-Chaos-Tamer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Decaded%2FCSV-Chaos-Tamer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Decaded%2FCSV-Chaos-Tamer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Decaded%2FCSV-Chaos-Tamer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Decaded","download_url":"https://codeload.github.com/Decaded/CSV-Chaos-Tamer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Decaded%2FCSV-Chaos-Tamer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31918651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-16T18:22:33.417Z","status":"online","status_checked_at":"2026-04-17T02:00:06.879Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["celestial","csv","json","parser"],"created_at":"2025-08-14T13:32:50.487Z","updated_at":"2026-04-17T06:33:54.031Z","avatar_url":"https://github.com/Decaded.png","language":"JavaScript","funding_links":["https://ko-fi.com/Decaded","https://ko-fi.com/decaded"],"categories":[],"sub_categories":[],"readme":"# CSV Chaos Tamer\n\nA Node.js script that processes multiple differently formatted CSV files into clean, normalized JSON. It’s built specifically to prepare data for\n[celestial.decaded.dev](https://celestial.decaded.dev), but it can work with any similar dataset structure.\n\nThis parser handles inconsistent headers, missing fields, and various formatting quirks so the data is ready to be consumed by the main project without manual cleanup.\n\n---\n\n## Why this exists\n\n[celestial.decaded.dev](https://celestial.decaded.dev) is a database-driven project that relies on a large number of community-sourced datasets. The problem is that these datasets\nare often messy:\n\n- Column names vary wildly between files.\n- Some datasets bury important metadata in filenames instead of proper columns.\n- Formatting can change mid-series due to different contributors.\n\nThe CSV Chaos Tamer standardizes this chaos into a consistent JSON format that celestial can read without breaking. It’s not just a parser -- it’s the gatekeeper that ensures\nimported data actually makes sense.\n\n---\n\n## What it does\n\n- Handles multiple CSV formats without requiring a separate config for each.\n- Normalizes headers so variations like `Price`, `cost`, and `CPCost` are unified under `cost`.\n- Detects chapter information from either a column or the filename.\n- Splits out specific chapters into their own JSON files if configured.\n- Cleans text by trimming whitespace, fixing newlines, and removing stray characters.\n- Can be easily extended to support new header mappings or split rules.\n\n---\n\n## How to set it up\n\n1. Place CSV files into subfolders inside `sheets/`:\n\n```bash\nsheets/\n└── DatasetName/\n    ├── file1.csv\n    └── file2.csv\n```\n\n2. Install dependencies:\n\n```bash\nnpm install\n```\n\n3. Run the parser:\n\n```bash\nnode index.js\n```\n\n4. Collect your cleaned JSON from `data/`.\n\n---\n\n## Customizing\n\nIf your CSVs use unique or inconsistent headers, edit the `headerMap`:\n\n```js\ncpcost: 'cost',\nprice: 'cost',\nperkname: 'name',\nsetting: 'source',\n```\n\nIf you need to split specific chapters into separate files, modify `SPLIT_CHAPTERS`:\n\n```js\n'waifu catalogue': 'waifu',\n'lewd': 'companion_(lewd)',\n```\n\n---\n\n## Output format\n\nThe script generates one JSON file per input folder, plus any additional split chapter files.\n\nExample output:\n\n```json\n{\n\t\"1\": [\n\t\t{\n\t\t\t\"id\": 1,\n\t\t\t\"cost\": 200,\n\t\t\t\"name\": \"Example Perk\",\n\t\t\t\"source\": \"Example Jump\",\n\t\t\t\"chapter\": \"Example Chapter\",\n\t\t\t\"description\": \"Cleaned description here.\",\n\t\t\t\"__source\": \"file1\",\n\t\t\t\"__line\": 2\n\t\t}\n\t]\n}\n```\n\n---\n\n## Requirements\n\n- Node.js 16 or newer\n- CSV files encoded in UTF-8\n\n---\n\n## License\n\n[MIT License](LICENSE) – free to use, modify, and distribute.\n\n---\n\n## Support the project\n\nIf this tool or [celestial.decaded.dev](https://celestial.decaded.dev) has been useful to you, consider supporting development:\n[https://ko-fi.com/decaded](https://ko-fi.com/decaded)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdecaded%2Fcsv-chaos-tamer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdecaded%2Fcsv-chaos-tamer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdecaded%2Fcsv-chaos-tamer/lists"}