{"id":13700415,"url":"https://github.com/cnuernber/charred","last_synced_at":"2025-05-16T16:01:33.096Z","repository":{"id":39667110,"uuid":"480119933","full_name":"cnuernber/charred","owner":"cnuernber","description":"zero dependency efficient read/write of json and csv data.","archived":false,"fork":false,"pushed_at":"2025-04-11T17:36:12.000Z","size":234,"stargazers_count":279,"open_issues_count":3,"forks_count":12,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-12T14:56:01.497Z","etag":null,"topics":["clojure","csv","java","json"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cnuernber.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"cnuernber"}},"created_at":"2022-04-10T19:32:57.000Z","updated_at":"2025-04-12T07:01:46.000Z","dependencies_parsed_at":"2024-10-26T21:11:11.502Z","dependency_job_id":"1228e713-d339-446c-952c-f030598fa610","html_url":"https://github.com/cnuernber/charred","commit_stats":{"total_commits":91,"total_committers":8,"mean_commits":11.375,"dds":"0.10989010989010994","last_synced_commit":"600eed9e08b88d05017ef59c46a74218f87322a5"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cnuernber%2Fcharred","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cnuernber%2Fcharred/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cnuernber%2Fcharred/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cnuernber%2Fcharred/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cnuernber","download_url":"https://codeload.github.com/cnuernber/charred/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254564108,"owners_count":22092119,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","csv","java","json"],"created_at":"2024-08-02T20:00:55.877Z","updated_at":"2025-05-16T16:01:32.662Z","avatar_url":"https://github.com/cnuernber.png","language":"Clojure","readme":"# Charred\n\nEfficient character-based file parsing for csv and json formats.\n\n\n[![Clojars Project](https://clojars.org/com.cnuernber/charred/latest-version.svg)](https://clojars.org/com.cnuernber/charred)\n\n* Zero dependencies.\n* As fast as univocity or jackson.\n* Same API as clojure.data.csv and clojure.data.json implemented far more\n  efficiently.\n\n\n* [API Documentation](https://cnuernber.github.io/charred/)\n* [Simple JSON benchmarks](https://github.com/cnuernber/fast-json)\n* [Large CSV Support](https://cnuernber.github.io/charred/charred.bulk.html#var-batch-csv-rows)\n\n\n## Usage\n\n```clojure\nuser\u003e (require '[charred.api :as charred])\nnil\nuser\u003e (charred/read-json \"{\\\"a\\\": 1, \\\"b\\\": 2}\")\n{\"a\" 1, \"b\" 2}\nuser\u003e (charred/read-json \"{\\\"a\\\": 1, \\\"b\\\": 2}\" :key-fn keyword)\n{:a 1, :b 2}\nuser\u003e (println (charred/write-json-str *1))\n{\n  \"a\": 1,\n  \"b\": 2\n}\n```\n\n#### A Note About Efficiency\n\n\nIf you are reading or writing a lot of small JSON objects the best option is to create a\n[specialized parse fn](https://cnuernber.github.io/charred/charred.api.html#var-parse-json-fn)\nto exactly the options that you need and pass in strings or char[] data.  A [similar pathway](https://cnuernber.github.io/charred/charred.api.html#var-write-json-fn) exists for high\nperformance writing of json objects.  The returned functions are safe to use in multithreaded\ncontexts.\n\nThe system is overall tuned for large files.  Small files or input streams should be setup with `:async?` false\nand smaller `:bufsize` arguments such as 8192 as there is no gain for async loading when the file/stream is smaller than 1MB.\nFor smaller streams slurping into strings in an offline threadpool will lead to the highest performance.  For a particular\nfile size if you know you are going to parse many of these then you should gridsearch `:bufsize` and `:async?` as\nthat is a tuning pathway that I haven't put a ton of time into.  In general the system is tuned towards larger\nfiles as that is when performance really does matter.\n\nAll the parsing systems have mutable options.  These can be somewhat faster and it is interesting to\nlook at the tradeoffs involved.  Parsing a csv using the raw supplier interface is a bit faster\nthan using the Clojure sequence pathway into persistent vectors and it probably doesn't really\nchange your consume pathway so it may be worth trying it.\n\n\n## Development\n\nBefore running a REPL you must compile the java files into target/classes.  This directory\nwill then be on your classpath.\n\n```console\nscripts/compile\n```\n\nTests can be run with `scripts/run-tests` which will compile the java and then run the tests.\n\n\n## Lies, Damn Lies, and Benchmarks!\n\nSee the [fast-json project](https://github.com/cnuernber/fast-json/blob/master/src/fjson.clj#L247).  These times are for \nparsing a 100k json document using keywords for map keys - `:key-fn keyword`.\n\n#### Intel JDK-8\n\n|       method | performance µs |\n|--------------|---------------:|\n|    data.json |           4275 |\n|     jsonista |            754 |\n|      charred |            638 |\n| charred-hamf |            486 |\n\n#### Intel JDK-19\n\n|       method | performance µs |\n|--------------|---------------:|\n|    data.json |           5608 |\n|     jsonista |            856 |\n|      charred |            673 |\n| charred-hamf |            531 |\n\n\n#### Mac m-1 JDK-19\n\n|        method | performance µs |\n|---------------|---------------:|\n|     data.json |           3164 |\n|      jsonista |            285 |\n|       charred |            249 |\n|  charred-hamf |            227 |\n\n\n## License\n\nMIT license.\n","funding_links":["https://github.com/sponsors/cnuernber"],"categories":["Clojure"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcnuernber%2Fcharred","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcnuernber%2Fcharred","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcnuernber%2Fcharred/lists"}