{"id":23796852,"url":"https://github.com/tonsky/fast-edn","last_synced_at":"2025-04-12T21:28:34.414Z","repository":{"id":261667445,"uuid":"884990405","full_name":"tonsky/fast-edn","owner":"tonsky","description":"Drop-in replacement for clojure.edn that is 6 times faster","archived":false,"fork":false,"pushed_at":"2025-01-05T17:04:04.000Z","size":370,"stargazers_count":159,"open_issues_count":2,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-04T03:12:18.029Z","etag":null,"topics":["clojure","edn","parser"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tonsky.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["tonsky"],"patreon":"tonsky"}},"created_at":"2024-11-07T18:48:29.000Z","updated_at":"2025-04-01T16:01:36.000Z","dependencies_parsed_at":"2024-11-24T20:31:33.802Z","dependency_job_id":"6f0cfcb4-5edc-42e3-814d-8a9a8ca2c478","html_url":"https://github.com/tonsky/fast-edn","commit_stats":{"total_commits":49,"total_committers":1,"mean_commits":49.0,"dds":0.0,"last_synced_commit":"7793d236310068ae7254c4fe3377f198831b49a9"},"previous_names":["tonsky/better-clojure","tonsky/fast-edn"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tonsky%2Ffast-edn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tonsky%2Ffast-edn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tonsky%2Ffast-edn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tonsky%2Ffast-edn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tonsky","download_url":"https://codeload.github.com/tonsky/fast-edn/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248633984,"owners_count":21136958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","edn","parser"],"created_at":"2025-01-01T20:11:45.108Z","updated_at":"2025-04-12T21:28:34.390Z","avatar_url":"https://github.com/tonsky.png","language":"Clojure","funding_links":["https://github.com/sponsors/tonsky","https://patreon.com/tonsky"],"categories":[],"sub_categories":[],"readme":"# Fast EDN parser\n\n\u003e EDN format is very similar to JSON, thus it should parse as fast as JSON.\n\nFast EDN is a drop-in replacement for `clojure.edn/read-string` that is roughly 6 times faster:\n\n| Test file         | clojure.edn | fast-edn.core | speed up, times |\n| :---              |        ---: |          ---: |            ---: |\n| basic_10          |       0.504 |         0.290 |          ×  1.7 |\n| basic_100         |       3.040 |         0.594 |          ×  5.1 |\n| basic_1000        |      19.495 |         2.815 |          ×  6.9 |\n| basic_10000       |     221.773 |        37.560 |          ×  5.9 |\n| basic_100000      |    2138.255 |       370.045 |          ×  5.8 |\n| ints_1400         |     431.432 |        33.164 |          × 13.0 |\n| keywords_10       |       3.961 |         0.625 |          ×  6.3 |\n| keywords_100      |      34.980 |         4.769 |          ×  7.3 |\n| keywords_1000     |     369.404 |        53.943 |          ×  6.8 |\n| keywords_10000    |    4168.732 |       662.099 |          ×  6.3 |\n| nested_100000     |    2585.372 |       503.644 |          ×  5.1 |\n| strings_1000      |     651.043 |        40.455 |          × 16.1 |\n| strings_uni_250   |     641.900 |       108.341 |          ×  5.9 |\n\nFast EDN achieves JSON parsing speeds (json + keywordize keys vs EDN of the same size):\n\n| File size    | cheshire | jsonista | charred | fast-edn |\n| :---         |     ---: |     ---: |    ---: |      --: |\n| basic_10     |    0.588 |    0.137 |   0.328 |    0.290 |\n| basic_100    |    1.043 |    0.594 |   0.721 |    0.594 |\n| basic_1000   |    4.224 |    2.999 |   3.016 |    2.815 |\n| basic_10000  |   37.793 |   34.374 |  32.623 |   37.560 |\n| basic_100000 |  359.558 |  327.997 | 313.280 |  370.045 |\n\nSpeed of EDN parsing makes Transit obsolete on JVM:\n\n| file         | clojure.edn | transit+msgpack | transit+json |   fast-edn |\n| :---         |        ---: |            ---: |         ---: |       ---: |\n| basic_10     |       0.481 |           2.832 |        1.474 |      0.290 |\n| basic_100    |       2.799 |           4.242 |        2.297 |      0.594 |\n| basic_1000   |      17.548 |          14.738 |        6.583 |      2.815 |\n| basic_10000  |     211.536 |         125.741 |       46.849 |     37.560 |\n| basic_100000 |    2016.885 |        1167.972 |      447.013 |    370.045 |\n\nAll execution times above are in µs, M1 Pro 16 Gb, single thread, JDK Zulu23.30+13-CA.\n\nTo run benchmarks yourself:\n\n```sh\n./script/bench_json.sh\n./script/bench_edn.sh\n./script/bench_transit.sh\n```\n\n## Other benefits\n\nFast EDN has more consistent error reporting. Clojure:\n\n```clojure\n(clojure.edn/read-string \"1a\")\n; =\u003e NumberFormatException: Invalid number: 1a\n\n(clojure.edn/read-string \"{:a 1 :b\")\n; =\u003e RuntimeException: EOF while reading\n\n(clojure.edn/read-string \"\\\"{:a 1 :b\")\n; =\u003e RuntimeException: EOF while reading string\n\n(clojure.edn/read-string \"\\\"\\\\u123\\\"\")\n; =\u003e IllegalArgumentException: Invalid character length: 3, should be: 4\n```\n\nFast EDN includes location information in exceptions:\n\n```clojure\n(fast-edn.core/read-string \"1a\")\n; =\u003e NumberFormatException: For input string: \"1a\", offset: 2, context:\n;    1a\n;     ^\n\n(fast-edn.core/read-string \"{:a 1 :b\")\n; =\u003e RuntimeException: Map literal must contain an even number of forms: {:a 1, :b, offset: 8, context:\n;    {:a 1 :b\n;           ^\n\n(fast-edn.core/read-string \"\\\"{:a 1 :b\")\n; =\u003e RuntimeException: EOF while reading string: \"{:a 1 :b, offset: 9, context:\n;    \"{:a 1 :b\n;            ^\n\n(fast-edn.core/read-string \"\\\"\\\\u123\\\"\")\n; =\u003e RuntimeException: Unexpected digit: \", offset: 7, context:\n;    \"\\u123\"\n;          ^\n```\n\nOptionally, you can include line number/column information at the cost of a little performance:\n\n```clojure\n(read-string {:count-lines true} \"\\\"abc\")\n; =\u003e RuntimeException: EOF while reading string: \"abc, line: 1, column: 5, offset: 4, context:\n;    \"abc\n;       ^\n```\n\n## Using\n\nAdd this to `deps.edn`:\n\n```clojure\nio.github.tonsky/fast-edn {:mvn/version \"1.1.0\"}\n```\n\n`read-string` works exactly the same as in `clojure.edn`:\n\n```clojure\n(require '[fast-edn.core :as edn])\n\n;; Read from string\n(edn/read-string \"{:a 1}\")\n\n;; Options\n(edn/read-string\n  {:eof     ::eof\n   :readers {'inst #(edn/parse-timestamp edn/construct-instant %)}\n   :default (fn [tag value]\n              (clojure.core/tagged-literal tag value))})\n```\n\nIn addition to strings, `fast-edn.core/read-once` allows you to read from `InputStream`, `File`, `byte[]`, `char[]` and `String`:\n\n```clojure\n(edn/read-once (io/file \"data.edn\"))\n```\n\nNote that `read-once` closes the Reader/InputStream you pass to it, so it’s not a direct analogue of `clojure.edn/read`.\n\nConsuming multiple sequential objects from the same Reader/InputStream is possible but looks slightly different. In Clojure:\n\n```clojure\n(let [r (java.io.PushbackReader. reader)]\n  (take-while #(not= ::eof %)\n    (repeatedly #(clojure.edn/read {:eof ::eof} r))))\n```\n\nIn Fast EDN:\n\n```clojure\n(let [p (fast-edn.core/parser {:eof ::eof} reader)]\n  (take-while #(not= ::eof %)\n    (repeatedly #(fast-edn.core/read-next p))))\n```\n\n## Compatibility\n\nFast EDN is 100% compatible with clojure.edn. It will read everything that clojure.edn would.\n\nMost cases that clojure.edn rejects, Fast EDN will reject too. There are some minor exceptions though: Fast EDN is a tiny bit more permissive than clojure.edn. We tried to follow intent and just simplify/streamline edge cases where it made sense.\n\nIn Fast EDN, ratios can be specified with arbitrary integers:\n\n```clojure\n(clojure.edn/read-string \"2r1111N\")\n; =\u003e NumberFormatException: For input string: \"1111N\" under radix 2\n\n(fast-edn.core/read-string \"2r1111N\")\n; =\u003e 15N\n\n(clojure.edn/read-string \"0xFF/0x02\")\n; =\u003e NumberFormatException: Invalid number: 0xFF/0x02\n\n(fast-edn.core/read-string \"0xFF/0x02\")\n; =\u003e 255/2\n```\n\nSymbols/keywords can have slashes anywhere, first slash is ns separator. Clojure allows them _almost_ anywhere but rules for when it doesn’t are _weird_:\n\n```clojure\n(clojure.edn/read-string \":ns/sym/\")\n; =\u003e RuntimeException: Invalid token: :ns/sym/\n\n(read-string \":ns/sym/\")\n; =\u003e :ns/sym/\n```\n\nSame goes for keywords starting with a number. Clojure allows `:1a` but not `:ns/1a` and it seems like an oversight rather than a deliberate design decision:\n\n```clojure\n(clojure.edn/read-string \":ns/1a\")\n; =\u003e RuntimeException: Invalid token: :ns/1a\n\n(fast-edn.core/read-string \":ns/1a\")\n; =\u003e :ns/1a\n```\n\nWe also support vectors in metadata since Clojure supports them and EDN parser was probably just not updated in time.\n\n```clojure\n(clojure.edn/read-string \"^[tag] {}\")\n; =\u003e IllegalArgumentException: Metadata must be Symbol,Keyword,String or Map\n\n(fast-edn.core/read-string \"^[tag] {}\")\n; =\u003e {:param-tags ['tag]} {}\n```\n\nAccording to [github.com/edn-format/edn](https://github.com/edn-format/edn), metadata should not be handled by EDN at all, but `clojure.edn` supports it and so are we.\n\n## Test coverage\n\nFast EDN is extensively tested by test suite from clojure.core, by our own generative test suite and by a set of hand-crafted test cases.\n\nTo run tests yourself:\n\n```sh\n./script/test.sh\n```\n\n## What’s the secret?\n\nFast EDN achieves its speed mainly by avoiding two things clojure.edn does:\n\n- reading from Reader one char at a time,\n- using regexps.\n\n## Appreciation\n\n- [charred](https://github.com/cnuernber/charred) for starting point\n- [clj-async-profiler](https://github.com/clojure-goes-fast/clj-async-profiler) and\n[criterium](https://github.com/hugoduncan/criterium/) for providing the tools\n\n## License\n\nCopyright © 2024 Nikita Prokopov\n\nLicensed under [MIT](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftonsky%2Ffast-edn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftonsky%2Ffast-edn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftonsky%2Ffast-edn/lists"}