{"id":13477214,"url":"https://github.com/mre/hyperjson","last_synced_at":"2025-03-27T04:32:57.073Z","repository":{"id":33428804,"uuid":"130034482","full_name":"mre/hyperjson","owner":"mre","description":"🐍 A hyper-fast Python module for reading/writing JSON data using Rust's serde-json.","archived":true,"fork":false,"pushed_at":"2023-01-04T04:09:29.000Z","size":4899,"stargazers_count":504,"open_issues_count":15,"forks_count":40,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-09-17T22:52:42.615Z","etag":null,"topics":["decode","encode","extension","hacktoberfest","json","module","python","python-json","rust","serde"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mre.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":"mre","patreon":"hellorust"}},"created_at":"2018-04-18T09:01:40.000Z","updated_at":"2024-09-05T01:46:45.000Z","dependencies_parsed_at":"2023-01-15T01:00:45.804Z","dependency_job_id":null,"html_url":"https://github.com/mre/hyperjson","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mre%2Fhyperjson","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mre%2Fhyperjson/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mre%2Fhyperjson/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mre%2Fhyperjson/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mre","download_url":"https://codeload.github.com/mre/hyperjson/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222194835,"owners_count":16946986,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["decode","encode","extension","hacktoberfest","json","module","python","python-json","rust","serde"],"created_at":"2024-07-31T16:01:39.509Z","updated_at":"2025-03-27T04:32:57.066Z","avatar_url":"https://github.com/mre.png","language":"Python","readme":"![hyperjson](assets/logo.gif)\n\n![Build Status](https://github.com/mre/hyperjson/workflows/CI/badge.svg)\n\n\nA hyper-fast, safe Python module to read and write JSON data. Works as a\ndrop-in replacement for Python's built-in\n[json](https://docs.python.org/3/library/json.html) module.\nThis is alpha software and there will be bugs, so maybe don't deploy to production _just_ yet. :wink:\n\n## :warning: NOTE\n\nThis project is not actively maintained. [orjson](https://github.com/ijl/orjson) is likely the better alternative. \n\n\n## Installation\n\n```\npip install hyperjson\n```\n\n## Usage\n\nhyperjson is meant as a drop-in replacement for Python's [json\nmodule](https://docs.python.org/3/library/json.html):\n\n```python\n\u003e\u003e\u003e import hyperjson\n\u003e\u003e\u003e hyperjson.dumps([{\"key\": \"value\"}, 81, True])\n'[{\"key\":\"value\"},81,true]'\n\u003e\u003e\u003e hyperjson.loads(\"\"\"[{\"key\": \"value\"}, 81, true]\"\"\")\n[{u'key': u'value'}, 81, True]\n```\n\n## Motivation\n\nParsing JSON is a solved problem; so, no need to reinvent the wheel, right?  \nWell, unless you care about **performance and safety**.\n\nTurns out, parsing JSON _correctly_ is a [hard problem](http://seriot.ch/parsing_json.php). Thanks to Rust however, we can minimize the risk of running into [stack overflows or segmentation faults](https://github.com/esnme/ultrajson/issues) however.\n\nhyperjson is a thin wrapper around Rust's [serde-json](https://github.com/serde-rs/json) and [pyo3](https://github.com/PyO3/pyo3). It is compatible with Python 3 (and 2 on a best-effort basis).\n\nFor a more in-depth discussion, [watch the talk about this project recorded at the Rust Cologne Meetup in August 2018.](https://media.ccc.de/v/rustcologne.2018.08.hyperjson)\n\n## Goals\n\n- **Compatibility**: Support the full feature-set of Python's `json` module.\n- **Safety**: No segfaults, panics, or overflows.\n- **Performance**: Significantly faster than `json` and as fast as `ujson` (both written in C).\n\n## Non-goals\n\n- **Support ujson and simplejson extensions**:  \n  Custom extensions like `encode()`, `__json__()`, or `toDict()` are not\n  supported. The reason is, that they go against PEP8 (e.g. `dunder` methods\n  are restricted to the standard library, camelCase is not Pythonic) and are not\n  available in Python's `json` module.\n- **Whitespace preservation**: Whitespace in JSON strings is not preserved.\n  Mainly because JSON is a whitespace-agnostic format and `serde-json` strips\n  them out by default. In practice this should not be a problem, since your\n  application must not depend on whitespace padding, but it's something to be\n  aware of.\n\n## Benchmark\n\nWe are _not_ fast yet. That said, we haven't done any big optimizations.\nIn the long-term we might explore features of newer CPUs like multi-core and SIMD.\nThat's one area other (C-based) JSON extensions haven't touched yet, because it might\nmake code harder to debug and prone to race-conditions. In Rust, this is feasible due to crates like\n[faster](https://github.com/AdamNiederer/faster) or\n[rayon](https://github.com/nikomatsakis/rayon).\n\nSo there's a chance that the following measurements might improve soon.  \nIf you want to help, check the instructions in the _Development Environment_ section below.\n\n**Test machine:**  \nMacBook Pro 15 inch, Mid 2015 (2,2 GHz Intel Core i7, 16 GB RAM) Darwin 17.6.18\n\n![Serialization benchmarks](assets/serialize.png)\n![Deserialization benchmarks](assets/deserialize.png)\n\n## Contributions welcome!\n\nIf you would like to hack on hyperjson, here's what needs to be done:\n\n- [x] Implement [`loads()`](https://docs.python.org/3/library/json.html#json.loads)\n- [x] Implement [`load()`](https://docs.python.org/3/library/json.html#json.load)\n- [x] Implement [`dumps()`](https://docs.python.org/3/library/json.html#json.dumps)\n- [x] Implement [`dump()`](https://docs.python.org/3/library/json.html#json.dump)\n- [x] Benchmark against [json](https://docs.python.org/3/library/json.html) and\n      [ujson](https://github.com/esnme/ultrajson/) (see [#1](https://github.com/mre/hyperjson/issues/1))\n- [x] Add a CI/CD pipeline for easier testing (see [#2](https://github.com/mre/hyperjson/issues/2))\n- [x] Create a proper pip package from it, to make installing easier (see [#3](https://github.com/mre/hyperjson/issues/3)).\n- [ ] Profile and optimize performance (see [#16](https://github.com/mre/hyperjson/issues/16))\n- [ ] Add remaining [keyword-only arguments](https://docs.python.org/3/library/json.html#basic-usage) to methods\n\nJust pick one of the open tickets. We can provide mentorship if you like. :smiley:\n\n## Developer guide\n\nThis project uses [poetry](https://python-poetry.org/docs/) for managing the development environment. If you don't have it installed, run\n\n```\ncurl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python\nexport PATH=\"$HOME/.poetry/bin:$PATH\"\n```\n\nThe project requires the `nightly` version of Rust.\n\nInstall it via `rustup`:\n\n```\nrustup install nightly\n```\n\nIf you have already installed the `nightly` version, make sure it is up-to-date:\n\n```\nrustup update nightly\n```\n\nAfter that, you can compile the current version of hyperjson and execute all tests and benchmarks with the following commands:\n\n```\nmake install\nmake test\nmake bench\n```\n\n🤫 Pssst!... run `make help` to learn more.\n\n## Drawing pretty diagrams\n\nIn order to recreate the benchmark histograms, you first need a few additional prerequisites:\n\n- [Matplotlib](https://matplotlib.org/)\n- [Numpy](http://www.numpy.org/)\n\nOn macOS, please also add the following to your `~/.matplotlib/matplotlibrc` ([reference](https://markhneedham.com/blog/2018/05/04/python-runtime-error-osx-matplotlib-not-installed-as-framework-mac/)):\n\n```\nbackend: TkAgg\n```\n\nAfter that, run the following:\n\n```\nmake plot\n```\n\n## License\n\nhyperjson is licensed under either of\n\n- Apache License, Version 2.0, (LICENSE-APACHE or\n  http://www.apache.org/licenses/LICENSE-2.0)\n- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)\n\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in hyperjson by you, as defined in the Apache-2.0 license, shall\nbe dual licensed as above, without any additional terms or conditions.\n","funding_links":["https://github.com/sponsors/mre","https://patreon.com/hellorust"],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmre%2Fhyperjson","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmre%2Fhyperjson","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmre%2Fhyperjson/lists"}