{"id":15664306,"url":"https://github.com/lucacappelletti94/compress_json","last_synced_at":"2025-08-20T16:30:58.491Z","repository":{"id":57417477,"uuid":"208207285","full_name":"LucaCappelletti94/compress_json","owner":"LucaCappelletti94","description":"The missing Python utility to read and write compressed JSONs.","archived":false,"fork":false,"pushed_at":"2024-10-26T19:24:30.000Z","size":122,"stargazers_count":16,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-10T02:20:26.739Z","etag":null,"topics":["compression","json","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LucaCappelletti94.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"LucaCappelletti94"}},"created_at":"2019-09-13T06:20:52.000Z","updated_at":"2024-10-26T19:24:33.000Z","dependencies_parsed_at":"2024-06-20T00:07:49.330Z","dependency_job_id":"beb31492-2f3d-4324-ad13-b81d244dd497","html_url":"https://github.com/LucaCappelletti94/compress_json","commit_stats":{"total_commits":47,"total_committers":1,"mean_commits":47.0,"dds":0.0,"last_synced_commit":"6f938131a84c01f75e8574019c389d092c25d13d"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LucaCappelletti94%2Fcompress_json","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LucaCappelletti94%2Fcompress_json/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LucaCappelletti94%2Fcompress_json/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LucaCappelletti94%2Fcompress_json/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LucaCappelletti94","download_url":"https://codeload.github.com/LucaCappelletti94/compress_json/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230438185,"owners_count":18225870,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","json","python3"],"created_at":"2024-10-03T13:42:00.734Z","updated_at":"2024-12-19T13:06:56.128Z","avatar_url":"https://github.com/LucaCappelletti94.png","language":"Python","readme":"# compress_json\n\n[![pip](https://badge.fury.io/py/compress-json.svg)](https://pypi.org/project/compress-json/)\n[![python](https://img.shields.io/pypi/pyversions/compress-json.svg)](https://pypi.org/project/compress-json/)\n[![license](https://img.shields.io/pypi/l/compress-json.svg)](https://pypi.org/project/compress-json/)\n[![downloads](https://pepy.tech/badge/compress-json)](https://pepy.tech/project/compress-json)\n[![Github Actions](https://github.com/LucaCappelletti94/ugly_csv_generator/actions/workflows/python.yml/badge.svg)](https://github.com/LucaCappelletti94/ugly_csv_generator/actions/)\n[![Codacy Badge](https://app.codacy.com/project/badge/Grade/6aa4b62b4ed34f7d8e2c37ef09848294)](https://app.codacy.com/gh/LucaCappelletti94/compress_json/dashboard?utm_source=gh\u0026utm_medium=referral\u0026utm_content=\u0026utm_campaign=Badge_grade)\n\nThe missing Python utility to read and write compressed JSONs.\n\nThe library is loosely based on the [compress_pickle](https://github.com/lucianopaz/compress_pickle) library.\n\n## How do I install this package?\n\nAs usual, just download it using pip:\n\n```shell\npip install compress_json\n```\n\n## Available compression modes\n\nThe compression modes, detected automatically by the file name, are **gzip**, **bz2**, and **lzma** (or **xz**), with the notable exception of **zip** which seems difficult to integrate into the JSON pipeline.\n\n## Usage example\n\nThe library is extremely easy to use:\n\n```python\nimport compress_json\n\nD = {\n    \"A\": {\n        \"B\": \"C\"\n    }\n}\ncompress_json.dump(D, \"filepath.json.gz\")   # for a gzip file\ncompress_json.dump(D, \"filepath.json.bz\")   # for a bz2 file\ncompress_json.dump(D, \"filepath.json.lzma\") # for a lzma file\ncompress_json.dump(D, \"filepath.json.xz\") # for a lzma file\n\nD1 = compress_json.load(\"filepath.json.gz\")   # for loading a gzip file\nD2 = compress_json.load(\"filepath.json.bz\")   # for loading a bz2 file\nD3 = compress_json.load(\"filepath.json.lzma\") # for loading a lzma file\nD3 = compress_json.load(\"filepath.json.xz\") # for loading a lzma file\n```\n\nIf it happens that you have to load or dump a JSON object with a custom extension, you can specify the compression mode by passing the `compression` parameter to the `load` and `dump` methods:\n\n```python\nimport compress_json\n\nD = {\n    \"A\": {\n        \"B\": \"C\"\n    }\n}\n\ncompress_json.dump(D, \"filepath.custom_extension1\", compression=\"gzip\")   # for a gzip file\ncompress_json.dump(D, \"filepath.custom_extension2\", compression=\"bz2\")   # for a bz2 file\ncompress_json.dump(D, \"filepath.custom_extension3\", compression=\"lzma\") # for a lzma file\n\nD1 = compress_json.load(\"filepath.custom_extension1\", compression=\"gzip\")   # for loading a gzip file\nD2 = compress_json.load(\"filepath.custom_extension2\", compression=\"bz2\")   # for loading a bz2 file\nD3 = compress_json.load(\"filepath.custom_extension3\", compression=\"lzma\") # for loading a lzma file\n\nassert D == D1 == D2 == D3\n```\n\n## Some extra perks: local loading and dumping\n\nThe library makes available, other than the usual `load` and `dump` from the JSON library, the methods `local_load` and `local_dump`, which let you load and dump files in the same directory as wherever you are calling them, by using the call stack.\n\nThis can be useful, especially when loading files within packages.\n\n```python\nimport compress_json\n\nD = {\n    \"A\": {\n        \"B\": \"C\"\n    }\n}\ncompress_json.local_dump(D, \"filepath.json.gz\")   # for a gzip file\ncompress_json.local_dump(D, \"filepath.json.bz\")   # for a bz2 file\ncompress_json.local_dump(D, \"filepath.json.lzma\") # for a lzma file\n\nD1 = compress_json.local_load(\"filepath.json.gz\")   # for loading a gzip file\nD2 = compress_json.local_load(\"filepath.json.bz\")   # for loading a bz2 file\nD3 = compress_json.local_load(\"filepath.json.lzma\") # for loading a lzma file\n\nassert D == D1 == D2 == D3\n```\n\n## Loading with RAM cache\n\nSometimes you need to load a compressed JSON file a LOT of times, and you may want to put this document in a cache or something of the sort. Fortunately, we already provide this option for you:\n\n```python\nimport compress_json\n\n# The first time you load the file, it will be cached in RAM\nD1 = compress_json.load(\n    \"filepath.json.gz\",\n    use_cache=True\n)\n\n# The second time you load the file, it will be loaded from the cache\nD2 = compress_json.local_load(\n    \"filepath.json.gz\",\n    use_cache=True\n)\n\nassert D1 == D2\n```\n\n## Advanced usage\n\nYou can pass parameters to either the chosen compression mode or the JSON library.\n\nWith the `json_kwargs` parameter, you can specify any of the kwargs that should be forwarded to the JSON library method, which you can obtain for your Python version by running `help(json.dump)` and `help(json.load)`, depending on whether you are dumping or loading the JSON object.\n\nSimilarly, with the `compression_kwargs` parameter, you can specify any parameter that has to be forwarded to the compression library that you intend to use, whether that is `lzma`, `gzip`, or `bz2`, and as per JSON will depend on which version you have installed.\n\nWhether you are dumping or loading a compressed JSON object, you can get the list of parameters you have available to forward to the compression method by running `help(lzma.open)`, `help(gzip.open)`, or `help(bz2.open)`, respectively.\n\n```python\nimport compress_json\n\nD = {\n    \"A\": {\n        \"B\": \"C\"\n    }\n}\ncompress_json.dump(\n    D, \"filepath.json.gz\",\n    compression_kwargs={\n        \"compresslevel\": 9 # The kwargs for gzip\n    },\n    json_kwargs={\n        \"indent\": 4 # The kwargs for json\n    }\n)\n\nD4 = compress_json.load(\n    \"filepath.json.gz\",\n    compression_kwargs={ \n        \"compresslevel\": 9 # The kwargs for gzip\n    },\n    json_kwargs={} # The kwargs for json\n)\n\nassert D == D4\n```\n\n## License\nThe library is released under the MIT license.\n","funding_links":["https://github.com/sponsors/LucaCappelletti94"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucacappelletti94%2Fcompress_json","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucacappelletti94%2Fcompress_json","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucacappelletti94%2Fcompress_json/lists"}