{"id":22611437,"url":"https://github.com/rmoralespp/jsonl","last_synced_at":"2026-03-11T18:44:03.431Z","repository":{"id":252932113,"uuid":"841928260","full_name":"rmoralespp/jsonl","owner":"rmoralespp","description":"A lightweight Python library for handling jsonlines files","archived":false,"fork":false,"pushed_at":"2025-09-16T14:30:11.000Z","size":3621,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-19T08:47:13.570Z","etag":null,"topics":["bzip2","deserialization","files","gzip","json","jsonl","jsonlines","ndjson","python","serialization","tar","utils","xz","zip"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rmoralespp.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-13T10:24:51.000Z","updated_at":"2025-09-17T11:12:38.000Z","dependencies_parsed_at":"2024-08-22T17:44:33.712Z","dependency_job_id":"e45bdf62-570c-467b-b930-40864389aa01","html_url":"https://github.com/rmoralespp/jsonl","commit_stats":null,"previous_names":["rmoralespp/jsonl"],"tags_count":28,"template":false,"template_full_name":null,"purl":"pkg:github/rmoralespp/jsonl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmoralespp%2Fjsonl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmoralespp%2Fjsonl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmoralespp%2Fjsonl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmoralespp%2Fjsonl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rmoralespp","download_url":"https://codeload.github.com/rmoralespp/jsonl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmoralespp%2Fjsonl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001425,"owners_count":26083078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bzip2","deserialization","files","gzip","json","jsonl","jsonlines","ndjson","python","serialization","tar","utils","xz","zip"],"created_at":"2024-12-08T16:10:55.150Z","updated_at":"2026-03-11T18:44:03.411Z","avatar_url":"https://github.com/rmoralespp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cstrong\u003ejsonl\u003c/strong\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cem\u003eA lightweight, dependency-free Python library for JSON Lines — read, write, compress, and stream with ease.\u003c/em\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://pypi.python.org/pypi/py-jsonl\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/py-jsonl.svg\" alt=\"PyPI version\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/rmoralespp/jsonl\"\u003e\u003cimg src=\"https://img.shields.io/pypi/pyversions/py-jsonl.svg\" alt=\"Python versions\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/rmoralespp/jsonl/actions?query=event%3Arelease+workflow%3ACI\"\u003e\u003cimg src=\"https://github.com/rmoralespp/jsonl/workflows/CI/badge.svg\" alt=\"CI\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://app.codecov.io/gh/rmoralespp/jsonl\"\u003e\u003cimg src=\"https://codecov.io/gh/rmoralespp/jsonl/branch/main/graph/badge.svg\" alt=\"Coverage\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/rmoralespp/jsonl/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/rmoralespp/jsonl.svg\" alt=\"License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pepy.tech/project/py-jsonl\"\u003e\u003cimg src=\"https://pepy.tech/badge/py-jsonl\" alt=\"Downloads\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://rmoralespp.github.io/jsonl/\"\u003eDocumentation\u003c/a\u003e\n  ·\n  \u003ca href=\"https://github.com/rmoralespp/jsonl/blob/main/CHANGELOG.md\"\u003eChangelog\u003c/a\u003e\n  ·\n  \u003ca href=\"https://github.com/rmoralespp/jsonl/issues\"\u003eIssues\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n**jsonl** provides a simple, Pythonic API for working with [JSON Lines](https://jsonlines.org/) data.\nIt follows the conventions of Python's standard `json` module — if you know `json.dump` and `json.load`,\nyou already know how to use **jsonl**.\n\nFully compliant with the [jsonlines](https://jsonlines.org/) and [ndjson](https://github.com/ndjson/ndjson-spec)\nspecifications.\n\n## Features\n\n| Feature                        | Description                                                                |\n|--------------------------------|----------------------------------------------------------------------------|\n| 🌎 **Familiar API**            | Interface similar to the standard `json` module (`dump`, `load`, `dumps`)  |\n| ⚡ **Streaming by default**     | Read and write incrementally via iterators, keeping memory usage low       |\n| 🗜️ **Built-in compression**   | Transparent support for `gzip`, `bzip2`, and `xz`                          |\n| 📦 **Archive support**         | Read and write `ZIP` and `TAR` archives (`.tar.gz`, `.tar.bz2`, `.tar.xz`) |\n| 📥 **Load from URLs**          | Pass a URL directly to `load()` or `load_archive()`                        |\n| 🚀 **Pluggable serialization** | Swap in [`orjson`](https://github.com/ijl/orjson), or any JSON library     |\n| 🔧 **Error tolerance**         | Optionally skip malformed lines instead of crashing                        |\n| 🐍 **Zero dependencies**       | Uses only the Python standard library — nothing else                       |\n\n## Installation\n\n```bash\npip install py-jsonl\n```\n\n\u003e Requires **Python 3.8+**. No external dependencies.\n\n## Quick Start\n\n### Write\n\n```python\nimport jsonl\n\ndata = [\n    {\"name\": \"Gilbert\", \"wins\": [[\"straight\", \"7♣\"], [\"one pair\", \"10♥\"]]},\n    {\"name\": \"May\", \"wins\": []},\n]\n\njsonl.dump(data, \"players.jsonl\")\n```\n\n### Read\n\n```python\nimport jsonl\n\nfor item in jsonl.load(\"players.jsonl\"):\n    print(item)\n```\n\n### Read from a URL\n\n```python\nimport jsonl\n\nfor item in jsonl.load(\"https://example.com/data.jsonl\"):\n    print(item)\n```\n\n### Compressed files\n\nThe compression format is determined automatically — by file extension when writing,\nand by [magic numbers](https://en.wikipedia.org/wiki/List_of_file_signatures) when reading \nif the file extension is not recognized:\n\n```python\nimport jsonl\n\ndata = [{\"key\": \"value\"}]\n\njsonl.dump(data, \"file.jsonl.gz\")  # gzip\njsonl.dump(data, \"file.jsonl.bz2\")  # bzip2\njsonl.dump(data, \"file.jsonl.xz\")  # xz\n\nfor item in jsonl.load(\"file.jsonl.gz\"):\n    print(item)\n```\n\n### Archives (ZIP / TAR)\n\n```python\nimport jsonl\n\n# Write multiple files into an archive\ndata = [\n    (\"users.jsonl\", [{\"name\": \"Alice\", \"age\": 30}, {\"name\": \"Bob\", \"age\": 25}]),\n    (\"orders.jsonl\", [{\"id\": 1, \"total\": 99.90}, {\"id\": 2, \"total\": 45.00}]),\n]\njsonl.dump_archive(\"data.tar.gz\", data)\n\n# Read them back\nfor filename, items in jsonl.load_archive(\"data.tar.gz\"):\n    print(f\"--- {filename} ---\")\n    for item in items:\n        print(item)\n```\n\n### Multiple output files\n\n```python\nimport jsonl\n\ndata = [\n    (\"file1.jsonl\", [{\"name\": \"Alice\"}, {\"name\": \"Bob\"}]),\n    (\"file2.jsonl\", [{\"name\": \"Charlie\"}]),\n    (\"file1.jsonl\", [{\"name\": \"Eve\"}]),  # appended to file1.jsonl\n]\n\njsonl.dump_fork(data)\n```\n\n## API Overview\n\n### Reading\n\n| Function                             | Description                                       |\n|--------------------------------------|---------------------------------------------------|\n| `jsonl.load(source, **kw)`           | Read from a file, URL, or file-like object        |\n| `jsonl.load_archive(file, **kw)`     | Unpack JSON Lines files from a ZIP or TAR archive |\n| `jsonl.loader(stream, broken, **kw)` | Low-level generator deserializing a line stream   |\n\n\u003e [!TIP]\n\u003e All **read** functions accept `json_loads` and `**json_loads_kwargs` for custom deserialization.\n\n### Writing\n\n| Function                               | Description                                              |\n|----------------------------------------|----------------------------------------------------------|\n| `jsonl.dump(iterable, file, **kw)`     | Write objects to a JSON Lines file                       |\n| `jsonl.dumps(iterable, **kw)`          | Serialize to a JSON Lines string                         |\n| `jsonl.dump_fork(paths, **kw)`         | Write to multiple JSON Lines files at once               |\n| `jsonl.dump_archive(path, data, **kw)` | Pack multiple JSON Lines files into a ZIP or TAR archive |\n| `jsonl.dumper(iterable, **kw)`         | Low-level generator yielding formatted lines             |\n\n\u003e [!TIP]\n\u003e All **write** functions accept `json_dumps` and `**json_dumps_kwargs` for custom serialization.\n\nFor complete parameter documentation, see the [full docs →](https://rmoralespp.github.io/jsonl/)\n\n## Custom Serialization\n\nPlug in any JSON-compatible serializer. For example, [`orjson`](https://github.com/ijl/orjson)\nfor high-performance encoding:\n\n```python\nimport orjson  # ensure orjson is installed: pip install orjson\nimport jsonl\n\ndata = [{\"name\": \"Alice\", \"age\": 30}, {\"name\": \"Bob\", \"age\": 25}]\n\n# Write with orjson (returns bytes → set text_mode=False)\njsonl.dump(data, \"fast.jsonl\", json_dumps=orjson.dumps, text_mode=False)\n\n# Read with orjson\nfor item in jsonl.load(\"fast.jsonl\", json_loads=orjson.loads):\n    print(item)\n```\n\nExtra keyword arguments are forwarded to the underlying serializer:\n\n```python\nimport jsonl\n\ndata = [{\"name\": \"Alice\", \"score\": 9.5}, {\"name\": \"Bob\", \"score\": 7.2}]\n\njsonl.dump(data, \"compact.jsonl\", separators=(\",\", \":\"))  # compact output\njsonl.dump(data, \"sorted.jsonl\", sort_keys=True)  # deterministic keys\n```\n\n## Supported Formats\n\n| Type        | Extensions                               |\n|-------------|------------------------------------------|\n| Plain       | `.jsonl`                                 |\n| Compressed  | `.jsonl.gz`, `.jsonl.bz2`, `.jsonl.xz`   |\n| ZIP archive | `.zip`                                   |\n| TAR archive | `.tar`, `.tar.gz`, `.tar.bz2`, `.tar.xz` |\n\n\u003e When reading, if the file extension is not recognized, **jsonl** falls back to\n\u003e [magic-number detection](https://en.wikipedia.org/wiki/List_of_file_signatures)\n\u003e to identify the compression format automatically.\n\n## Contributing\n\n```bash\n# Install dev dependencies\npip install --group=test --upgrade\n\n# Run tests\npython -m pytest tests/\npython -m pytest tests/ --cov  # run with coverage reporting\n\n# Lint\npip install --group=lint --upgrade\nruff check .\n\n# Docs\npip install --group=doc --upgrade\n\n# zensical usage: https://zensical.org/docs/usage/\nzensical build \nzensical serve\n```\n\n## License\n\nMIT — see [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmoralespp%2Fjsonl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frmoralespp%2Fjsonl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmoralespp%2Fjsonl/lists"}