{"id":46566945,"url":"https://github.com/dupontcyborg/zpickle","last_synced_at":"2026-03-07T07:32:42.733Z","repository":{"id":282512656,"uuid":"948783680","full_name":"dupontcyborg/zpickle","owner":"dupontcyborg","description":"Drop-in pickle replacement with transparent compression - smaller files, same API","archived":false,"fork":false,"pushed_at":"2025-12-06T18:25:36.000Z","size":608,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-12-08T01:58:52.064Z","etag":null,"topics":["bz2","compress","compression","decompress","gzip","lz4","lzma","pickle","python","xz","zlib","zstd"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dupontcyborg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-03-15T00:33:28.000Z","updated_at":"2025-12-07T04:05:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"e6349078-93c5-4492-b237-1be91ad3edcc","html_url":"https://github.com/dupontcyborg/zpickle","commit_stats":null,"previous_names":["dupontcyborg/zpickle"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/dupontcyborg/zpickle","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dupontcyborg%2Fzpickle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dupontcyborg%2Fzpickle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dupontcyborg%2Fzpickle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dupontcyborg%2Fzpickle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dupontcyborg","download_url":"https://codeload.github.com/dupontcyborg/zpickle/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dupontcyborg%2Fzpickle/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30209732,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-07T05:23:27.321Z","status":"ssl_error","status_checked_at":"2026-03-07T05:00:17.256Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bz2","compress","compression","decompress","gzip","lz4","lzma","pickle","python","xz","zlib","zstd"],"created_at":"2026-03-07T07:32:42.184Z","updated_at":"2026-03-07T07:32:42.726Z","avatar_url":"https://github.com/dupontcyborg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# zpickle\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/actions/workflow/status/dupontcyborg/zpickle/test_and_package_wheel.yml\" alt=\"Build Status\"/\u003e\n  \u003cimg src=\"https://img.shields.io/github/v/release/dupontcyborg/zpickle\" alt=\"GitHub Release\"/\u003e\n  \u003cimg src=\"https://img.shields.io/github/license/dupontcyborg/zpickle\" alt=\"License\"/\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/pyversions/zpickle\" alt=\"Python Versions\"/\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/v/zpickle\" alt=\"PyPI Version\"/\u003e\n  \u003cimg src=\"https://static.pepy.tech/badge/zpickle\" alt=\"PyPI Downloads\"\u003e\n\u003c/p\u003e\n\n**Transparent, drop-in compression for Python's pickle — smaller files, same API.**\n\n[`zpickle`](https://pypi.org/project/zpickle/) adds high-performance compression to your serialized Python objects using multiple state-of-the-art algorithms without changing how you work with pickle.\n\n```python\n# Replace this:\nimport pickle\n\n# With this:\nimport zpickle as pickle\n\n# Everything else stays the same!\n```\n\n## Features\n\n- **Drop-in replacement** for the standard `pickle` module\n- **Transparent compression** — everything happens automatically\n- **Multiple algorithms** — choose `zstd`, `brotli`, `zlib`, `lzma`, `bzip2`, or `lz4` (powered by [`compress_utils`](https://github.com/dupontcyborg/compress-utils))\n- **Configure once, use everywhere** — set global defaults for your entire app\n- **Smaller data** — 2-10× smaller serialized data (depending on content and algorithm)\n- **Backward compatible** — automatically reads both compressed and regular pickle data\n- **Complete API compatibility** — all pickle functions work as expected\n\n## Installation\n\n```bash\npip install zpickle\n```\n\n## Quick Start\n\n### Basic Usage\n\n```python\nimport zpickle as pickle\n\n# Serializing works exactly like pickle\ndata = {\"complex\": [\"nested\", {\"data\": \"structure\"}], \"with\": \"lots of repetition\"}\nserialized = pickle.dumps(data)  # Automatically compressed!\n\n# Deserializing works the same way\nrestored = pickle.loads(serialized)  # Automatically decompressed!\n\n# File operations work too\nwith open(\"data.zpkl\", \"wb\") as f:\n    pickle.dump(data, f)\n\nwith open(\"data.zpkl\", \"rb\") as f:\n    restored = pickle.load(f)\n```\n\n### Custom Configuration\n\n```python\nimport zpickle\n\n# Configure global settings\nzpickle.configure(algorithm='brotli', level=9)  # Higher compression\n\n# Or configure for a single operation\ndata = [1, 2, 3] * 1000\ncompressed = zpickle.dumps(data, algorithm='zstd', level=6)\n```\n\n## Performance\n\nCompression ratios versus standard `pickle` (higher is better):\n\n\u003cimg src=\"https://raw.githubusercontent.com/dupontcyborg/zpickle/main/benchmarks/results/compression_ratio.png\" alt=\"Bar graph showing data compression ratios versus pickle\"\u003e\n\nSerialization speed (MB/s, higher is better):\n\n\u003cimg src=\"https://raw.githubusercontent.com/dupontcyborg/zpickle/main/benchmarks/results/compression_speed.png\" alt=\"Bar graph showing data compression speeds versus pickle\"\u003e\n\n*Note: Performance varies by data characteristics. Run benchmarks on your specific data for accurate results.*\n\nTo run your own benchmarks, you can use:\n\n```bash\npython -m benchmarks.benchmark\n```\n\n## How It Works\n\n`zpickle` applies compression with minimal overhead:\n\n1. Objects are first serialized using standard pickle\n2. The pickle data is compressed using the selected algorithm\n3. A small header (8 bytes) is added to identify the format and algorithm\n4. When deserializing, `zpickle` auto-detects the format and decompresses if needed\n\n## API Reference\n\n`zpickle` maintains complete API compatibility with the standard pickle module:\n\n### Core Functions\n\n- `dumps(obj, protocol=None, ..., algorithm=None, level=None)` - Serialize and compress object\n- `loads(data, ...)` - Deserialize and decompress object\n- `dump(obj, file, protocol=None, ..., algorithm=None, level=None)` - Serialize to file\n- `load(file, ...)` - Deserialize from file\n\n### Configuration\n\n- `configure(algorithm=None, level=None, min_size=None)` - Set global defaults\n- `get_config()` - Get current configuration\n\n### Classes\n\n- `Pickler(file, ...)` - Subclass of pickle.Pickler with compression\n- `Unpickler(file, ...)` - Subclass of pickle.Unpickler with decompression\n\n## Alternatives\n\n- **Standard `pickle`**: No compression, but native to Python\n- **`compressed_pickle`**: Similar concept, but less configurable\n- **`joblib`**: More focused on large NumPy arrays and parallel processing\n- **`msgpack`, `protobuf`**: Different serialization formats (not pickle-compatible)\n\n## License\n\nThis project is distributed under the MIT License. [Read more \u003e](LICENSE)\n\n## Links\n\n- [GitHub Repository](https://github.com/dupontcyborg/zpickle)\n- [PyPI Package](https://pypi.org/project/zpickle/)\n- [Issue Tracker](https://github.com/dupontcyborg/zpickle/issues)\n- [compress-utils](https://github.com/dupontcyborg/compress-utils) - The underlying compression library","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdupontcyborg%2Fzpickle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdupontcyborg%2Fzpickle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdupontcyborg%2Fzpickle/lists"}