https://github.com/pushshift/zreader

Read compressed NDJSON .zst files easily
https://github.com/pushshift/zreader

ndjson reddit zstandard zstd

Last synced: 5 months ago
JSON representation

Read compressed NDJSON .zst files easily

• Host: GitHub
• URL: https://github.com/pushshift/zreader
• Owner: pushshift
• License: mit
• Created: 2019-12-18T21:36:57.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2022-07-05T21:38:21.000Z (over 3 years ago)
• Last Synced: 2025-04-07T07:51:15.473Z (6 months ago)
• Topics: ndjson, reddit, zstandard, zstd
• Language: Python
• Size: 152 KB
• Stars: 32
• Watchers: 2
• Forks: 6
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# zreader

Read compressed NDJSON .zst files easily

Usage:

Take a look at the example.py script. When creating a new zreader, you can pass a chunk size or let it default to 16k.