https://github.com/pushshift/zreader

Read compressed NDJSON .zst files easily
https://github.com/pushshift/zreader

ndjson reddit zstandard zstd

Last synced: about 2 months ago
JSON representation

Read compressed NDJSON .zst files easily

• Host: GitHub
• URL: https://github.com/pushshift/zreader
• Owner: pushshift
• License: mit
• Created: 2019-12-18T21:36:57.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2022-07-05T21:38:21.000Z (over 2 years ago)
• Last Synced: 2023-03-03T16:09:57.203Z (almost 2 years ago)
• Topics: ndjson, reddit, zstandard, zstd
• Language: Python
• Size: 152 KB
• Stars: 22
• Watchers: 3
• Forks: 4
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# zreader

Read compressed NDJSON .zst files easily

Usage:

Take a look at the example.py script. When creating a new zreader, you can pass a chunk size or let it default to 16k.