https://github.com/cryptosense/streamcat

Encode and decode concatenated objects as streams
https://github.com/cryptosense/streamcat

codec concatenation file-like json python-library streaming

Last synced: about 1 year ago
JSON representation

Encode and decode concatenated objects as streams

• Host: GitHub
• URL: https://github.com/cryptosense/streamcat
• Owner: cryptosense
• License: bsd-2-clause
• Created: 2017-04-10T11:24:39.000Z (about 9 years ago)
• Default Branch: master
• Last Pushed: 2019-06-06T07:45:49.000Z (about 7 years ago)
• Last Synced: 2025-03-24T13:21:21.084Z (about 1 year ago)
• Topics: codec, concatenation, file-like, json, python-library, streaming
• Language: Python
• Size: 20.5 KB
• Stars: 2
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • Changelog: CHANGELOG.rst
  • License: LICENSE

Awesome Lists containing this project

README

          
streamcat

=========

Examples

--------

Encoding a JSON iterator into a stream:

.. code-block:: python

    def gen_records():

        yield b'{"foo": "bar"}'

        yield b'{"baz": [1, 2, 3]}'

    stream = streamcat.iterator_to_stream(gen_records())

    # `stream` can then be used just like any other `io.RawIOBase`

    with open('/tmp/jsoncat', 'wb') as destination:

        shutil.copyfileobj(stream, destination)

Decoding a stream into a generator:

.. code-block:: python

    decoder = json.JSONDecoder()

    with open('/tmp/jsoncat', 'rb') as source:

        records = streamcat.stream_to_iterator(source, decoder)

        for record in records:

            print(record)