https://github.com/wdonne/pincette-mongo-streams
JSON Streaming With Mongo Streams
https://github.com/wdonne/pincette-mongo-streams
data-pipelines json-streaming kafka-stream mongo-streams pipeline-stages
Last synced: 6 months ago
JSON representation
JSON Streaming With Mongo Streams
- Host: GitHub
- URL: https://github.com/wdonne/pincette-mongo-streams
- Owner: wdonne
- License: bsd-2-clause
- Created: 2020-06-29T16:37:02.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2025-10-22T17:05:25.000Z (8 months ago)
- Last Synced: 2025-10-22T19:09:47.907Z (8 months ago)
- Topics: data-pipelines, json-streaming, kafka-stream, mongo-streams, pipeline-stages
- Language: Java
- Homepage:
- Size: 171 KB
- Stars: 0
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# JSON Streaming With Mongo Streams
This small library allows you to write JSON data streaming pipelines using [MongoDB Aggregation Pipeline Stages](https://docs.mongodb.com/manual/reference/operator/aggregation-pipeline/). It works with
[Kafka Streams](https://kafka.apache.org/documentation/streams/) and the MongoDB database. Its only class has a ```create``` method to which you pass a Kafka stream. It will return another Kafka stream. Only the pipeline stages that are relevant for infinite streams are implemented. There are also a few extension stages to trace and probe your data pipelines. Finally, it includes a
[JSLT](https://github.com/schibsted/jslt) stage, which is useful for transformations that would be too complicated with the MongoDB language alone.
The details are documented in the [API documentation](https://www.javadoc.io/doc/net.pincette/pincette-mongo-streams/latest/index.html).