https://github.com/solidlabresearch/streaming-query-hive
Combining Multiple Streaming Queries to provide actionable insights
https://github.com/solidlabresearch/streaming-query-hive
Last synced: 8 months ago
JSON representation
Combining Multiple Streaming Queries to provide actionable insights
- Host: GitHub
- URL: https://github.com/solidlabresearch/streaming-query-hive
- Owner: SolidLabResearch
- License: other
- Created: 2025-04-03T18:42:33.000Z (11 months ago)
- Default Branch: master
- Last Pushed: 2025-06-10T00:53:57.000Z (8 months ago)
- Last Synced: 2025-06-10T01:33:53.243Z (8 months ago)
- Language: TypeScript
- Homepage:
- Size: 3.34 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Streaming Query Hive
> Combining Multiple Streaming Queries to provide actionable insights.
## Linting
You run the linter via
```shell
npm run lint:ts
```
You can automatically fix some issues via
```shell
npm run lint:ts:fix
```
## Example Architecture
The Streaming Query Hive can handle multiple streaming queries from different sources, and utilizes different streaming operators to process the data. The architecture is designed to be modular, allowing for easy integration of new sources and operators. An example architecture combining three different sources and the results from the RDF Stream Processing Agents to solve for a specific Parent Query is shown below:
In the architecuture, it is assumed that the query results from the RSSP Agents is being streamed to a MQTT topic. The MQTT topic is then consumed by the Streaming Query Hive, which processes the aggregated results using different streaming operators to solve for the Parent Query. The relationship between the queries to esatablish that the Child Queries are part of the Parent Query is established using the Query Containment [[1](#footnote-1)] Relationship.
The Streaming Queries utilized in the architecture are described in the RSP-QL query language [[2](#footnote-2)]. The tool utilized to find if the queries have the Query Containment [[1](#footnote-1)] or the Query Isomorphism [[3](#footnote-3)] is the RSP-QL Containment Checker [[4](#footnote-4)]. The RSP-QL Containment Checker is a tool that checks if a query is contained in another query, and can be used to determine if the results of one query can be used to solve another query. The tool is designed to work with the RSP-QL query language, and builds on the work done by the SPeCS Solver [[5](#footnote-5)] to support aggregation functions and the streaming semantics of the RSP-QL query language.
The MQTT broker can be easily changed with another broker, such as RabbitMQ or Kafka, in the future.
The resultant query results for the parent query can be streamed to a different MQTT topic, or can be stored in a database for further analysis. Moreover, the results can be reasoned over using a reasoning engine such as EYE-JS[[6](#footnote-6)]. The reasoning engine can be used to infer new knowledge from the query results, and can be used to provide actionable insights from the data.
## License
This code is copyrighted by [Ghent University - imec](https://www.ugent.be/ea/idlab/en) and released under the [MIT Licence](./LICENCE)
## Contact
For any questions, please contact [Kush](mailto:kushbisen@proton.me) or create an issue in the repository [here](https://github.com/SolidLabResearch/streaming-query-hive/issues).
### Footnotes
[1]: Query Containment
[2]: RSP-QL Semantics: A Unifying Query Model to Explain Heterogeneity of RDF Stream Processing Systems
[3]: Matching RDF Graphs
[4]: RSP-QL Containment Checker
[5]: SPeCS Solver
[6]: EYE-JS