https://github.com/srgchrksv/stream-crypto
Crypto trades streaming with azure services
https://github.com/srgchrksv/stream-crypto
azure binance crypto data databricks dataengineering pyspark python streaming websocket
Last synced: about 2 months ago
JSON representation
Crypto trades streaming with azure services
- Host: GitHub
- URL: https://github.com/srgchrksv/stream-crypto
- Owner: srgchrksv
- Created: 2024-03-05T20:32:51.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-06T21:53:10.000Z (over 2 years ago)
- Last Synced: 2025-02-26T08:43:55.577Z (over 1 year ago)
- Topics: azure, binance, crypto, data, databricks, dataengineering, pyspark, python, streaming, websocket
- Language: Jupyter Notebook
- Homepage:
- Size: 311 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
## Streaming crypto trades data
Got a python script that consumes trades data from binance and produces messages to azure eventhubs.
Databricks notebook processes eventhubs with pyspark and writes delta tables.
Power BI using partner connect to Databricks streams and visualize data
### Streaming system

## Silver layer table

The gold layer of this stream system holds separate table for each symbol
## PowerBI databricks streaming visualizations

Useful links:
- [https://github.com/malvik01/Real-Time-Streaming-with-Azure-Databricks](https://github.com/malvik01/Real-Time-Streaming-with-Azure-Databricks)
- [https://www.youtube.com/watch?v=pwWIegHgNRw](https://www.youtube.com/watch?v=pwWIegHgNRw)
- [https://github.com/binance/binance-spot-api-docs/blob/master/web-socket-streams.md](https://github.com/binance/binance-spot-api-docs/blob/master/web-socket-streams.md)