https://github.com/dev-michael-schmidt/spark-scala-stocks
An ETL that loads 20+ years of stock price data for the top stocks, performs some quantitative analysis, and visualizes the results. It uses Spark, Airflow, Grafana, Python, and Scala. That could be cool, right?
https://github.com/dev-michael-schmidt/spark-scala-stocks
Last synced: 3 months ago
JSON representation
An ETL that loads 20+ years of stock price data for the top stocks, performs some quantitative analysis, and visualizes the results. It uses Spark, Airflow, Grafana, Python, and Scala. That could be cool, right?
- Host: GitHub
- URL: https://github.com/dev-michael-schmidt/spark-scala-stocks
- Owner: dev-michael-schmidt
- Created: 2024-08-26T22:33:29.000Z (10 months ago)
- Default Branch: master
- Last Pushed: 2024-10-24T14:02:01.000Z (8 months ago)
- Last Synced: 2024-10-24T17:25:36.224Z (8 months ago)
- Language: Scala
- Homepage:
- Size: 312 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# spark-scala-stocks
I delivered on a complete, end-to-end, financial ETL pipeline. I learned
- Apache Airflow
- Scala
- Apache Spark
- Grafana
- Kafka (soon)
- Broker-controller (static scaling)
- Calculate important events to trigger email alerts
- Loads 20+ years of stock price data from the S&P 500
- Performs quantitative analysis (currently, parameterized SMA, EMA)
- Visualizes the results
I knew PostgreSQL, Docker (and compose), and Python (yeah, Duh!)
Pretty nifty? There's more to come!