https://github.com/dev-michael-schmidt/spark-scala-stocks

An ETL that loads 20+ years of stock price data for the top stocks, performs some quantitative analysis, and visualizes the results. It uses Spark, Airflow, Grafana, Python, and Scala. That could be cool, right?
https://github.com/dev-michael-schmidt/spark-scala-stocks

Last synced: about 2 months ago
JSON representation

An ETL that loads 20+ years of stock price data for the top stocks, performs some quantitative analysis, and visualizes the results. It uses Spark, Airflow, Grafana, Python, and Scala. That could be cool, right?

• Host: GitHub
• URL: https://github.com/dev-michael-schmidt/spark-scala-stocks
• Owner: dev-michael-schmidt
• Created: 2024-08-26T22:33:29.000Z (almost 2 years ago)
• Default Branch: master
• Last Pushed: 2024-10-24T14:02:01.000Z (over 1 year ago)
• Last Synced: 2024-10-24T17:25:36.224Z (over 1 year ago)
• Language: Scala
• Homepage:
• Size: 312 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# spark-scala-stocks

I delivered on a complete, end-to-end, financial ETL pipeline. I learned

- Apache Airflow

- Scala

- Apache Spark

- Grafana

- Kafka (soon)

  - Broker-controller (static scaling)

  - Calculate important events to trigger email alerts



- Loads 20+ years of stock price data from the S&P 500

- Performs quantitative analysis (currently, parameterized SMA, EMA)

- Visualizes the results

I knew PostgreSQL, Docker (and compose), and Python (yeah, Duh!)

Pretty nifty? There's more to come!