https://github.com/camilajaviera91/sql-mock-data

Generate a synthetic dataset with one million records of employee information from a fictional company, load it into a PostgreSQL database, create analytical reports using PySpark and large-scale data analysis techniques, and implement machine learning models to predict trends in hiring and layoffs on a monthly and yearly basis.

connection faker locale logging matplotlib os postgresql psycopg2 pyspark pyspark-sql python random random-python shutil sparksession sql sys unicode

Last synced: 17 May 2026

https://github.com/thunchanokbow/inventory-amazon

Inventory value is also important for determining a company's liquidity, or its ability to meet its short-term financial obligations. A high inventory value can indicate that a company has too much money tied up in inventory, which could make it difficult for the company to pay its bills.

azure bigquery cloudcomposer clouddatabase cloudstorage compute-engine dataproc postgresql powerbi pyspark-sql python3

Last synced: 12 Apr 2026

https://github.com/neha-dev-dot/pyspark-tutorial

This repository is part of my journey to learn **PySpark**, the Python API for Apache Spark. I explored the fundamentals of distributed data processing using Spark and practiced with real-world data transformation and querying use cases.

actions data-partitioning dataframes pyspark-basics pyspark-sql rdds sparkbasics sparkcontext sparksession transformations udfs window-functions

Last synced: 17 Apr 2026

https://github.com/essien1990/apache-spark

Batch Processing using Apache Spark and Python for data exploration

apache-spark jupyter-lab jupyter-notebook pyspark pyspark-sql python3

Last synced: 14 Apr 2026

https://github.com/vincentlimarus/machinelearning-models

Clustering vs Classification

classification clustering machine-learning pyspark pyspark-sql

Last synced: 22 Jul 2025

https://github.com/vara-co/home_sales

Module 22 challenge: Using Google Colab to work on Big Data queries with PySpark SQL, parquet, and cache partitions

big-data big-data-analytics cache google-colab google-colaboratory parquet pyspark pyspark-sql

Last synced: 28 Mar 2025

https://github.com/camilajaviera91/pyspark-first-approach

This code demonstrates how to integrate PySpark with datasets and perform simple data transformations. It loads a sample dataset using PySpark's built-in functionalities or reads data from external sources and converts it into a PySpark DataFrame for distributed processing and manipulation.

curses fpdf google-oauth2 gspread kaggle kaggle-api matplotlib os pandas path pathlib pyspark-python pyspark-sql shutil sparksession

Last synced: 07 May 2026

https://github.com/lmizner/codecademy_big_data_with_pyspark

pyspark pyspark-sql python

Last synced: 28 Apr 2026

https://github.com/wb-az/mlib-pyspark-soundlevel-prediction

Creates a ML Pipeline leveraging PySpark SQL and PySpark MLib to predict sound level

correlation-analysis data-engineering-pipeline data-science grid-search mean-absolute-error mean-square-error metrics multivariate-regression nasa pyspark pyspark-mllib pyspark-sql python r-squared regressor rmse sound-processing

Last synced: 05 Mar 2025

https://github.com/bayunova28/airbnb_market_analytics

This repository contains about data analytics project using PySpark SQL for Airbnb at NYC

airbnb business-intelligence data-analytics data-engineering data-science data-visualization pyspark-sql

Last synced: 05 Apr 2025

https://github.com/razamehar/hr-data-analysis-using-pyspark

pyspark pyspark-sql python

Last synced: 17 May 2026

https://github.com/nazif96/disease-prediction

Cardiovascular Disease Prediction

pandas-python pyspark pyspark-ml pyspark-sql python3 scikit-learn

Last synced: 18 Apr 2026

https://github.com/lefteris-souflas/spark-movies-analytics

Utilizing Apache Spark & PySpark to analyze a movie dataset. Tasks include data exploration, identifying top-rated movies, training a linear regression model, and experimenting with Airflow.

apache-airflow cross-validation dag data-splitting hyperparameter-tuning linear-regression model-evaluation one-hot-encoding pipeline pyspark pyspark-mllib pyspark-sql spark-session

Last synced: 22 Jul 2025

https://github.com/bhavanachitragar/data-analysis-using-pyspark

Working with pyspark module in python and using google colab environment in order to apply some queries to the dataset. The dataset consist of two csv files listening.csv and genre.csv. Also, visualizing query results using matplotlib.

data-analysis google-colab pyspark-sql

Last synced: 30 Jun 2025

https://github.com/asenacak/recommendersystems-steamvideogames

als-algorithm alternating-least-squares collaborative-filtering implicit-feedback ncf neural-collaborative-filtering pyspark pyspark-sql python recommendation-system recommender-system steam-games tensorflow

Last synced: 28 Jan 2026