https://github.com/varsha-vraj/airport_parking_toolkit

This toolkit is designed to simulate and manage airport parking events. It provides a command-line interface (CLI) for managing vehicles, zones, and parking events. It includes full integration with PostgreSQL for data storage, SQL for advanced queries, and Apache Spark for big data batch processing of parquet logs.
https://github.com/varsha-vraj/airport_parking_toolkit

big-data cli dataengineering hadoop java parquet-files poetry postgresql pyspark python3 spark sqlalchemy

Last synced: about 2 months ago
JSON representation

This toolkit is designed to simulate and manage airport parking events. It provides a command-line interface (CLI) for managing vehicles, zones, and parking events. It includes full integration with PostgreSQL for data storage, SQL for advanced queries, and Apache Spark for big data batch processing of parquet logs.

• Host: GitHub
• URL: https://github.com/varsha-vraj/airport_parking_toolkit
• Owner: varsha-vraj
• License: mit
• Created: 2025-07-19T14:29:37.000Z (3 months ago)
• Default Branch: main
• Last Pushed: 2025-07-19T16:08:59.000Z (3 months ago)
• Last Synced: 2025-08-11T11:02:27.546Z (about 2 months ago)
• Topics: big-data, cli, dataengineering, hadoop, java, parquet-files, poetry, postgresql, pyspark, python3, spark, sqlalchemy
• Language: Python
• Homepage:
• Size: 232 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          


  

  

  
🚗 Airport Parking Management Toolkit


  
Author: varshaa112003@gmail.com

  
Project Type: Python CLI Toolkit + PostgreSQL + Apache Spark


  


    
📖 Description

    
This toolkit is designed to simulate and manage airport parking events. It provides a command-line interface (CLI) for managing vehicles, zones, and parking events. It includes full integration with PostgreSQL for data storage, SQL for advanced queries, and Apache Spark for big data batch processing of parquet logs.

  


  


    
🛠️ Technologies Used

    

      TechnologyPurpose

      PythonCore programming

      CLICommand Line Interface

      PostgreSQLDatabase backend

      SQLAlchemyDatabase ORM

      Apache Spark - PySparkETL on parquet data

      PoetryDependency management

    

  


  


    
📁 Folder Structure

    

airport_parking_project_demo/

│

├── airport_parking_toolkit/

│   ├── cli_tool.py

│   ├── spark_jobs.py

│

├── output_parquet/

│   ├── vehicles.parquet

│   ├── parking_zones.parquet

│   └── parking_events.parquet

│

├── tests/

│   ├── test_cli.py

│

├── queries/

│   ├── queries.sql

│   └── advanced_sql/

│       └── triggers.sql

│

├── logs/

│   └── parking_etl.log

│

├── load_parquet_to_db.py

├── pyproject.toml

└── README.html

    

  


  


    
🚀 How to Run

    


      
📦 Install dependencies:

        
poetry install

      

      
🧹 Clean old logs (if needed):

        
rm -rf logs/parking_etl.log

      

      
▶️ Run the CLI:

        
python -m airport_parking_toolkit.cli_tool

      

      
📥 Load parquet data to PostgreSQL:

        
python load_parquet_to_db.py

      

      
🔥 Run Apache Spark jobs:

        
python airport_parking_toolkit/spark_jobs.py

      

    

  


  


    
📸 Images

    
 1. Vehicle Entry/Exit Logging

    
Validate Parking Records

    
Analyze Zone Utilization

    
 Track Frequent Parkers

    
 Data Quality Issues

    
 Compare Parking Zone Performance 

    
 Schema Diagram

  















  


    
📚 Advanced SQL Features

    


      
Trigger to log insertions into parking_events



      
Stored Functions for computing durations

      
Views for summarised parking statistics

    

  


  


    
📦 Logs

    
ETL logs are written to logs/parking_etl.log after every Spark job run.

  


  

    
✨ Thank You