https://github.com/leisurelyleon/mastercard-lead-data-engineer
A tailored list of exemplified files corresponding to required skills for an appliable career position at Mastercard Inc.
https://github.com/leisurelyleon/mastercard-lead-data-engineer
apache apache-spark big-data hive impala java kafka nifi nosql nosql-database nosql-databases object-oriented object-oriented-programming oozie postgresql python scala spark sqoop
Last synced: about 2 months ago
JSON representation
A tailored list of exemplified files corresponding to required skills for an appliable career position at Mastercard Inc.
- Host: GitHub
- URL: https://github.com/leisurelyleon/mastercard-lead-data-engineer
- Owner: leisurelyleon
- License: mit
- Created: 2025-04-24T05:29:40.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-10-17T04:45:18.000Z (8 months ago)
- Last Synced: 2025-11-02T20:04:09.085Z (7 months ago)
- Topics: apache, apache-spark, big-data, hive, impala, java, kafka, nifi, nosql, nosql-database, nosql-databases, object-oriented, object-oriented-programming, oozie, postgresql, python, scala, spark, sqoop
- Language: Java
- Homepage: https://careers.mastercard.com/us/en/job/R-243536/Lead-Data-Engineer?
- Size: 60.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Mastercard Lead Data Engineer (Demo Project)
Welcome to the `mastercard-lead-data-engineer` repository. This project serves as a **comprehensive demonstration portfolio** to simulate the technical responsibilities and skills required for a **Lead Data Engineer** role at Mastercard.
It is structured around a mock job description and showcases a curated selection of coding samples, configuration files, shell scripts, and architectural documentation that align with the expectations of enterprise-grade data engineering work.
> 📍 **Note:** This project is not affiliated with or endorsed by Mastercard. All files and examples are hypothetical and solely for educational purposes.
---
## 📂 Directory Structure
Each folder corresponds to a critical area of expertise listed in the job role:
```
mastercard-lead-data-engineer/
├── 01_cloudera_integration/ # HDFS & Cloudera configuration integration
├── 02_spark_processing/ # Spark batch and streaming pipelines
├── 03_programming_fundamentals/ # OOP, data structures, and algorithms
├── 04_databases/ # Relational (Postgres) and NoSQL (MongoDB, Cassandra)
├── 05_streaming_tools/ # Hive, Impala, NiFi, Kafka, Sqoop, Oozie workflows
├── 06_shell_scripting/ # Bash automation scripts for pipelines and user ops
└── 07_systems_and_design/ # Linux essentials, software paradigms, system diagrams
```
---
## 🔧 Technologies Covered
- Apache Spark (Scala, Java, PySpark)
- Hadoop HDFS & Cloudera integration
- PostgreSQL, MongoDB, Cassandra
- Kafka, NiFi, Sqoop, Oozie, Hive, Impala
- Bash scripting for DevOps-style tasks
- Object-Oriented Programming and Algorithms
- Modern software engineering and system design principles
---
## 📘 Getting Started
To explore the code:
1. Clone this repository:
```bash
git clone https://github.com/leisurelyleon/mastercard-lead-data-engineer.git
cd mastercard-lead-data-engineer
2. Open it in Visual Studio Code or your preferred IDE.
3. Browse each folder independently — each has self-contained examples with inline comments or README.md files when needed.
## 🚨 Disclaimer
This repository is intended **strictly for educational and demonstration purposes only.**
No proprietary code, tooling, or architecture from Mastercard or any affiliated organization is used or reproduced here.
All configurations, scripts, schemas, and designs are **fictional and simplified** placeholders.
This repo was created to demonstrate relevant technical skills and organizational competency for potential future job roles.
⚠️ **Do not interpret any files in this repository as being sourced from Mastercard or used in production environments.**
## 🏁 License
This repository is licensed under the MIT License. See the LICENSE file for details.