Projects in Awesome Lists by Wittline

https://github.com/wittline/uber-expenses-tracking

The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such as Apache Airflow, AWS Redshift and Power BI.

airflow-docker apache-airflow aws aws-redshift data-engineering data-modeling etl-pipeline expenses-dashboard expenses-tracker power-bi python uber uber-data uber-eats

Last synced: 13 Apr 2025

https://github.com/wittline/apache-spark-docker

Dockerizing an Apache Spark Standalone Cluster

apache-spark dataengineer dataengineering docker docker-compose hadoop-cluster hadoop-docker hdfs hive hive-metastore hue pyspark

Last synced: 13 Apr 2025

https://github.com/wittline/csv-schema-inference

A tool to automatically infer columns data types in .csv files

big-data csv csv-files inference large-csv large-files parallel-programming schema-inference

Last synced: 13 Apr 2025

https://github.com/wittline/pyspark-on-aws-emr

The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on writing pyspark code.

aws aws-emr big-data big-data-analytics dataengineering ec2-spot ec2-spot-instances emr-cluster pyspark python spark wordcloud-generator

Last synced: 13 Apr 2025

https://github.com/wittline/data-engineer-challenge

Challenge Data Engineer

data-engineering data-pipeline dataengineering docker docker-compose fastapi postgresql

Last synced: 05 Sep 2025

https://github.com/wittline/pydag

Scheduling Big Data Workloads and Data Pipelines in the Cloud with pyDag

big-data bigquery cloud dag data-engineering data-pipeline dataengineering dataproc dataproc-cluster directed-acyclic-graph google-cloud google-cloud-platform parallel-processing task-scheduler task-scheduling workflow-engine

Last synced: 13 Apr 2025

https://github.com/wittline/dropout-students-prediction

The goal of this project is to identify students at risk of dropping out the school

genetic-algorithm k-means k-means-clustering neural-networks r

Last synced: 23 Aug 2025

https://github.com/wittline/data-engineering-challenge-th

Dockerizing a Python Script for Web Scraping and consume the scraped data using FastApi (www.metroscubicos.com)

data-engineering docker fastapi python scraping sqlite

Last synced: 13 Apr 2025

https://github.com/wittline/d3js-dashboard

Building Responsive DashBoard with D3.js and ASP.NET MVC from scratch (SQL SERVER - SSIS - API REST)

api-rest asp-net-mvc d3-visualization d3js datavisualization javascript mvc-pattern responsive-design sqlserver ssis

Last synced: 13 Jul 2025

https://github.com/wittline/tf-idf

Term Frequency-Inverse Document Frequency from Scratch

feature-engineering python text-analytics tfidf

Last synced: 13 Apr 2025

https://github.com/wittline/wbz

A parallel implementation of the bzip2 data compressor in python, this data compression pipeline is using algorithms like Burrows–Wheeler transform (BWT) and Move to front (MTF) to improve the Huffman compression. For now, this tool only will be focused on compressing .csv files, and other files on tabular format.

big-data bigdata burrows-wheeler-transform csv csv-files data-compression data-engineering huffman-coding huffman-compression-algorithm move-to-front tabular-data

Last synced: 24 Aug 2025

https://github.com/wittline/recommendation-system

Build a Content-Based Movie Recommender System (TF-IDF, BM25, BERT)

bert bm25 nlp python recommender-system recsys text-analysis tf-idf word2vec

Last synced: 03 Jul 2025

https://github.com/wittline/docker-livy

Dockerizing and Consuming an Apache Livy environment

apache-livy apache-spark big-data docker docker-compose dockerized dockerizing postgresql

Last synced: 24 Aug 2025

https://github.com/wittline/text-analysis-speeches-amlo

Text analysis of the speeches, conferences and interviews of the current president of Mexico

mexican mexico mexico-datos mexico-estados nlp political-science politics president presidential-candidates python speech speech-synthesis text-analysis word-embeddings

Last synced: 13 Apr 2025

https://github.com/wittline/dataengineering-assignment

Prescreening Tasks for Data Engineer

dataengineering docker jupyter-notebook postgresql

Last synced: 13 Apr 2025

https://github.com/wittline/huffman-decoding

A New Approach for Efficient Sequential Decoding of Static Huffman Codes

burrows-wheeler-transform compression data-compression huffman-coding huffman-decoder move-to-front

Last synced: 06 Jan 2026

https://github.com/wittline/moving-average-spark

How to Compute Moving Average with Spark

databricks hadoop moving-average spark

Last synced: 13 Apr 2025

https://github.com/wittline/distance-metrics

Distance metrics are one of the most important parts of some machine learning algorithms, supervised and unsupervised learning, it will help us to calculate and measure similarities between numerical values expressed as data points

distance-metrics machine-learning python similarity-measures

Last synced: 17 Oct 2025

https://github.com/wittline/csv-shuffler

A tool to automatically Shuffle lines in .csv files

big-data csv csv-files data-engineering large-files shuffle

Last synced: 14 Oct 2025

https://github.com/wittline/csv-estimate-rows

Last synced: 14 Feb 2026

https://github.com/wittline/machinelearning

The repository contains basic experiments using machine learning algorithms with python

machine-learning machine-learning-algorithms python

Last synced: 24 Mar 2025

https://github.com/wittline/restful-apis-nodejs

Building fast, scalable and secure RESTful services with Node, Express and MongoDB

coverage-report express-js mocha-chai mongodb mongoose nodejs rest-api unittest

Last synced: 07 Jul 2025

https://github.com/wittline/livyc

Apache Spark as a Service with Apache Livy Client

apache-livy apache-spark big-data data-engineering dataengineering docker livy-client livy-docker pyhton spark

Last synced: 19 Jun 2026

https://github.com/wittline/optimizing-public-transportation

Streaming event pipeline around Apache Kafka and its ecosystem. Using public data from the Chicago Transit Authority we will construct an event pipeline around Kafka that allows us to simulate and display the status of train lines in real time.

kafka stream-processing streaming udacity-nanodegree

Last synced: 24 Mar 2025

https://github.com/wittline/wittline

Take a look at my repository

Last synced: 24 Feb 2026

https://github.com/wittline/contextual-data-transforms

This repository contain the most important contextual data transformation algorithms which help to improve the rate compression reached by statistical encoders. Ramses Alexander Coraspe Valdez

burrows-wheeler-transform delta-encoding move-to-front run-length-encoding xor-encoder

Last synced: 18 Mar 2026

https://github.com/wittline/gpu-programming-with-python

GPU programming with Python, you can take advantage of the incredible computing power of your graphics processing unit GPU. we will work with NVIDIA’s CUDA library.

Last synced: 18 Mar 2026

https://github.com/wittline/computer-vision-and-deep-learning

This repository contains information on the basic techniques and algorithms used in computer image processing, in addition to some projects related to pattern recognition using deep learning.

computer-vision deep-learning image-processing

Last synced: 19 Jul 2025