https://github.com/gthomas08/big-data-management-systems-project
Big Data project using Apache Spark
https://github.com/gthomas08/big-data-management-systems-project
apache-spark java jupyter-notebook pyspark python
Last synced: 9 days ago
JSON representation
Big Data project using Apache Spark
- Host: GitHub
- URL: https://github.com/gthomas08/big-data-management-systems-project
- Owner: gthomas08
- Created: 2021-05-28T08:12:45.000Z (about 4 years ago)
- Default Branch: master
- Last Pushed: 2023-01-06T11:51:12.000Z (over 2 years ago)
- Last Synced: 2025-03-14T09:21:20.306Z (4 months ago)
- Topics: apache-spark, java, jupyter-notebook, pyspark, python
- Language: Jupyter Notebook
- Homepage:
- Size: 64.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Big Data Management Systems Project
## About The Project
This project was part of the Computer Engineering and Informatics Department (CEID) of University of Patras curriculum.
## Exercise
The goal of this project was to write queries for a big [dataset](https://www.kaggle.com/grouplens/movielens-20m-dataset) and calculate the time elapsed for each query to return the results. The queries run in a local machine and a virtual one (with different configurations) that was setup by the University. Apache Spark was used to execute the queries. PySpark was used to write the queries.
## Technologies
- Java
- Python
- Pyspark
- Apache Spark
- Jupyter Notebook