https://github.com/srvn-nm/hadoop-and-spark

https://github.com/srvn-nm/hadoop-and-spark

Last synced: 28 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/srvn-nm/hadoop-and-spark
• Owner: srvn-nm
• Created: 2024-05-19T06:37:31.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-06-08T07:22:55.000Z (8 months ago)
• Last Synced: 2024-06-08T18:15:04.331Z (8 months ago)
• Language: Jupyter Notebook
• Size: 252 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# BigData Solution for Hadoop and Spark.

### Scale your data management by distributing workload and storage on Hadoop and Spark Clusters, explore and transform your data in Jupyter Notebook.

## About The Project

Purpose for this tutorial is to show how to get started with Hadoop, Spark and Jupyter for your BigData solution, deployed as Docker Containers.

## Pre-requisite

- Apple Silicon might use arm64 branch to install.

- Ensure Docker is installed.

## Start

Execute `bash master-build.sh` to start the the build and start the containers.

## Stop

Execute `bash master-delete.sh` to stop the containers.

### Hadoop

Access Hadoop UI on ' http://localhost:9870 '

### Spark

Access Spark Master UI on ' http://localhost:8080 '

### Jupyter

Access Jupyter UI on ' http://localhost:8888 '

# Hadoop-and-spark