https://github.com/sumonta056/hadoop-clustering-docker-guide
Hadoop-Clustering-Docker-Guide : A Complete Documentation to setting up Hadoop and try clustering.
https://github.com/sumonta056/hadoop-clustering-docker-guide
clustering docker hadoop mapreduce-java
Last synced: 5 months ago
JSON representation
Hadoop-Clustering-Docker-Guide : A Complete Documentation to setting up Hadoop and try clustering.
- Host: GitHub
- URL: https://github.com/sumonta056/hadoop-clustering-docker-guide
- Owner: Sumonta056
- Created: 2024-01-12T03:40:01.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-01-23T15:36:31.000Z (almost 2 years ago)
- Last Synced: 2024-01-28T00:09:31.238Z (almost 2 years ago)
- Topics: clustering, docker, hadoop, mapreduce-java
- Language: Java
- Homepage:
- Size: 51.5 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
*------ **🧠 For Lab Exam : This step are enough 🧠** ------*
# 🌲 Hadoop Clusterting using Docker
#### ✍️ Follow Every Blog Instruction of [মেহরাজুল ইসলাম](https://github.com/codermehraj) : **([👉Click Here](https://medium.com/@genius_meringue_gnat_79/running-single-node-mapreduce-using-docker-and-hadoop-cluster-41f5ec5b7650))**
#### ✍️ To Run Hadoop Docker using Your Project Jar File : **([👉Click Here](./CMD.md))**
#### ✍️ 19 Lab Final Question + Answer : **([👉Click Here](./Lab%20Final/))**
*------ **🎥 Video Tutorial of Blog : [Rifat Shariar Sakil](https://github.com/Rifat-Shariar-Sakil-24) 🎥** ------*
##### ✍️ Video of Blog Steps 1 - 5 : **([👉Click Here](https://drive.google.com/file/d/1iZnPTpOrcLlHRYkJSsV6CM2Q6uVYBkd-/view))**
##### ✍️ Video of Blog Steps 6 - 10 : **([👉Click Here](https://drive.google.com/file/d/10Kw1I6i2ZpSQk_zM7vces1um3EswGm5z/view))**
##### ✍️ Video of Blog Steps 11 - 13 : **([👉Click Here](https://drive.google.com/file/d/1Bf86YzzApHFwgcdfMCncI0uAsLtskO_Z/view))**
*-- **🧠 Do The Below Steps if only want to see how hadoop works 🧠** --*
# 🎥 Tutorial Installation of Hadoop
#### 🧠 Follow Every Instruction Carefully **([👉Click Here](https://youtu.be/knAS0w-jiUk?si=DffG6Zoubzt2Dad3))**
## ✅ Download Links
#### 🧠 Java - 8 (Must Needed) : **([👉Click Here](https://www.oracle.com/java/technologies/javase/javase8-archive-downloads.html#license-lightbox))**
#### 🧠 Apache Hadoop : **([👉Click Here](https://hadoop.apache.org/releases.html))**
#### 🧠 Notepad++ Editor : **([👉Click Here](https://notepad-plus-plus.org/downloads/))**
#### 🧠 Hadoop Fixed Bin Folder : **([👉Click Here](https://drive.google.com/file/d/1nCN_jK7EJF2DmPUUxgOggnvJ6k6tksYz/view))**
#### 🧠 Hadoop All Version Bin Folder Fixed : **([👉Click Here](https://github.com/kontext-tech/winutils))**
#### 🧠 Microsoft Visual C++ Redistributable : **([👉Click Here](https://drive.google.com/file/d/1nCN_jK7EJF2DmPUUxgOggnvJ6k6tksYz/view))**
## 👩💻 Code : Setting up Hadoop **([👉Click Here](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170))**
#### **👉 For core-site.xml**
```
fs.defaultFS
hdfs://localhost:9000
```
#### **👉 For mapred-site.xml**
```
mapreduce.framework.name
yarn
```
#### **👉 For hdfs-site.xml or https-site.xml**
```
dfs.replication
1
dfs.namenode.name.dir
C:\hadoop\data\namenode
dfs.datanode.data.dir
C:\hadoop\data\datanode
```
#### **👉 For yarn-site.xml**
```
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.auxservices.mapreduce.shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
```
## ✅ Testing : Haadoop Properly Installed or Not
### **Method - 1 : 👉 Opem CMD Terminal**
```
hdfs namenode -format
```
```
cd \
```
```
cd hadoop
```
```
cd sbin
```
```
start-dfs.cmd
```
```
cd jps
```
```
start-yarn.cmd
```
```
cd jps
```
### **Method - 2 : 👉 Download The Folder and Run The Project in Intellij**
##### 🧠 Video Link of Class **([👉Click Here](https://drive.google.com/file/d/1VMzOIztVlbw-2wOt8s-h7RB8AC0ld7th/view))**
##### 🧠 Project Code **([👉Click Here](https://github.com/CosmicBeing09/Hadoop-Demo-Word-Count-with-Java-and-Maven))**
# 🎥 Docker Installation
### 🧠 Install Docker on Window 11 **([👉Click Here](https://youtu.be/WDEdRmTCSs8?si=8gRvkju3qT8o3mox))**
##### 🧠 Docker Guide and Basic Information **([👉Click Here](https://youtu.be/31k6AtW-b3Y?si=ftTXqU15KWkAtKyP))**