https://github.com/divinenaman/mapreduce-matrix-multipy

A python implementation of matrix multiplication using Hadoop streaming API
https://github.com/divinenaman/mapreduce-matrix-multipy

hadoop hadoop-hdfs hadoop-mapreduce python

Last synced: about 2 months ago
JSON representation

A python implementation of matrix multiplication using Hadoop streaming API

• Host: GitHub
• URL: https://github.com/divinenaman/mapreduce-matrix-multipy
• Owner: divinenaman
• Created: 2021-05-30T14:11:12.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2021-05-30T14:16:28.000Z (over 3 years ago)
• Last Synced: 2024-10-30T08:23:33.367Z (3 months ago)
• Topics: hadoop, hadoop-hdfs, hadoop-mapreduce, python
• Language: Python
• Homepage:
• Size: 1.95 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
## MapReduce and Hadoop streaming API

- Matrix Multilication program on the hadoop architecture using Python

## Input format

- semi-colon(;) separated rows

- space(' ') separated columns

- next-line (\n) end of martix

- example : ```	1 2 3;5 6 7 \n

		2 3 3;0 9 0;4 5 6

	     ```

## Output format

- key: row number,column number

- value: value of element at key

	

## Run (hadoop-3.2.2)

``` 

	$ bin/hadoop jar share/hadoop/tools/lib/hadoop-streaming-3.2.2.jar \

	-file /path/to/mapper.py    -mapper /path/to/func/mapper.py \

	-file /path/to/reducer.py   -reducer /path/to/func/reducer.py \

	-input /path/to/input/* 	-output /path/to/output

````