https://github.com/oaarnikoivu/mapreduce
MapReduce architecture in Python
https://github.com/oaarnikoivu/mapreduce
concurrency mapreduce pydantic python rpc xmlrpc
Last synced: about 1 year ago
JSON representation
MapReduce architecture in Python
- Host: GitHub
- URL: https://github.com/oaarnikoivu/mapreduce
- Owner: oaarnikoivu
- License: mit
- Created: 2024-05-04T06:57:54.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-05T12:28:25.000Z (almost 2 years ago)
- Last Synced: 2024-05-05T21:40:38.779Z (almost 2 years ago)
- Topics: concurrency, mapreduce, pydantic, python, rpc, xmlrpc
- Language: Python
- Homepage:
- Size: 1.19 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# mapreduce
MapReduce architecture in Python
## Running the Coordinator
```bash
make coordinator ARGS="--port 8000"
```
## Running Workers
You can modify the number of workers to run in the `Makefile`. Currently it defaults to 8.
```bash
make worker ARGS="--task word_count --cport 8000"
```
where `--task` is the MapReduce task you would like to run, and `--cport` is the coordinator port.