https://github.com/daleroberts/pypar

Efficient and scalable parallelism using the message passing interface (MPI) to handle big data and highly computational problems.
https://github.com/daleroberts/pypar

big-data map-reduce mpi python

Last synced: 2 months ago
JSON representation

Efficient and scalable parallelism using the message passing interface (MPI) to handle big data and highly computational problems.

• Host: GitHub
• URL: https://github.com/daleroberts/pypar
• Owner: daleroberts
• License: gpl-3.0
• Created: 2013-05-21T00:41:23.000Z (about 12 years ago)
• Default Branch: master
• Last Pushed: 2016-11-11T00:51:58.000Z (over 8 years ago)
• Last Synced: 2025-03-26T23:04:25.967Z (3 months ago)
• Topics: big-data, map-reduce, mpi, python
• Language: Python
• Homepage:
• Size: 604 KB
• Stars: 69
• Watchers: 14
• Forks: 16
• Open Issues: 5
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
[![Logo](https://raw.githubusercontent.com/daleroberts/pypar/master/doc/logo.png)](https://raw.githubusercontent.com/daleroberts/pypar/master/doc/logo.png)

**PyPar** is a python library that provides efficient and scalable parallelism using the message passing interface (MPI) to handle **big data** and **highly computational problems**.

[![Build Status](https://travis-ci.org/daleroberts/pypar.svg?branch=master)](https://travis-ci.org/daleroberts/pypar)

**PyPar** is used by a number of large projects, such as:

 - [ANUGA shallow water equation solver](https://github.com/GeoscienceAustralia/anuga_core)

 - [TCRM A statistical-parametric model for assessing wind hazard from tropical cyclones](https://github.com/GeoscienceAustralia/tcrm)

 - [Wind multipliers: for produce wind terrain, shielding and topographic multipliers](https://github.com/GeoscienceAustralia/Wind_multipliers)

## Example

A simple 'pass the parcel' example.

```python

import pypar as pp

ncpus = pp.size()

rank = pp.rank()

node = pp.get_processor_name()

print 'I am rank %d of %d on node %s' % (rank, ncpus, node)

if rank == 0:

  msg = 'P0'

  pp.send(msg, destination=1)

  msg = pp.receive(source=rank-1)

  print 'Processor 0 received message "%s" from rank %d' % (msg, rank-1)

else:

  source = rank-1

  destination = (rank+1) % ncpus

  msg = pp.receive(source)

  msg = msg + 'P' + str(rank)

  pp.send(msg, destination)

pp.finalize()

```