https://github.com/rossant/playdoh
UNMAINTAINED - USE AT YOUR OWN RISKS
https://github.com/rossant/playdoh
Last synced: 10 months ago
JSON representation
UNMAINTAINED - USE AT YOUR OWN RISKS
- Host: GitHub
- URL: https://github.com/rossant/playdoh
- Owner: rossant
- Created: 2013-01-28T21:23:29.000Z (almost 13 years ago)
- Default Branch: master
- Last Pushed: 2013-03-16T11:25:09.000Z (almost 13 years ago)
- Last Synced: 2024-10-19T22:51:42.842Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 603 KB
- Stars: 66
- Watchers: 6
- Forks: 14
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Playdoh: pure Python library for distributed computing and optimization
=======================================================================
Playdoh is a pure Python library for distributing computations across the
free computing units (CPUs and GPUs) available in a small network of
multicore computers. Playdoh supports independent (embarassingly) parallel
problems as well as loosely coupled tasks such as global optimizations,
Monte Carlo simulations and numerical integration of partial differential
equations. It is designed to be *lightweight* and *easy-to-use* and should
be of interest to scientists wanting to turn their lab computers into a
small cluster at no cost.
## Features
* **Standalone, lightweight, easy-to-use pure Python library** for
small-scale distributed scientific computing
* **Parallel/distributed version of `map`**: call any Python function
with different parameters in parallel over different CPUs/computers
interconnected within a standard Ethernet network
import playdoh
result = playdoh.map(lambda x: x * x, [1, 2], cpu=2) # result == [1, 4]
* **Built-in support for GPU through PyCUDA**: if you provide the
PyCUDA/CUDA code, Playdoh can run it on several GPUs (on one or more
computers) in parallel
* **Built-in distributed optimization toolbox**: you provide an
objective Python function, and Playdoh will minimize or maximize it in
parallel over several CPUs/computers (or GPUs if you give the
PyCUDA/CUDA code) using a particle-based gradient-free optimization
algorithm (CMA-ES, Genetic Algorithms or Particle Swarm Optimization
algorithm)
* **Simple coarse-grained distributed computing interface**: you can
distribute your computations into loosely-coupled tasks and run them in
parallel over CPUs and computers, inter-node communication happening
through blocking FIFO tubes exposing a straightforward push/pull
interface (examples include distributed PDE solvers, cellular automata,
Monte Carlo simulators, optimizations...)
* **Code handling**: Playdoh handles the transport of your Python code
between machines
* **Resources sharing**: specify how many CPUs you dedicate to others'
computations and how many you keep for yourself
## Documentation
[The documentation can be found here](https://playdoh-python.readthedocs.org/en/latest/index.html).
## Paper
Rossant C, Fontaine B, Goodman DFM (2011).
[**Playdoh: a lightweight Python package for distributed computing and optimisation**](http://www.sciencedirect.com/science/article/pii/S1877750311000561). *Journal of Computational Science*
### Abstract
Parallel computing is now an essential paradigm for high performance
scientific computing. Most existing hardware and software solutions are
expensive or difficult to use. We developed Playdoh, a Python library for
distributing computations across the free computing units available in a
small network of multicore computers. Playdoh supports independent and
loosely coupled parallel problems such as global optimisations, Monte
Carlo simulations and numerical integration of partial differential
equations. It is designed to be lightweight and easy to use and should be
of interest to scientists wanting to turn their lab computers into a small
cluster at no cost.
## Contribute
Playdoh is an open-source project and anyone is welcome to contribute to
the project. Here are some info about the source code.
* There's a unit testing suite in the `test` directory, it can be
launched with the `test.py` script.
* A document explaining how to add new features to Playdoh (like a new
optimization algorithm) will be available soon.
* A document explaining the internal implementation details of Playdoh
will be available soon, it will be the starting point if you want to
contribute to the code.
* You can submit issues, ideas, comments on the
[Playdoh Google group](http://groups.google.com/group/playdoh-library).