https://github.com/unixjunkie/par

Execute shell commands in a parallel and/or distributed way
https://github.com/unixjunkie/par

bag-of-tasks command-line-tool distributed-computing parallel-computing parameter-sweep python-program

Last synced: 13 days ago
JSON representation

Execute shell commands in a parallel and/or distributed way

• Host: GitHub
• URL: https://github.com/unixjunkie/par
• Owner: UnixJunkie
• License: gpl-3.0
• Created: 2012-07-13T02:25:54.000Z (over 12 years ago)
• Default Branch: master
• Last Pushed: 2024-09-30T06:58:16.000Z (about 2 months ago)
• Last Synced: 2024-11-01T01:42:22.319Z (20 days ago)
• Topics: bag-of-tasks, command-line-tool, distributed-computing, parallel-computing, parameter-sweep, python-program
• Language: Python
• Homepage: https://savannah.nongnu.org/projects/par
• Size: 389 KB
• Stars: 2
• Watchers: 1
• Forks: 4
• Open Issues: 9
• Metadata Files:
  • Readme: README

Awesome Lists containing this project

README

        

If you use this software, please cite the corresponding publication:

@article{Berenger2010,

author = {Berenger, Francois and Coti, Camille and Zhang, Kam Y. J.},

title = {{PAR: A PARallel And Distributed Job Crusher}},

doi = {10.1093/bioinformatics/btq542},

year = {2010},

journal = {Bioinformatics}

}

# you need to install Pyro, cf. http://pyro.sourceforge.net

# (sudo yum install pyro _OR_ sudo aptitude install pyro)

# On the server, TCP port 7766 must be open

#                TCP port 9090 must also be open if the server is also

#                the Pyro nameserver (default case)

# if Pyro is not installed system wide, try this:

export PYTHONPATH=~/usr/Pyro-3.10/build/lib

# Some usage examples:

# run in parallel commands from test_parallel.input

cat test_parallel.input | ./parallel.py -i /dev/stdin

# same than before but with a progress bar

cat test_parallel.input | ./parallel.py -i /dev/stdin -v

# run in parallel commands from test_parallel.input and store output

# in output.log

./parallel.py -i test_parallel.input -o output.log

# same than before but with a user defined output printer

./parallel.py -i test_parallel.input -o output.log -p post_proc_example

## real world usage example

# 1) server side

./parallel.py -v -i many_commands.sh -o par_many_commands.log -s

# 2) client side, on each machine you want to join the computation

#    replace SERVER_NAME by the machine name from where you launched

#    parallel.py using -s

./parallel.py -c SERVER_NAME

# 3) be thrilled! ;)