Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Parsl/parsl

Parsl - a Python parallel scripting library
https://github.com/Parsl/parsl

hacktoberfest

Last synced: 11 days ago
JSON representation

Parsl - a Python parallel scripting library

Awesome Lists containing this project

README

        

Parsl - Parallel Scripting Library
==================================
|licence| |docs| |NSF-1550588| |NSF-1550476| |NSF-1550562| |NSF-1550528| |CZI-EOSS|

Parsl extends parallelism in Python beyond a single computer.

You can use Parsl
`just like Python's parallel executors `_
but across *multiple cores and nodes*.
However, the real power of Parsl is in expressing multi-step workflows of functions.
Parsl lets you chain functions together and will launch each function as inputs and computing resources are available.

.. code-block:: python

import parsl
from parsl import python_app

# Make functions parallel by decorating them
@python_app
def f(x):
return x + 1

@python_app
def g(x, y):
return x + y

# Start Parsl on a single computer
with parsl.load():
# These functions now return Futures
future = f(1)
assert future.result() == 2

# Functions run concurrently, can be chained
f_a, f_b = f(2), f(3)
future = g(f_a, f_b)
assert future.result() == 7

Start with the `configuration quickstart `_ to learn how to tell Parsl how to use your computing resource,
then explore the `parallel computing patterns `_ to determine how to use parallelism best in your application.

.. |licence| image:: https://img.shields.io/badge/License-Apache%202.0-blue.svg
:target: https://github.com/Parsl/parsl/blob/master/LICENSE
:alt: Apache Licence V2.0
.. |docs| image:: https://readthedocs.org/projects/parsl/badge/?version=stable
:target: http://parsl.readthedocs.io/en/stable/?badge=stable
:alt: Documentation Status
.. |NSF-1550588| image:: https://img.shields.io/badge/NSF-1550588-blue.svg
:target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550588
:alt: NSF award info
.. |NSF-1550476| image:: https://img.shields.io/badge/NSF-1550476-blue.svg
:target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550476
:alt: NSF award info
.. |NSF-1550562| image:: https://img.shields.io/badge/NSF-1550562-blue.svg
:target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550562
:alt: NSF award info
.. |NSF-1550528| image:: https://img.shields.io/badge/NSF-1550528-blue.svg
:target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550528
:alt: NSF award info
.. |NSF-1550475| image:: https://img.shields.io/badge/NSF-1550475-blue.svg
:target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550475
:alt: NSF award info
.. |CZI-EOSS| image:: https://chanzuckerberg.github.io/open-science/badges/CZI-EOSS.svg
:target: https://czi.co/EOSS
:alt: CZI's Essential Open Source Software for Science


Quickstart
==========

Install Parsl using pip::

$ pip3 install parsl

To run the Parsl tutorial notebooks you will need to install Jupyter::

$ pip3 install jupyter

Detailed information about setting up Jupyter with Python is available `here `_

Note: Parsl uses an opt-in model to collect usage statistics for reporting and improvement purposes. To understand what stats are collected and enable collection please refer to the `usage tracking guide `__

Documentation
=============

The complete parsl documentation is hosted `here `_.

The Parsl tutorial is hosted on live Jupyter notebooks `here `_

For Developers
--------------

1. Download Parsl::

$ git clone https://github.com/Parsl/parsl

2. Build and Test::

$ cd parsl # navigate to the root directory of the project
$ make # show all available makefile targets
$ make virtualenv # create a virtual environment
$ source .venv/bin/activate # activate the virtual environment
$ make deps # install python dependencies from test-requirements.txt
$ make test # make (all) tests. Run "make config_local_test" for a faster, smaller test set.
$ make clean # remove virtualenv and all test and build artifacts

3. Install::

$ cd parsl # only if you didn't enter the top-level directory in step 2 above
$ python3 setup.py install

4. Use Parsl!

Requirements
============

Parsl is supported in Python 3.9+. Requirements can be found `here `_. Requirements for running tests can be found `here `_.

Code of Conduct
===============

Parsl seeks to foster an open and welcoming environment - Please see the `Parsl Code of Conduct `_ for more details.

Contributing
============

We welcome contributions from the community. Please see our `contributing guide `_.