Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Parsl/parsl

Parsl - a Python parallel scripting library
https://github.com/Parsl/parsl

hacktoberfest

Last synced: 14 days ago
JSON representation

Parsl - a Python parallel scripting library

Awesome Lists containing this project

README 

        
Parsl - Parallel Scripting Library

==================================

|licence| |docs| |NSF-1550588| |NSF-1550476| |NSF-1550562| |NSF-1550528| |CZI-EOSS|



Parsl extends parallelism in Python beyond a single computer.



You can use Parsl

`just like Python's parallel executors `_

but across *multiple cores and nodes*.

However, the real power of Parsl is in expressing multi-step workflows of functions.

Parsl lets you chain functions together and will launch each function as inputs and computing resources are available.



.. code-block:: python



    import parsl

    from parsl import python_app



    # Make functions parallel by decorating them

    @python_app

    def f(x):

        return x + 1



    @python_app

    def g(x, y):

        return x + y



    # Start Parsl on a single computer

    with parsl.load():

        # These functions now return Futures

        future = f(1)

        assert future.result() == 2



        # Functions run concurrently, can be chained

        f_a, f_b = f(2), f(3)

        future = g(f_a, f_b)

        assert future.result() == 7



Start with the `configuration quickstart `_ to learn how to tell Parsl how to use your computing resource,

then explore the `parallel computing patterns `_ to determine how to use parallelism best in your application.



.. |licence| image:: https://img.shields.io/badge/License-Apache%202.0-blue.svg

   :target: https://github.com/Parsl/parsl/blob/master/LICENSE

   :alt: Apache Licence V2.0

.. |docs| image:: https://readthedocs.org/projects/parsl/badge/?version=stable

   :target: http://parsl.readthedocs.io/en/stable/?badge=stable

   :alt: Documentation Status

.. |NSF-1550588| image:: https://img.shields.io/badge/NSF-1550588-blue.svg

   :target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550588

   :alt: NSF award info

.. |NSF-1550476| image:: https://img.shields.io/badge/NSF-1550476-blue.svg

   :target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550476

   :alt: NSF award info

.. |NSF-1550562| image:: https://img.shields.io/badge/NSF-1550562-blue.svg

   :target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550562

   :alt: NSF award info

.. |NSF-1550528| image:: https://img.shields.io/badge/NSF-1550528-blue.svg

   :target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550528

   :alt: NSF award info

.. |NSF-1550475| image:: https://img.shields.io/badge/NSF-1550475-blue.svg

   :target: https://nsf.gov/awardsearch/showAward?AWD_ID=1550475

   :alt: NSF award info

.. |CZI-EOSS| image:: https://chanzuckerberg.github.io/open-science/badges/CZI-EOSS.svg

   :target: https://czi.co/EOSS

   :alt: CZI's Essential Open Source Software for Science



   

Quickstart

==========



Install Parsl using pip::



    $ pip3 install parsl



To run the Parsl tutorial notebooks you will need to install Jupyter::



    $ pip3 install jupyter



Detailed information about setting up Jupyter with Python is available `here `_



Note: Parsl uses an opt-in model to collect usage statistics for reporting and improvement purposes. To understand what stats are collected and enable collection please refer to the `usage tracking guide `__



Documentation

=============



The complete parsl documentation is hosted `here `_.



The Parsl tutorial is hosted on live Jupyter notebooks `here `_



For Developers

--------------



1. Download Parsl::



    $ git clone https://github.com/Parsl/parsl



2. Build and Test::



    $ cd parsl # navigate to the root directory of the project

    $ make   # show all available makefile targets

    $ make virtualenv # create a virtual environment

    $ source .venv/bin/activate # activate the virtual environment

    $ make deps # install python dependencies from test-requirements.txt

    $ make test # make (all) tests. Run "make config_local_test" for a faster, smaller test set.

    $ make clean # remove virtualenv and all test and build artifacts



3. Install::



    $ cd parsl # only if you didn't enter the top-level directory in step 2 above

    $ python3 setup.py install



4. Use Parsl!



Requirements

============



Parsl is supported in Python 3.9+. Requirements can be found `here `_. Requirements for running tests can be found `here `_.



Code of Conduct

===============



Parsl seeks to foster an open and welcoming environment - Please see the `Parsl Code of Conduct `_ for more details.



Contributing

============



We welcome contributions from the community. Please see our `contributing guide `_.