Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/joblib/joblib

Computing with Python functions.
https://github.com/joblib/joblib

caching memoization multiprocessing parallel-computing python threading

Last synced: about 2 months ago
JSON representation

Computing with Python functions.

Awesome Lists containing this project

README 

        
|PyPi| |Azure| |ReadTheDocs| |Codecov| 



.. |PyPi| image:: https://badge.fury.io/py/joblib.svg

   :target: https://badge.fury.io/py/joblib

   :alt: Joblib version



.. |Azure| image:: https://dev.azure.com/joblib/joblib/_apis/build/status/joblib.joblib?branchName=main

   :target: https://dev.azure.com/joblib/joblib/_build?definitionId=3&_a=summary&branchFilter=40

   :alt: Azure CI status



.. |ReadTheDocs| image:: https://readthedocs.org/projects/joblib/badge/?version=latest

    :target: https://joblib.readthedocs.io/en/latest/?badge=latest

    :alt: Documentation Status



.. |Codecov| image:: https://codecov.io/gh/joblib/joblib/branch/main/graph/badge.svg

   :target: https://codecov.io/gh/joblib/joblib

   :alt: Codecov coverage



The homepage of joblib with user documentation is located on:



https://joblib.readthedocs.io



Getting the latest code

=======================



To get the latest code using git, simply type::



    git clone https://github.com/joblib/joblib.git



If you don't have git installed, you can download a zip

of the latest code: https://github.com/joblib/joblib/archive/refs/heads/main.zip



Installing

==========



You can use `pip` to install joblib::



    pip install joblib



from any directory or::



    python setup.py install



from the source directory.



Dependencies

============



- Joblib has no mandatory dependencies besides Python (supported versions are

  3.8+).

- Joblib has an optional dependency on Numpy (at least version 1.6.1) for array

  manipulation.

- Joblib includes its own vendored copy of

  `loky `_ for process management.

- Joblib can efficiently dump and load numpy arrays but does not require numpy

  to be installed.

- Joblib has an optional dependency on

  `python-lz4 `_ as a faster alternative to

  zlib and gzip for compressed serialization.

- Joblib has an optional dependency on psutil to mitigate memory leaks in

  parallel worker processes.

- Some examples require external dependencies such as pandas. See the

  instructions in the `Building the docs`_ section for details.



Workflow to contribute

======================



To contribute to joblib, first create an account on `github

`_. Once this is done, fork the `joblib repository

`_ to have your own repository,

clone it using 'git clone' on the computers where you want to work. Make

your changes in your clone, push them to your github account, test them

on several computers, and when you are happy with them, send a pull

request to the main repository.



Running the test suite

======================



To run the test suite, you need the pytest (version >= 3) and coverage modules.

Run the test suite using::



    pytest joblib



from the root of the project.



Building the docs

=================



To build the docs you need to have sphinx (>=1.4) and some dependencies

installed::



    pip install -U -r .readthedocs-requirements.txt



The docs can then be built with the following command::



    make doc



The html docs are located in the ``doc/_build/html`` directory.



Making a source tarball

=======================



To create a source tarball, eg for packaging or distributing, run the

following command::



    python setup.py sdist



The tarball will be created in the `dist` directory. This command will

compile the docs, and the resulting tarball can be installed with

no extra dependencies than the Python standard library. You will need

setuptool and sphinx.



Making a release and uploading it to PyPI

=========================================



This command is only run by project manager, to make a release, and

upload in to PyPI::



    python setup.py sdist bdist_wheel

    twine upload dist/*



Note that the documentation should automatically get updated at each git

push. If that is not the case, try building th doc locally and resolve

any doc build error (in particular when running the examples).



Updating the changelog

======================



Changes are listed in the CHANGES.rst file. They must be manually updated

but, the following git command may be used to generate the lines::



    git log --abbrev-commit --date=short --no-merges --sparse