https://github.com/daggaz/python-multiprocessing-utils

Python multiprocessing utilities
https://github.com/daggaz/python-multiprocessing-utils

multiprocessing python

Last synced: about 1 month ago
JSON representation

Python multiprocessing utilities

• Host: GitHub
• URL: https://github.com/daggaz/python-multiprocessing-utils
• Owner: daggaz
• License: mit
• Created: 2017-10-19T14:55:50.000Z (almost 9 years ago)
• Default Branch: master
• Last Pushed: 2017-10-20T10:49:21.000Z (almost 9 years ago)
• Last Synced: 2025-10-29T01:52:34.888Z (9 months ago)
• Topics: multiprocessing, python
• Language: Python
• Homepage: https://pypi.python.org/pypi/multiprocessing-utils
• Size: 10.7 KB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE.txt

Awesome Lists containing this project

README

          
============================

python-multiprocessing-utils

============================

Multiprocessing utilities

************

Shared locks

************

"Shared" version of the standard ``Lock()`` and ``RLock()`` classes found in

the ``multiprocessing``/``threading`` modules.

Shared locks can be acquired in two modes, shared and exclusive.

Any number of processes/threads can acquire the lock in shared mode.

Only one process/thread can acquire the lock in exclusive mode.

A process/thread attempting to exclusively acquire the lock will block

until the lock has been released by all other threads/processes.

A process/thread attempting to shared acquire the lock will only block

while there is an exclusive lock.

This is a little like database locks, which can be acquired for shared reading,

or exclusive writing.

::

    lock = multiprocessing_utils.SharedLock()

    def exclusive_worker():

        with lock.exclusive():

            # this code will only run when no other

            # process/thread in a lock context

    def shared_worker():

        with lock:

            # this code will run so long as no

            # thread/process holds an exclusive lock

***************************************

multiprocess-safe ``threading.local()``

***************************************

A process (and thread) safe version of ``threading.local()``

::

    l = multiprocessing_utils.local()

    l.x = 1

    def f():

        try:

            print(l.x)

        except Attribute:

            print("x not set")

    f()                                        # prints "1"

    threading.Thread(target=f).start()         # prints "x not set"

    multiprocessing.Process(target=f).start()  # prints "x not set"

Difference to standard ``threading.local()``

--------------------------------------------

A standard ``threading.local()`` instance created before forking (via

``os.fork()`` or ``multiprocessing.Process()``) will be "local" as expected

and the new process will have access to any data set before the fork.

Using a standard ``threading.local()`` in the example above would yield:

::

    f()                                        # prints "1"

    threading.Thread(target=f).start()         # prints "x not set"

    multiprocessing.Process(target=f).start()  # prints "1" :(