https://github.com/crowsonkb/shared_ndarray

A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX shared memory.
https://github.com/crowsonkb/shared_ndarray

numpy python

Last synced: 2 months ago
JSON representation

A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX shared memory.

• Host: GitHub
• URL: https://github.com/crowsonkb/shared_ndarray
• Owner: crowsonkb
• License: mit
• Created: 2017-02-03T02:39:21.000Z (almost 8 years ago)
• Default Branch: master
• Last Pushed: 2017-02-23T13:56:24.000Z (almost 8 years ago)
• Last Synced: 2024-09-13T04:12:04.764Z (3 months ago)
• Topics: numpy, python
• Language: Python
• Size: 3.91 KB
• Stars: 15
• Watchers: 4
• Forks: 9
• Open Issues: 1
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

README

        
shared\_ndarray

===============

A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX

shared memory.

SharedNDArrays are designed to be sent over multiprocessing.Pipe and Queue

without serializing or transmitting the underlying ndarray or buffer. While the

associated file descriptor is closed when the SharedNDArray is garbage

collected, the underlying buffer is not released when the process ends: you

must manually call the ``unlink()`` method from the last process to use it.

Usage

-----

.. code:: python

    from __future__ import print_function

    import multiprocessing as mp

    import numpy as np

    from shared_ndarray import SharedNDArray

    try:

        shm = SharedNDArray((4, 4))

        shm.array[0, 0] = 1

        p = mp.Process(target=lambda shm: print(shm.array), args=(shm,))

        p.start()

        p.join()

    finally:

        shm.unlink()

This should print::

    [[ 1.  0.  0.  0.]

     [ 0.  0.  0.  0.]

     [ 0.  0.  0.  0.]

     [ 0.  0.  0.  0.]]

There are also convenience methods to create a new SharedNDArray from an

existing NumPy array:

.. code:: python

    arr = np.array([0, 0])

    shm1 = SharedNDArray.copy(arr)

    shm2 = SharedNDArray.zeros_like(arr)

    shm1.unlink()

    shm2.unlink()

Dependencies

------------

- `numpy `_

- `posix_ipc `_