https://github.com/bsc-dom/npp2nvm

NumPy Persistence to Non-Volatile Memory
https://github.com/bsc-dom/npp2nvm

numpy nvm persistent-data-structure pmdk

Last synced: 3 days ago
JSON representation

NumPy Persistence to Non-Volatile Memory

• Host: GitHub
• URL: https://github.com/bsc-dom/npp2nvm
• Owner: bsc-dom
• License: apache-2.0
• Created: 2020-03-27T08:49:16.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2023-11-02T13:41:31.000Z (about 1 year ago)
• Last Synced: 2024-10-12T16:40:57.085Z (about 1 month ago)
• Topics: numpy, nvm, persistent-data-structure, pmdk
• Language: Python
• Size: 12.7 KB
• Stars: 1
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# NumPy Persistence to Non-Volatile Memory

This library will help you persist numpy array structures into NVM devices.

Internally, it uses [`pynvm`](https://github.com/pmem/pynvm) and buffers --which are

natively supported by `numpy`.

## Installation

`pip install npp2nvm`

## Requirements

- A NVM device, such as Intel(R) Optane(TM) DC Persistent Memory modules.

- Python 3.x

- numpy arrays to be stored.

## Usage

Two environment variables are used for configuring the storage:

- **NPP2NVM_SIZE** which specifies the size (in MiB) that will be assigned to

 persistence

- **NPP2NVM_PATH** which specifies the file path that will be used for persistence.

 Note that this file should be located into a _Storage Class Memory_ device, which will

 allow `pynvm` to leverage its characteristics --keep in mind that `pynvm` are bindings of the

 [PMDK](https://pmem.io/pmdk/) libraries.

## Example

```python

import numpy as np

import npp2nvm

a = np.random.random([256, 256])

a = npp2nvm.np_persist(a)

print(a.shape)

# > (256, 256)

```

## Limitations

Storage and retrieval of data structures (outside the application lifecycle)

are not implemented in this library (yet?). You can implement your own

mechanism and/or propose a PR. It will be appreciated!