https://github.com/mycroftai/pyache

Python numpy caching library
https://github.com/mycroftai/pyache

caching library numpy

Last synced: 2 days ago
JSON representation

Python numpy caching library

• Host: GitHub
• URL: https://github.com/mycroftai/pyache
• Owner: MycroftAI
• License: apache-2.0
• Created: 2019-03-29T11:30:09.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2020-07-31T04:32:37.000Z (over 4 years ago)
• Last Synced: 2024-10-12T16:40:57.962Z (about 1 month ago)
• Topics: caching, library, numpy
• Language: Python
• Size: 11.7 KB
• Stars: 2
• Watchers: 8
• Forks: 5
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Pyache

*Python numpy caching library*

This library caches numpy data that is generated from files and saves them in chunks to the disk.

This is useful any time a computationally expensive task is done to files to transform them into a form needed in memory.

# Usage

Create a `Pycache` object and call `load` with your filenames.

```python

import numpy as np

from time import sleep

from pyache import Pyache

def load_file(filename) -> np.ndarray:

    print('Processing {}...'.format(filename))

    sleep(0.5)

    return np.ones([100])

pyache = Pyache('.cache', load_file, 'ones-processor')

data = pyache.load(

    ['thing-1.png', 'thing-2.png', 'thing-3.png'],

    on_gen=lambda x: print('Just reprocessed', x),

    on_loop=lambda: print('Loaded one more...')

)  # Takes 1.5 seconds

# ... Run a second time (or program re-run):

data = pyache.load(

    ['thing-1.png', 'thing-2.png', 'thing-3.png']

)  # Takes 0.0 seconds

data = pyache.load(

    ['thing-1.png', 'thing-2.png', 'thing-3.png', 'thing-4.png']

)  # Takes 0.5 seconds

```

# Installation

```bash

pip install pyache

```