Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/SiggiGue/hdfdict
Helps h5py to dump and load dictionaries.
https://github.com/SiggiGue/hdfdict
Last synced: 4 months ago
JSON representation
Helps h5py to dump and load dictionaries.
- Host: GitHub
- URL: https://github.com/SiggiGue/hdfdict
- Owner: SiggiGue
- License: mit
- Created: 2015-02-27T12:25:29.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2023-10-14T13:30:36.000Z (9 months ago)
- Last Synced: 2024-02-16T11:02:23.860Z (4 months ago)
- Language: Python
- Size: 28.3 KB
- Stars: 21
- Watchers: 3
- Forks: 9
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- awesome-stars - SiggiGue/hdfdict - Helps h5py to dump and load dictionaries. (Python)
README
# hdfdict helps h5py to dump and load python dictionaries
[](https://www.codefactor.io/repository/github/siggigue/hdfdict)
[](https://coveralls.io/github/SiggiGue/hdfdict?branch=master)
If you have a hierarchical data structure of numpy arrays in a dictionary for example, you can use this tool to save this dictionary into a h5py `File()` or `Group()` and load it again.
This tool just maps the hdf `Groups` to dict `keys` and the `Datset` to dict `values`.
Only types supported by h5py can be used.
The dicitonary-keys need to be strings until now.
A lazy loading option is activated per default. So big h5 files are not loaded at once. Instead a dataset gets only loaded if it is accessed from the LazyHdfDict instance.
## Example
```python
import hdfdict
import numpy as np
d = {
'a': np.random.randn(10),
'b': [1, 2, 3],
'c': 'Hallo',
'd': np.array(['a', 'b']).astype('S'),
'e': True,
'f': (True, False),
}
fname = 'test_hdfdict.h5'
hdfdict.dump(d, fname)
res = hdfdict.load(fname)
print(res)
```
Output:
`
{'a': , 'b': , 'c': , 'd': , 'e': , 'f': }
`
This are all lazy loding fields in the result `res`.
Just call `res.unlazy()` or `dict(res)` to get all fields loaded.
If you only want to load specific fields, just use item access e.g. `res['a']` so only field 'a' will be loaded from the file.
```python
print(dict(res))`
```
Output:
`
{'a': array([-0.47666824, 0.11787749, 0.51405835, -1.49557787, -0.33617182,
-0.22381693, 0.25966526, 0.58160661, 0.17019176, 1.3167669 ]), 'b': array([1, 2, 3]), 'c': 'Hallo', 'd': array([b'a', b'b'], dtype='|S1'), 'e': True, 'f': array([ True, False])}
`
## Installation
+ `pip install hdfdict`
+ `poetry install hdfdict`
+ `git clone https://github.com/SiggiGue/hdfdict.git` and `python hdfdict/setup.py install`