https://github.com/d1618033/listful
Efficient filtering of lists of objects
https://github.com/d1618033/listful
database dict filter list pandas python python3 query search
Last synced: about 2 months ago
JSON representation
Efficient filtering of lists of objects
- Host: GitHub
- URL: https://github.com/d1618033/listful
- Owner: d1618033
- License: mit
- Created: 2020-02-11T17:53:33.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2021-01-17T15:48:39.000Z (over 5 years ago)
- Last Synced: 2025-09-10T10:38:22.044Z (10 months ago)
- Topics: database, dict, filter, list, pandas, python, python3, query, search
- Language: Python
- Size: 64.5 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# listful
[](https://pypi.org/project/listful)
[](https://pypi.org/project/listful)
[](https://pypistats.org/packages/listful)
[](https://travis-ci.org/d1618033/listful)
[](https://codecov.io/gh/d1618033/listful)
[](https://en.wikipedia.org/wiki/MIT_License)
[](https://github.com/ambv/black)
## Description
Efficient filtering of lists of objects
## Installation
pip install listful
## Usage
Initialize with the fields you want to filter by:
>>> from listful import Listful
>>> data = Listful(
... [{'x': 1, 'y': 10}, {'x': 2, 'y': 20}, {'x': 2, 'y': 30}],
... fields=['x', 'y']
... )
(If you don't specify the fields, all the fields whose corresponding values are hashable will be chosen)
### Filtering:
* By one field:
>>> data.filter(x=1).one_or_none()
{'x': 1, 'y': 10}
>>> data.filter(y=20).one_or_none()
{'x': 2, 'y': 20}
* By one field, with more than one result:
>>> data.filter(x=2).to_list()
[{'x': 2, 'y': 20}, {'x': 2, 'y': 30}]
* By two fields:
>>> data.filter(x=2, y=30).one_or_none()
{'x': 2, 'y': 30}
* Raise exception if more than one found
>>> data.filter(x=2).one_or_raise()
Traceback (most recent call last):
<...>
listful.exceptions.MoreThanOneResultException: Found more than one result for filter {'x': 2}: [{'x': 2, 'y': 20}, {'x': 2, 'y': 30}]
* Get all values for a specific field
>>> data.get_all_for_field('x')
[1, 2, 2]
### Updating indexes:
`Listful` has the same api as `list`, so you can get/set/delete items the same way
and the indices will be updated automatically
>>> data[0] = {'x': 17, 'y': 17}
>>> data.filter(x=17).one_or_none()
{'x': 17, 'y': 17}
>>> data[0]
{'x': 17, 'y': 17}
>>> del data[0]
>>> data.filter(x=17).one_or_none()
If you want to modify an element and update the indices you can do so explicitly:
>>> data[0]['x'] = 1
>>> data.rebuild_indexes_for_item(data[0])
>>> data.filter(x=1).one_or_none()
{'x': 1, 'y': 20}
### Objects:
Listful supports also lists of objects:
>>> class Item:
... def __init__(self, x, y):
... self.x = x
... self.y = y
...
... def __repr__(self):
... return f"Item(x={self.x}, y={self.y})"
>>> items = Listful(
... [Item(x=1, y=10), Item(x=2, y=20), Item(x=2, y=30)],
... fields=['x', 'y']
... )
>>> items.filter(x=1).one_or_none()
Item(x=1, y=10)
Here too, if you don't specify the fields, all fields with hashable values will be chosen:
>>> items = Listful(
... [Item(x=1, y=10), Item(x=2, y=20), Item(x=2, y=30)],
... )
>>> items.fields
['x', 'y']
## Performance
See `scripts/timing.py`.
A comparison of filtering with listful vs filtering with pandas (with/without index)
| | listful | pandas | pandas_with_index |
| --- | --- | --- | --- |
| init | 7.63e-02 | 3.03e-01 | 5.24e-02 |
| filter:1 | 2.07e-05 | 1.46e-03 | 1.79e-03 |
| filter:n | 2.02e-01 | 7.40e+01 | 1.54e+01 |
70x faster than pandas with indexing, 360x faster than pandas without indexing.
## For developers
### Create venv and install deps
make init
### Install git precommit hook
make precommit_install
### Run linters, autoformat, tests etc.
make pretty lint test
### Bump new version
make bump_major
make bump_minor
make bump_patch
## License
MIT
## Change Log
Unreleased
-----
* ...
0.3.0 - 2021-01-17
-----
* ...
0.2.1 - 2020-04-08
-----
* ...
0.2.0 - 2020-04-08
-----
* Add support for default fields
0.1.3 - 2020-02-14
-----
* ...
0.1.1 - 2020-02-12
-----
* ...
0.1.0 - 2020-02-12
-----
* initial