https://github.com/ionelmc/python-mongoql-conv
Library to convert those MongoDB queries to something else, like a python expresion, a function or a django query (Q) object tree
https://github.com/ionelmc/python-mongoql-conv
Last synced: 6 months ago
JSON representation
Library to convert those MongoDB queries to something else, like a python expresion, a function or a django query (Q) object tree
- Host: GitHub
- URL: https://github.com/ionelmc/python-mongoql-conv
- Owner: ionelmc
- License: other
- Created: 2014-01-09T21:38:07.000Z (over 11 years ago)
- Default Branch: master
- Last Pushed: 2017-07-20T21:57:00.000Z (about 8 years ago)
- Last Synced: 2025-03-25T06:41:37.494Z (7 months ago)
- Language: Python
- Size: 120 KB
- Stars: 7
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.rst
- Changelog: CHANGELOG.rst
- Contributing: CONTRIBUTING.rst
- License: LICENSE
Awesome Lists containing this project
README
========
Overview
========
.. start-badges
.. list-table::
:stub-columns: 1
* - docs
- |docs|
* - tests
- | |travis| |appveyor| |requires|
| |coveralls| |codecov|
* - package
- | |version| |wheel| |supported-versions| |supported-implementations|
| |commits-since|
.. |docs| image:: https://readthedocs.org/projects/python-mongoql-conv/badge/?style=flat
:target: https://readthedocs.org/projects/python-mongoql-conv
:alt: Documentation Status
.. |travis| image:: https://travis-ci.org/ionelmc/python-mongoql-conv.svg?branch=master
:alt: Travis-CI Build Status
:target: https://travis-ci.org/ionelmc/python-mongoql-conv
.. |appveyor| image:: https://ci.appveyor.com/api/projects/status/github/ionelmc/python-mongoql-conv?branch=master&svg=true
:alt: AppVeyor Build Status
:target: https://ci.appveyor.com/project/ionelmc/python-mongoql-conv
.. |requires| image:: https://requires.io/github/ionelmc/python-mongoql-conv/requirements.svg?branch=master
:alt: Requirements Status
:target: https://requires.io/github/ionelmc/python-mongoql-conv/requirements/?branch=master
.. |coveralls| image:: https://coveralls.io/repos/ionelmc/python-mongoql-conv/badge.svg?branch=master&service=github
:alt: Coverage Status
:target: https://coveralls.io/r/ionelmc/python-mongoql-conv
.. |codecov| image:: https://codecov.io/github/ionelmc/python-mongoql-conv/coverage.svg?branch=master
:alt: Coverage Status
:target: https://codecov.io/github/ionelmc/python-mongoql-conv
.. |version| image:: https://img.shields.io/pypi/v/mongoql-conv.svg
:alt: PyPI Package latest release
:target: https://pypi.python.org/pypi/mongoql-conv
.. |commits-since| image:: https://img.shields.io/github/commits-since/ionelmc/python-mongoql-conv/v0.4.1.svg
:alt: Commits since latest release
:target: https://github.com/ionelmc/python-mongoql-conv/compare/v0.4.1...master
.. |wheel| image:: https://img.shields.io/pypi/wheel/mongoql-conv.svg
:alt: PyPI Wheel
:target: https://pypi.python.org/pypi/mongoql-conv
.. |supported-versions| image:: https://img.shields.io/pypi/pyversions/mongoql-conv.svg
:alt: Supported versions
:target: https://pypi.python.org/pypi/mongoql-conv
.. |supported-implementations| image:: https://img.shields.io/pypi/implementation/mongoql-conv.svg
:alt: Supported implementations
:target: https://pypi.python.org/pypi/mongoql-conv
.. end-badges
Library to convert those MongoDB queries to something else, like a python expresion, a function or a Django Q object
tree to be used with a ORM query.
For now, only supports flat operations. No subdocuments. It might work but results are undefined/buggy. *Could be fixed
though ...*
* Free software: BSD license
Installation
============
::
pip install mongoql-conv
Or::
pip install mongoql-conv[django]
API
===
* ``mongoql_conv.to_string``: to_string_
* ``mongoql_conv.to_func``: to_func_
* ``mongoql_conv.django.to_Q``: to_Q_
to_string
=========
::
>>> from mongoql_conv import to_string
>>> to_string({"myfield": 1})
"row['myfield'] == 1"
>>> to_string({})
'True'
>>> set(to_string({"field1": 1, "field2": 2}).split(' and ')) == {"(row['field2'] == 2)", "(row['field1'] == 1)"}
True
>>> to_string({"myfield": 1}, object_name='item')
"item['myfield'] == 1"
>>> to_string({"myfield": {"$in": [1, 2]}})
"row['myfield'] in {1, 2}"
>>> to_string({"myfield": {"$in": {1: 2}}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
>>> to_string({"myfield": {"$and": []}})
'True'
to_string: Supported operators
------------------------------
to_string: Supported operators: Arithmetic
``````````````````````````````````````````
* **$gt**::
>>> to_string({"myfield": {"$gt": 1}})
"row['myfield'] > 1"
>>> to_string({"myfield": {"$gt": [1]}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part [1]. Expected one of: int, ...float, str, ...bool, None.
* **$gte**::
>>> to_string({"myfield": {"$gte": 1}})
"row['myfield'] >= 1"
* **$lt**::
>>> to_string({"myfield": {"$lt": 1}})
"row['myfield'] < 1"
* **$lte**::
>>> to_string({"myfield": {"$lte": 1}})
"row['myfield'] <= 1"
* **$eq**::
>>> to_string({"myfield": {"$eq": 1}})
"row['myfield'] == 1"
>>> to_string({"myfield": 1})
"row['myfield'] == 1"
* **$ne**::
>>> to_string({"myfield": {"$ne": 1}})
"row['myfield'] != 1"
* **$mod**::
>>> to_string({"myfield": {"$mod": [2, 1]}})
"row['myfield'] % 2 == 1"
>>> to_string({"myfield": {"$mod": [2, 1, 3]}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
>>> to_string({"myfield": {"$mod": 2}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
>>> to_string({"myfield": {"$mod": (2, 1)}})
"row['myfield'] % 2 == 1"
to_string: Supported operators: Containers
``````````````````````````````````````````
* **$in**::
>>> to_string({"myfield": {"$in": (1, 2, 3)}})
"row['myfield'] in {1, 2, 3}"
* **$nin**::
>>> to_string({"myfield": {"$nin": [1, 2, 3]}})
"row['myfield'] not in {1, 2, 3}"
>>> to_string({"myfield": {"$nin": {1: 2}}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
* **$size**::
>>> to_string({"myfield": {"$size": 3}})
"len(row['myfield']) == 3"
>>> to_string({"myfield": {"$size": "3"}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part '3'. Expected one of: int...
* **$all**::
>>> to_string({"myfield": {"$all": [1, 2, 3]}})
"set(row['myfield']) >= {1, 2, 3}"
>>> to_string({"myfield": {"$all": 1}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
* **$exists**::
>>> to_string({"myfield": {"$exists": True}})
"'myfield' in row"
>>> to_string({"myfield": {"$exists": False}})
"'myfield' not in row"
to_string: Supported operators: Boolean operators
`````````````````````````````````````````````````
* **$or**::
>>> to_string({'$or': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]})
"(row['bubu'] > 1) or (row['bubu'] < 2)"
>>> to_string({'$or': "invalid value"})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
* **$and**::
>>> to_string({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]})
"(row['bubu'] > 1) and (row['bubu'] < 2)"
>>> to_string({'$or': "invalid value"})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
* **$*nesting***::
>>> to_string({'$and': [
... {"bubu": {"$gt": 1}},
... {'$or': [
... {'bubu': {'$lt': 2}},
... {'$and': [
... {'bubu': {'$lt': 3}},
... {'bubu': {'$lt': 4}},
... ]}
... ]}
... ]})
"(row['bubu'] > 1) and ((row['bubu'] < 2) or ((row['bubu'] < 3) and (row['bubu'] < 4)))"
to_string: Supported operators: Regular expressions
```````````````````````````````````````````````````
* **$regex**::
>>> to_string({"myfield": {"$regex": 'a'}})
"re.search('a', row['myfield'], 0)"
>>> to_string({"bubu": {"$regex": ".*x"}}, object_name='X')
"re.search('.*x', X['bubu'], 0)"
>>> to_string({"myfield": {"$regex": 'a', "$options": 'i'}})
"re.search('a', row['myfield'], ...2...)"
>>> closure = {}
>>> to_string({"bubu": {"$regex": ".*x"}}, closure=closure), closure
("var0.search(row['bubu'])", {'var0': "re.compile('.*x', 0)"})
>>> to_string({"myfield": {"$regex": 'junk('}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid regular expression 'junk(':...
>>> to_string({"myfield": {"$regex": 'a', 'junk': 'junk'}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
>>> set(to_string({"myfield": {"$regex": 'a', '$nin': ['aaa']}}).split(' and ')) == {
... "(re.search('a', row['myfield'], 0))",
... "(row['myfield'] not in {'aaa'})"
... }
True
>>> to_string({"bubu": {"$regex": ".*", "$options": "junk"}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !
>>> to_string({"bubu": {"$options": "i"}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
to_string (lax mode)
====================
to_string (lax mode): Supported operators: Containers
-----------------------------------------------------
* **$all**::
>>> to_string({"myfield": {"$all": [1, 2, 3]}}, lax=True)
"set(row.get('myfield', LaxNone)) >= {1, 2, 3}"
>>> to_string({"myfield": {"$all": 1}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
to_func
=======
::
>>> from mongoql_conv import to_func
>>> to_func({"myfield": 1}).source
"lambda item: (item['myfield'] == 1) # compiled from {'myfield': 1}"
>>> to_func({}).source
'lambda item: (True) # compiled from {}'
>>> list(filter(to_func({"myfield": 1}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}]
>>> list(filter(to_func({}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]
>>> to_func({"myfield": {"$in": [1, 2]}}).source
"lambda item, var0={1, 2}: (item['myfield'] in var0) # compiled from {'myfield': {'$in': [1, 2]}}"
>>> list(filter(to_func({"myfield": {"$in": [1, 2]}}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]
>>> to_func({"myfield": {"$in": {1: 2}}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
>>> to_func({"myfield": {"$and": []}}).source
"lambda item: (True) # compiled from {'myfield': {'$and': []}}"
>>> list(filter(to_func({"myfield": {"$and": []}}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]
to_func: Supported operators
----------------------------
to_func: Supported operators: Arithmetic
````````````````````````````````````````
* **$gt**::
>>> to_func({"myfield": {"$gt": 1}}).source
"lambda item: (item['myfield'] > 1) # compiled from {'myfield': {'$gt': 1}}"
>>> to_func({"myfield": {"$gt": [1]}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part [1]. Expected one of: int,...float, str,...bool, None.
>>> list(filter(to_func({"myfield": {"$gt": 1}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 2}, {'myfield': 3}, {'myfield': 4}]
* **$gte**::
>>> to_func({"myfield": {"$gte": 1}}).source
"lambda item: (item['myfield'] >= 1) # compiled from {'myfield': {'$gte': 1}}"
>>> list(filter(to_func({"myfield": {"$gte": 2}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 2}, {'myfield': 3}, {'myfield': 4}]
* **$lt**::
>>> to_func({"myfield": {"$lt": 1}}).source
"lambda item: (item['myfield'] < 1) # compiled from {'myfield': {'$lt': 1}}"
>>> list(filter(to_func({"myfield": {"$lt": 1}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 0}]
* **$lte**::
>>> to_func({"myfield": {"$lte": 1}}).source
"lambda item: (item['myfield'] <= 1) # compiled from {'myfield': {'$lte': 1}}"
>>> list(filter(to_func({"myfield": {"$lte": 1}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 0}, {'myfield': 1}]
* **$eq**::
>>> to_func({"myfield": {"$eq": 1}}).source
"lambda item: (item['myfield'] == 1) # compiled from {'myfield': {'$eq': 1}}"
>>> to_func({"myfield": 1}).source
"lambda item: (item['myfield'] == 1) # compiled from {'myfield': 1}"
>>> list(filter(to_func({"myfield": {"$eq": 2}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 2}]
* **$ne**::
>>> to_func({"myfield": {"$ne": 1}}).source
"lambda item: (item['myfield'] != 1) # compiled from {'myfield': {'$ne': 1}}"
>>> list(filter(to_func({"myfield": {"$ne": 2}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 0}, {'myfield': 1}, {'myfield': 3}, {'myfield': 4}]
* **$mod**::
>>> to_func({"myfield": {"$mod": [2, 1]}}).source
"lambda item: (item['myfield'] % 2 == 1) # compiled from {'myfield': {'$mod': [2, 1]}}"
>>> to_func({"myfield": {"$mod": [2, 1, 3]}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
>>> to_func({"myfield": {"$mod": 2}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
>>> to_func({"myfield": {"$mod": (2, 1)}}).source
"lambda item: (item['myfield'] % 2 == 1) # compiled from {'myfield': {'$mod': (2, 1)}}"
>>> list(filter(to_func({"myfield": {"$mod": (2, 1)}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 1}, {'myfield': 3}]
to_func: Supported operators: Containers
````````````````````````````````````````
* **$in**::
>>> to_func({"myfield": {"$in": (1, 2, 3)}}).source
"lambda item, var0={1, 2, 3}: (item['myfield'] in var0) # compiled from {'myfield': {'$in': (1, 2, 3)}}"
>>> list(filter(to_func({"myfield": {"$in": (1, 2, 3)}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 1}, {'myfield': 2}, {'myfield': 3}]
* **$nin**::
>>> to_func({"myfield": {"$nin": [1, 2, 3]}}).source
"lambda item, var0={1, 2, 3}: (item['myfield'] not in var0) # compiled from {'myfield': {'$nin': [1, 2, 3]}}"
>>> to_func({"myfield": {"$nin": {1: 2}}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
>>> list(filter(to_func({"myfield": {"$nin": (1, 2, 3)}}), [{"myfield": i} for i in range(5)]))
[{'myfield': 0}, {'myfield': 4}]
* **$size**::
>>> to_func({"myfield": {"$size": 3}}).source
"lambda item: (len(item['myfield']) == 3) # compiled from {'myfield': {'$size': 3}}"
>>> to_func({"myfield": {"$size": "3"}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part '3'. Expected one of: int...
>>> list(filter(to_func({"myfield": {"$size": 3}}), [{"myfield": 'x'*i} for i in range(5)]))
[{'myfield': 'xxx'}]
>>> list(filter(to_func({"myfield": {"$size": 3}}), [{"myfield": list(range(i))} for i in range(5)]))
[{'myfield': [0, 1, 2]}]
* **$all**::
>>> to_func({"myfield": {"$all": [1, 2, 3]}}).source
"lambda item, var0={1, 2, 3}: (set(item['myfield']) >= var0) # compiled from {'myfield': {'$all': [1, 2, 3]}}"
>>> to_func({"myfield": {"$all": 1}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
>>> list(filter(to_func({"myfield": {"$all": [3, 4]}}), [{"myfield": list(range(i))} for i in range(7)]))
[{'myfield': [0, 1, 2, 3, 4]}, {'myfield': [0, 1, 2, 3, 4, 5]}]
* **$exists**::
>>> to_func({"myfield": {"$exists": True}}).source
"lambda item: ('myfield' in item) # compiled from {'myfield': {'$exists': True}}"
>>> to_func({"myfield": {"$exists": False}}).source
"lambda item: ('myfield' not in item) # compiled from {'myfield': {'$exists': False}}"
>>> list(filter(to_func({"$or": [{"field1": {"$exists": True}}, {"field2": {"$exists": False}}]}), [{"field%s" % i: i} for i in range(5)]))
[{'field0': 0}, {'field1': 1}, {'field3': 3}, {'field4': 4}]
to_func: Supported operators: Boolean operators
```````````````````````````````````````````````
* **$or**::
>>> to_func({'$or': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}).source
"lambda item: ((item['bubu'] > 1) or (item['bubu'] < 2)) # compiled from {'$or': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
>>> to_func({'$or': "invalid value"}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
>>> list(filter(to_func({'$or': [{"bubu": {"$gt": 3}}, {'bubu': {'$lt': 2}}]}), [{"bubu": i} for i in range(5)]))
[{'bubu': 0}, {'bubu': 1}, {'bubu': 4}]
* **$and**::
>>> to_func({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}).source
"lambda item: ((item['bubu'] > 1) and (item['bubu'] < 2)) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
>>> to_func({'$or': "invalid value"}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
>>> list(filter(to_func({'$and': [{"bubu": {"$lt": 3}}, {'bubu': {'$gt': 1}}]}), [{"bubu": i} for i in range(5)]))
[{'bubu': 2}]
* **$*nesting***::
>>> to_func({'$and': [
... {"bubu": {"$gt": 1}},
... {'$or': [
... {'bubu': {'$lt': 2}},
... {'$and': [
... {'bubu': {'$lt': 3}},
... {'bubu': {'$lt': 4}},
... ]}
... ]}
... ]}).source
"lambda item: ((item['bubu'] > 1) and ((item['bubu'] < 2) or ((item['bubu'] < 3) and (item['bubu'] < 4)))) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'$or': [{'bubu': {'$lt': 2}}, {'$and': [{'bubu': {'$lt': 3}}, {'bubu': {'$lt': 4}}]}]}]}"
to_func: Supported operators: Regular expressions
`````````````````````````````````````````````````
* **$regex**::
>>> to_func({"myfield": {"$regex": 'a'}}).source
"lambda item, var0=re.compile('a', 0): (var0.search(item['myfield'])) # compiled from {'myfield': {'$regex': 'a'}}"
>>> to_func({"myfield": {"$regex": 'a', "$options": 'i'}}).source
"lambda item, var0=re.compile('a', 2): (var0.search(item['myfield'])) # compiled from {'myfield': {...}}"
>>> to_func({"myfield": {"$regex": 'junk('}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid regular expression 'junk(':...
>>> to_func({"myfield": {"$regex": 'a', 'junk': 'junk'}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
>>> import re
>>> set(re.match(r'(.*): \((.*) and (.*)\)',
... to_func({"myfield": {"$regex": 'a', '$nin': ['aaa']}}, use_arguments=False).source
... ).groups()) == {
... 'lambda item',
... "(item['myfield'] not in {'aaa'})",
... "(re.search('a', item['myfield'], 0))"
... }
True
>>> to_func({"bubu": {"$regex": ".*", "$options": "junk"}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !
>>> to_func({"bubu": {"$options": "i"}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
>>> import string
>>> list(filter(to_func({"myfield": {"$regex": '[a-c]', "$options": 'i'}}), [{"myfield": i} for i in string.ascii_letters]))
[{'myfield': 'a'}, {'myfield': 'b'}, {'myfield': 'c'}, {'myfield': 'A'}, {'myfield': 'B'}, {'myfield': 'C'}]
>>> list(filter(to_func({"myfield": {"$regex": '[a-c]', "$nin": ['c']}}), [{"myfield": i} for i in string.ascii_letters]))
[{'myfield': 'a'}, {'myfield': 'b'}]
>>> total = len(string.ascii_letters)
>>> 2 * len(list(filter(
... to_func({"myfield": {"$regex": '[a-z]'}}),
... [{"myfield": i} for i in string.ascii_letters]
... ))) == total
True
>>> len(list(filter(
... to_func({"myfield": {"$regex": '[a-z]', '$options': 'i'}}),
... [{"myfield": i} for i in string.ascii_letters]
... ))) == total
True
>>> len(list(filter(
... to_func({"myfield": {"$regex": '[^\d]'}}),
... [{"myfield": i} for i in string.ascii_letters]
... ))) == total
True
to_func (lax mode)
==================
.. note::
Lax mode allows testing against missing fields.
::
>>> from mongoql_conv import LaxNone
>>> LaxNone < 1, LaxNone > 1, LaxNone == 0, LaxNone < 0, LaxNone > 0
(False, False, False, False, False)
>>> from mongoql_conv import to_func
>>> to_func({"myfield": 1}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': 1}"
>>> to_func({}, lax=True).source
'lambda item: (True) # compiled from {}'
>>> list(filter(to_func({"bogus": 1}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[]
>>> list(filter(to_func({}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]
>>> to_func({"myfield": {"$in": [1, 2]}}, lax=True).source
"lambda item, var0={1, 2}: ('myfield' in item and item.get('myfield', LaxNone) in var0) # compiled from {'myfield': {'$in': [1, 2]}}"
>>> list(filter(to_func({"bogus": {"$in": [1, 2]}}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[]
>>> to_func({"myfield": {"$in": {1: 2}}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
>>> to_func({"myfield": {"$and": []}}, lax=True).source
"lambda item: (True) # compiled from {'myfield': {'$and': []}}"
>>> list(filter(to_func({"bogus": {"$and": []}}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]
to_func (lax mode): Supported operators
---------------------------------------
to_func (lax mode): Supported operators: Arithmetic
```````````````````````````````````````````````````
* **$gt**::
>>> to_func({"myfield": {"$gt": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) > 1) # compiled from {'myfield': {'$gt': 1}}"
>>> to_func({"myfield": {"$gt": [1]}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part [1]. Expected one of: int,...float, str,...bool, None.
>>> list(filter(to_func({"bogus": {"$gt": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
[]
* **$gte**::
>>> to_func({"myfield": {"$gte": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) >= 1) # compiled from {'myfield': {'$gte': 1}}"
>>> list(filter(to_func({"bogus": {"$gte": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
[]
* **$lt**::
>>> to_func({"myfield": {"$lt": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) < 1) # compiled from {'myfield': {'$lt': 1}}"
>>> list(filter(to_func({"bogus": {"$lt": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
[]
* **$lte**::
>>> to_func({"myfield": {"$lte": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) <= 1) # compiled from {'myfield': {'$lte': 1}}"
>>> list(filter(to_func({"bogus": {"$lte": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
[]
* **$eq**::
>>> to_func({"myfield": {"$eq": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': {'$eq': 1}}"
>>> to_func({"myfield": 1}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': 1}"
>>> list(filter(to_func({"bogus": {"$eq": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
[]
* **$ne**::
>>> to_func({"myfield": {"$ne": 1}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) != 1) # compiled from {'myfield': {'$ne': 1}}"
>>> list(filter(to_func({"bogus": {"$ne": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
[]
* **$mod**::
>>> to_func({"myfield": {"$mod": [2, 1]}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) % 2 == 1) # compiled from {'myfield': {'$mod': [2, 1]}}"
>>> to_func({"myfield": {"$mod": [2, 1, 3]}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
>>> to_func({"myfield": {"$mod": 2}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
>>> to_func({"myfield": {"$mod": (2, 1)}}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) % 2 == 1) # compiled from {'myfield': {'$mod': (2, 1)}}"
>>> list(filter(to_func({"bogus": {"$mod": (2, 1)}}, lax=True), [{"myfield": i} for i in range(5)]))
[]
to_func (lax mode): Supported operators: Containers
```````````````````````````````````````````````````
* **$in**::
>>> to_func({"myfield": {"$in": (1, 2, 3)}}, lax=True).source
"lambda item, var0={1, 2, 3}: ('myfield' in item and item.get('myfield', LaxNone) in var0) # compiled from {'myfield': {'$in': (1, 2, 3)}}"
>>> list(filter(to_func({"bogus": {"$in": (1, 2, 3)}}, lax=True), [{"myfield": i} for i in range(5)]))
[]
* **$nin**::
>>> to_func({"myfield": {"$nin": [1, 2, 3]}}, lax=True).source
"lambda item, var0={1, 2, 3}: ('myfield' not in item or item.get('myfield', LaxNone) not in var0) # compiled from {'myfield': {'$nin': [1, 2, 3]}}"
>>> to_func({"myfield": {"$nin": {1: 2}}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
>>> list(filter(to_func({"bogus": {"$nin": (1, 2, 3)}}, lax=True), [{"myfield": i} for i in range(3)]))
[{'myfield': 0}, {'myfield': 1}, {'myfield': 2}]
* **$size**::
>>> to_func({"myfield": {"$size": 3}}, lax=True).source
"lambda item: (len(item.get('myfield', LaxNone)) == 3) # compiled from {'myfield': {'$size': 3}}"
>>> to_func({"myfield": {"$size": "3"}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part '3'. Expected one of: int...
>>> list(filter(to_func({"bogus": {"$size": 3}}, lax=True), [{"myfield": 'x'*i} for i in range(5)]))
[]
>>> list(filter(to_func({"bogus": {"$size": 3}}, lax=True), [{"myfield": list(range(i))} for i in range(5)]))
[]
* **$all**::
>>> to_func({"myfield": {"$all": [1, 2, 3]}}, lax=True).source
"lambda item, var0={1, 2, 3}: (set(item.get('myfield', LaxNone)) >= var0) # compiled from {'myfield': {'$all': [1, 2, 3]}}"
>>> to_func({"myfield": {"$all": 1}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
>>> list(filter(to_func({"bogus": {"$all": [3, 4]}}, lax=True), [{"myfield": list(range(i))} for i in range(7)]))
[]
* **$exists**::
>>> to_func({"myfield": {"$exists": True}}, lax=True).source
"lambda item: ('myfield' in item) # compiled from {'myfield': {'$exists': True}}"
>>> to_func({"myfield": {"$exists": False}}, lax=True).source
"lambda item: ('myfield' not in item) # compiled from {'myfield': {'$exists': False}}"
>>> list(filter(to_func({"$or": [{"bogus": {"$exists": True}}]}, lax=True), [{"field%s" % i: i} for i in range(5)]))
[]
to_func (lax mode): Supported operators: Boolean operators
``````````````````````````````````````````````````````````
* **$or**::
>>> to_func({'$or': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}, lax=True).source
"lambda item: ((item.get('bubu', LaxNone) > 1) or (item.get('bubu', LaxNone) < 2)) # compiled from {'$or': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
>>> to_func({'$or': "invalid value"}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
>>> list(filter(to_func({'$or': [{"bogus": {"$gt": 3}}, {'bogus': {'$lt': 2}}]}, lax=True), [{"bubu": i} for i in range(5)]))
[]
* **$and**::
>>> to_func({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}, lax=True).source
"lambda item: ((item.get('bubu', LaxNone) > 1) and (item.get('bubu', LaxNone) < 2)) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
>>> to_func({'$or': "invalid value"}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
>>> list(filter(to_func({'$and': [{"bogus": {"$lt": 3}}, {'bogus': {'$gt': 1}}]}, lax=True), [{"bubu": i} for i in range(5)]))
[]
* **$*nesting***::
>>> to_func({'$and': [
... {"bubu": {"$gt": 1}},
... {'$or': [
... {'bubu': {'$lt': 2}},
... {'$and': [
... {'bubu': {'$lt': 3}},
... {'bubu': {'$lt': 4}},
... ]}
... ]}
... ]}, lax=True).source
"lambda item: ((item.get('bubu', LaxNone) > 1) and ((item.get('bubu', LaxNone) < 2) or ((item.get('bubu', LaxNone) < 3) and (item.get('bubu', LaxNone) < 4)))) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'$or': [{'bubu': {'$lt': 2}}, {'$and': [{'bubu': {'$lt': 3}}, {'bubu': {'$lt': 4}}]}]}]}"
to_func (lax mode): Supported operators: Regular expressions
````````````````````````````````````````````````````````````
* **$regex**::
>>> to_func({"myfield": {"$regex": 'a'}}, lax=True).source
"lambda item, var0=re.compile('a', 0): (var0.search(item.get('myfield', ''))) # compiled from {'myfield': {'$regex': 'a'}}"
>>> to_func({"myfield": {"$regex": 'a', "$options": 'i'}}, lax=True).source
"lambda item, var0=re.compile('a', 2): (var0.search(item.get('myfield', ''))) # compiled from {'myfield': {...}}"
>>> to_func({"myfield": {"$regex": 'junk('}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid regular expression 'junk(':...
>>> to_func({"myfield": {"$regex": 'a', 'junk': 'junk'}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
>>> set(re.match(r'(.*): \((.*) and (.*)\)',
... to_func({"myfield": {"$regex": 'a', '$nin': ['aaa']}}, lax=True, use_arguments=False).source
... ).groups()) == {
... "lambda item",
... "(re.search('a', item.get('myfield', ''), 0))",
... "('myfield' not in item or item.get('myfield', LaxNone) not in {'aaa'})"
... }
True
>>> to_func({"bubu": {"$regex": ".*", "$options": "junk"}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !
>>> to_func({"bubu": {"$options": "i"}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
>>> import string
>>> list(filter(to_func({"bogus": {"$regex": '[a-c]', "$options": 'i'}}, lax=True), [{"myfield": i} for i in string.ascii_letters]))
[]
>>> list(filter(to_func({"bogus": {"$regex": '[a-c]', "$nin": ['c']}}, lax=True), [{"myfield": i} for i in string.ascii_letters]))
[]
>>> total = len(string.ascii_letters)
>>> 2 * len(list(filter(
... to_func({"bougs": {"$regex": '[a-z]'}}, lax=True),
... [{"myfield": i} for i in string.ascii_letters]
... ))) == 0
True
>>> len(list(filter(
... to_func({"bogus": {"$regex": '[a-z]', '$options': 'i'}}, lax=True),
... [{"myfield": i} for i in string.ascii_letters]
... ))) == 0
True
>>> len(list(filter(
... to_func({"bougs": {"$regex": '[^\d]'}}, lax=True),
... [{"myfield": i} for i in string.ascii_letters]
... ))) == 0
True
to_Q
====
Compiles down to a Django Q object tree::
>>> from mongoql_conv.django import to_Q
>>> print(to_Q({"myfield": 1}))
(AND: ('myfield', 1))
>>> print(to_Q({}))
(AND: )
>>> from test_app.models import MyModel
>>> MyModel.objects.clean_and_create([(i, i) for i in range(5)])
>>> list(MyModel.objects.filter(to_Q({"field1": 1})))
[]
>>> list(MyModel.objects.filter(to_Q({"field1": 1, "field2": 1})))
[]
>>> print(to_Q({"myfield": {"$in": [1, 2]}}))
(AND: ('myfield__in', [1, 2]))
>>> list(MyModel.objects.filter(to_Q({"field1": {"$in": [1, 2]}})))
[, ]
>>> print(to_Q({"myfield": {"$in": {1: 2}}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
>>> print(to_Q({"myfield": {"$and": []}}))
(AND: )
>>> list(MyModel.objects.filter(to_Q({"field1": {"$and": []}})))
[, , , , ]
to_Q: Supported operators
-------------------------
to_Q: Supported operators: Arithmetic
`````````````````````````````````````
* **$gt**::
>>> print(to_Q({"myfield": {"$gt": 1}}))
(AND: ('myfield__gt', 1))
>>> list(MyModel.objects.filter(to_Q({"field1": {"$gt": 2}})))
[, ]
* **$gte**::
>>> print(to_Q({"myfield": {"$gte": 1}}))
(AND: ('myfield__gte', 1))
>>> list(MyModel.objects.filter(to_Q({"field1": {"$gte": 2}})))
[, , ]
* **$lt**::
>>> print(to_Q({"myfield": {"$lt": 1}}))
(AND: ('myfield__lt', 1))
>>> list(MyModel.objects.filter(to_Q({"field1": {"$lt": 1}})))
[]
* **$lte**::
>>> print(to_Q({"myfield": {"$lte": 1}}))
(AND: ('myfield__lte', 1))
>>> list(MyModel.objects.filter(to_Q({"field1": {"$lte": 1}})))
[, ]
* **$eq**::
>>> print(to_Q({"myfield": {"$eq": 1}}))
(AND: ('myfield', 1))
>>> list(MyModel.objects.filter(to_Q({"field1": 1})))
[]
>>> print(to_Q({"myfield": 1}))
(AND: ('myfield', 1))
>>> list(MyModel.objects.filter(to_Q({"field1": {"$eq": 1}})))
[]
* **$ne**::
>>> str(to_Q({"myfield": {"$ne": 1}})) in ["(NOT (AND: ('myfield', 1)))", "(AND: (NOT (AND: ('myfield', 1))))"]
True
>>> list(MyModel.objects.filter(to_Q({"field1": {"$ne": 1}})))
[, , , ]
* **$mod**::
>>> print(to_Q({"myfield": {"$mod": [2, 1]}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: DjangoVisitor doesn't support operator '$mod'
to_Q: Supported operators: Containers
`````````````````````````````````````
* **$in**::
>>> print(to_Q({"myfield": {"$in": (1, 2, 3)}}))
(AND: ('myfield__in', (1, 2, 3)))
>>> list(MyModel.objects.filter(to_Q({"field1": {"$in": (1, 2)}})))
[, ]
* **$nin**::
>>> str(to_Q({"myfield": {"$nin": [1, 2, 3]}})) in ["(NOT (AND: ('myfield__in', [1, 2, 3])))", "(AND: (NOT (AND: ('myfield__in', [1, 2, 3]))))"]
True
>>> list(MyModel.objects.filter(to_Q({"field1": {"$nin": (1, 2)}})))
[, , ]
* **$size**::
>>> print(to_Q({"myfield": {"$size": 3}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: DjangoVisitor doesn't support operator '$size'
* **$all**::
>>> print(to_Q({"myfield": {"$all": [1, 2, 3]}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: DjangoVisitor doesn't support operator '$all'
* **$exists**::
>>> print(to_Q({"myfield": {"$exists": True}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: DjangoVisitor doesn't support operator '$exists'
to_Q: Supported operators: Boolean operators
````````````````````````````````````````````
* **$or**::
>>> print(to_Q({'$or': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}))
(OR: ('bubu__gt', 1), ('bubu__lt', 2))
>>> list(MyModel.objects.filter(to_Q({'$or': [{"field1": {"$gt": 3}}, {'field1': {'$lt': 2}}]})))
[, , ]
* **$and**::
>>> print(to_Q({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}))
(AND: ('bubu__gt', 1), ('bubu__lt', 2))
>>> list(MyModel.objects.filter(to_Q({'$and': [{"field1": {"$gt": 1}}, {'field1': {'$lt': 3}}]})))
[]
* **$*nesting***::
>>> print(to_Q({'$and': [
... {"bubu": {"$gt": 1}},
... {'$or': [
... {'bubu': {'$lt': 2}},
... {'$and': [
... {'bubu': {'$lt': 3}},
... {'bubu': {'$lt': 4}},
... ]}
... ]}
... ]}))
(AND: ('bubu__gt', 1), (OR: ('bubu__lt', 2), (AND: ('bubu__lt', 3), ('bubu__lt', 4))))
>>> list(MyModel.objects.filter(to_Q({'$and': [
... {"field1": {"$gt": 1}},
... {'$or': [
... {'field2': {'$lt': 2}},
... {'$and': [
... {'field2': {'$lt': 5}},
... {'field2': {'$gt': 2}},
... ]}
... ]}
... ]})))
[, ]
to_Q: Supported operators: Regular expressions
``````````````````````````````````````````````
* **$regex**::
>>> print(to_Q({"myfield": {"$regex": 'a'}}))
(AND: ('myfield__regex', 'a'))
>>> print(to_Q({"myfield": {"$regex": 'a', "$options": 'i'}}))
(AND: ('myfield__iregex', 'a'))
>>> print(to_Q({"myfield": {"$regex": 'junk('}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid regular expression 'junk(':...
>>> print(to_Q({"myfield": {"$regex": 'a', 'junk': 'junk'}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
>>> "('myfield__regex', 'a')" in str(to_Q({"myfield": {"$regex": 'a', '$nin': ['aaa']}}))
True
>>> "(NOT (AND: ('myfield__in', ['aaa'])))" in str(to_Q({"myfield": {"$regex": 'a', '$nin': ['aaa']}}))
True
>>> print(to_Q({"bubu": {"$regex": ".*", "$options": "mxs"}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part 'mxs'. Unsupported regex option 'm'. Only i are supported !
>>> print(to_Q({"bubu": {"$options": "i"}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
>>> MyModel.objects.clean_and_create([(None, "prefix__"+i) for i in string.ascii_letters])
>>> list(MyModel.objects.filter(to_Q({"field2": {"$regex": '[a-b]', "$options": 'i'}})))
[, , , ]
>>> list(MyModel.objects.filter(to_Q({"field2": {"$regex": '[a-c]', "$nin": ['prefix__c']}})))
[, ]
>>> total = MyModel.objects.count()
>>> total == 2 * MyModel.objects.filter(to_Q({"field2": {"$regex": '__[a-z]'}})).count()
True
>>> total == MyModel.objects.filter(to_Q({"field2": {"$regex": '__[a-z]', '$options': 'i'}})).count()
True
>>> total == MyModel.objects.filter(to_Q({"field2": {"$regex": '[^\d]'}})).count()
True
Extending (implementing a custom visitor)
=========================================
There are few requirements for a visitor. Fist, you need to be able to render boolean $and::
>>> from mongoql_conv import BaseVisitor
>>> class MyVisitor(BaseVisitor):
... def __init__(self, object_name):
... self.object_name = object_name
... def visit_foobar(self, value, field_name, context):
... return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
... def validate_foobar(self, value, field_name, context):
... return value
>>> MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
Traceback (most recent call last):
...
TypeError: Can't instantiate abstract class MyVisitor with abstract methods render_and
This is the minimal code to have a custom generator::
>>> class MyVisitor(BaseVisitor):
... def __init__(self, object_name):
... self.object_name = object_name
... def visit_foobar(self, value, field_name, context):
... return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
... def render_and(self, parts, field_name, context):
... return ' & '.join(parts)
... def validate_foobar(self, value, field_name, context):
... return value
>>> MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
"foobar(obj['field'], 'test')"
Ofcourse, it won't do much::
>>> MyVisitor('obj').visit({'field': {'$ne': 'test'}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: MyVisitor doesn't support operator '$ne'
Take a look at ``ExprVisitor`` too see all the methods you *should* implement.
Note that the validate method is optional but if you don't implement it you'll get a warning::
>>> import warnings
>>> class MyVisitor(BaseVisitor):
... def __init__(self, object_name):
... self.object_name = object_name
... def visit_foobar(self, value, field_name, context):
... return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
... def render_and(self, parts, field_name, context):
... return ' & '.join(parts)
>>> with warnings.catch_warnings(record=True) as w:
... MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
"foobar(obj['field'], 'test')"
>>> len(w)
1
>>> w[0].message
UserWarning('Missing validator validate_foobar in MyVisitor',)