{"id":25426786,"url":"https://github.com/comic31/mongodbqueriesmanager","last_synced_at":"2025-10-31T16:30:26.738Z","repository":{"id":57442964,"uuid":"338293167","full_name":"comic31/MongoDBQueriesManager","owner":"comic31","description":"Convert query parameters from API urls to MongoDB queries !","archived":false,"fork":false,"pushed_at":"2022-11-01T15:24:27.000Z","size":130,"stargazers_count":5,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-16T10:55:21.600Z","etag":null,"topics":["aiohttp","api","fastapi","library","mongodb","mongodb-query","motor","pymongo","query","query-builder","rest","sanic","url"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/comic31.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-12T10:58:27.000Z","updated_at":"2024-11-15T16:57:16.000Z","dependencies_parsed_at":"2022-09-04T22:22:21.235Z","dependency_job_id":null,"html_url":"https://github.com/comic31/MongoDBQueriesManager","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comic31%2FMongoDBQueriesManager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comic31%2FMongoDBQueriesManager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comic31%2FMongoDBQueriesManager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/comic31%2FMongoDBQueriesManager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/comic31","download_url":"https://codeload.github.com/comic31/MongoDBQueriesManager/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239214050,"owners_count":19601077,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiohttp","api","fastapi","library","mongodb","mongodb-query","motor","pymongo","query","query-builder","rest","sanic","url"],"created_at":"2025-02-17T00:22:07.045Z","updated_at":"2025-10-31T16:30:26.680Z","avatar_url":"https://github.com/comic31.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MongoDBQueriesManager\n[![Codecov](https://img.shields.io/codecov/c/github/comic31/MongoDBQueriesManager?style=for-the-badge)](https://app.codecov.io/gh/comic31/MongoDBQueriesManager)\n[![Main workflow](https://img.shields.io/github/workflow/status/comic31/MongoDBQueriesManager/Main%20workflow?style=for-the-badge)](https://github.com/comic31/MongoDBQueriesManager/actions/workflows/main.yaml)\n[![PyPI](https://img.shields.io/pypi/v/mongo-queries-manager?style=for-the-badge)](https://pypi.org/project/mongo-queries-manager/)\n[![GitHub](https://img.shields.io/github/license/comic31/MongoDBQueriesManager?style=for-the-badge)](https://github.com/comic31/MongoDBQueriesManager/blob/main/LICENSE)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/mongo-queries-manager?style=for-the-badge)](https://pypi.org/project/mongo-queries-manager/)\n[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg?style=for-the-badge)](https://github.com/comic31/MongoDBQueriesManager/blob/main/code_of_conduct.md)\n\nConvert query parameters from API urls to MongoDB queries !\n\nThis project was inspired by [api-query-params](https://github.com/loris/api-query-params) (JS Library).\n\n## Features:\n- **Powerful**: Supports most of MongoDB operators ($in, $regexp, ...) and features (nested objects, type casting, projection, range filter...)\n- **Agnostic**: Works with any web frameworks (Flask, Sanic, AIOHTTP, Django ...) and/or MongoDB libraries (pymongo, motor, ...)\n- **Simple**: ~500 LOC, Python typing\n- **Tested**: 100% code coverage\n\n\n## Installation:\n\n\u003e ⚠️ **In version 1.0.0 dateparser is an extra dependencies**\n\n```shell script\npip install mongo-queries-manager\npip install mongo-queries-manager['dateparser']\n\n# OR\n\npipenv install mongo-queries-manager\npipenv install mongo-queries-manager['dateparser']\n\n# OR\n\npoetry add mongo-queries-manager\npoetry add mongo-queries-manager['dateparser']\n```\n\n## Usages:\n### Api\n`mqm(string_query: str, blacklist: Optional[List[str]] = None, casters: Optional[Dict[str, Callable]] = None, populate: bool = False) -\u003e Dict[str, Any]:`\n\n##### Description\nConverts `string_query` into a MongoDB query dict.\n\n##### Arguments\n- `string_query`: query string of the requested API URL (ie, `frist_name=John\u0026limit=10`), Works with url encoded. [required]\n- `casters`: Custom caster dict, used to define custom type (ie, `casters={'string': str}` / `price=string(5.5)` -\u003e `{'price': '5'}`) [optional]\n- `blacklist`: Custom blacklist word, used to ignore specific value from query (ie, `blacklist=[where]` / `company=id,where=43.60,1.44,` -\u003e `{'company': 'id'}`) [optional]\n- `populate`: A boolean value, used to activate the population logic (add a population field into returned dict)\n\n##### Returns\nThe resulting dictionary contains the following properties:\n- `filter`: Contains the query criteria.\n- `projection`: Contains the query projection\n- `sort`: Contains the sort criteria (cursor modifiers).\n- `skip`: Contains the skip criteria (cursor modifiers).\n- `limit`:  Contains the limit criteria (cursor modifiers).\n- `population`:  Contains the population criteria. (Only when populate arg is true. To use this population list, a manual implementation is required)\n\n##### Exception\nIn case of error the following exception was raised:\n\n- `MongoDBQueriesManagerBaseError`: Base MongoDBQueriesManager errors.\n- `SkipError`: Raised when skip is negative / bad value.\n- `LimitError`: Raised when limit is negative / bad value.\n- `ListOperatorError`: Raised list operator was not possible.\n- `FilterError`: Raised when parse filter method fail to find a valid match.\n- `TextOperatorError`: Raised when parse text operator contain an empty string.\n- `CustomCasterFail`: Raised when a custom cast fail.\n- `ProjectionError`: Raised when projection json is invalid.\n- `LogicalPopulationError`: Raised when method fail to find logical population item.\n- `LogicalSubPopulationError`: Raised when method fail to find logical sub population item.\n\n##### Examples:\n\n**Simple demo**\n\n```python\nfrom mongo_queries_manager import mqm\n\nmongodb_query = mqm(string_query=\"status=sent\u0026price\u003e=5.6\u0026active=true\u0026timestamp\u003e\"\n                                 \"2016-01-01\u0026author.firstName=/john/i\u0026limit=100\u0026skip=50\u0026sort=-timestamp\u0026fields=-_id,-created_at\")\n\n# {\n#   'filter':\n#       {\n#           'status': 'sent',\n#           'price': {'$gte': 5.6},\n#           'active': True,\n#           'timestamp': {'$gt': datetime.datetime(2016, 1, 1, 0, 0)},\n#           'author.firstName': re.compile('/john/i')\n#       },\n#   'projection': {'_id': 0, 'created_at': 0},\n#   'sort': [('timestamp', -1)],\n#   'skip': 50,\n#   'limit': 100\n# }\n```\n\n**Examples with PyMongo**\n\n```python\nfrom typing import Dict, Any\n\nfrom pymongo import MongoClient\nfrom pymongo.collection import Collection\nfrom pymongo.database import Database\n\nfrom mongo_queries_manager import mqm\n\nclient: MongoClient = MongoClient('localhost', 27017)\ndb: Database = client['test-database']\ncollection: Collection = db['test-collection']\n\nmongodb_query: Dict[str, Any] = mqm(string_query=\"status=sent\u0026toto=true\u0026timestamp\u003e2016-01-01\u0026\"\n                                                 \"author.firstName=/john/i\u0026limit=100\u0026skip=50\u0026sort=-timestamp\")\n\nresult = collection.find(**mongodb_query)\n```\n\n## Supported features\n\n#### Filter operators:\n| MongoDB   | URI                  | Example                 | Result                                                                        |\n| :-------: | :------------------: | :---------------------: | :---------------------------------------------------------------------------: |\n| `$eq`     | `key=val`            | `type=public`           | `{'filter': {'type': 'public'}}`                                              |\n| `$gt`     | `key\u003eval`            | `count\u003e5`               | `{'filter': {'count': {'$gt': 5}}}`                                           |\n| `$gte`    | `key\u003e=val`           | `rating\u003e=9.5`           | `{'filter': {'rating': {'$gte': 9.5}}}`                                       |\n| `$lt`     | `key\u003cval`            | `createdAt\u003c2016-01-01`  | `{'filter': {'createdAt': {'$lt': datetime.datetime(2016, 1, 1, 0, 0)}}}`     |\n| `$lte`    | `key\u003c=val`           | `score\u003c=-5`             | `{'filter': {'score': {'$lte': -5}}}`                                         |\n| `$ne`     | `key!=val`           | `status!=success`       | `{'filter': {'status': {'$ne': 'success'}}}`                                  |\n| `$in`     | `key=val1,val2`      | `country=GB,US`         | `{'filter': {'country': {'$in': ['GB', 'US']}}}`                              |\n| `$nin`    | `key!=val1,val2`     | `lang!=fr,en`           | `{'filter': {'lang': {'$nin': ['fr', 'en']}}}`                                |\n| `$exists` | `key`                | `phone`                 | `{'filter': {'phone': {'$exists': True}}}`                                    |\n| `$exists` | `!key`               | `!email`                | `{'filter': {'email': {'$exists': False}}}`                                   |\n| `$regex`  | `key=/value/\u003copts\u003e`  | `email=/@gmail\\.com$/i` | `{'filter': {'email': re.compile('/@gmail.com$/i')}}`                         |\n| `$regex`  | `key!=/value/\u003copts\u003e` | `phone!=/^06/`          | `{'filter': {'phone': { '$not': re.compile('/^06/')}}}`                       |\n| `$text`   | `$text=val`          | `$text=toto -java`      | `{'filter': {'$text': { '$search': 'toto -java'}}}`                             |\n| `$text`   | `$text=val`          | `$text=\"toto\"`          | `{'filter': {'$text': { '$search': '\"toto\"'}}}`                             |\n\n#### Skip / Limit operators:\n\n- Default operator keys are `skip` and `limit`.\n- Used to limit the number of records returned by the query (pagination, result limitation, ...).\n- Support empty value (ie, `...\u0026skip=\u0026...` / `...\u0026limit=\u0026...` ).\n\n```python\nfrom typing import Dict, Any\n\nfrom mongo_queries_manager import mqm\n\nmongodb_query: Dict[str, Any] = mqm(string_query=\"skip=50\u0026limit=50\")\n# {\n#   'filter': {},\n#   'sort': None,\n#   'projection': None,\n#   'skip': 50,\n#   'limit': 50\n# }\n\nmongodb_query: Dict[str, Any] = mqm(string_query=\"skip=\u0026limit=\")\n# {\n#   'filter': {},\n#   'sort': None,\n#   'projection': None,\n#   'skip': 0,\n#   'limit': 0\n# }\n```\n\n#### Sort operator:\n- Used to sort returned records.\n- Default operator key is `sort`.\n- Support empty value (ie, `...\u0026sort=\u0026...`).\n- Sort accepts a comma-separated list of fields.\n- Default behavior is to sort in ascending order.\n- Use `-` prefixes to sort in descending order, use `+` prefixes to sort in ascending order.\n\n```python\nfrom typing import Dict, Any\n\nfrom mongo_queries_manager import mqm\n\nmongodb_query: Dict[str, Any] = mqm(string_query=\"sort=created_at,-_id,+price\")\n#{\n#   'filter': {},\n#   'sort': [('created_at', 1), ('_id', -1), ('price', 1)],\n#   'projection': None,\n#   'skip': 0,\n#   'limit': 0\n#}\n```\n\n#### Projection operator:\n- Useful to limit fields to return in each records.\n- It accepts a comma-separated list of fields. Default behavior is to specify fields to return. Use - prefixes to return all fields except some specific fields.\n- Due to a MongoDB limitation, you cannot combine inclusion and exclusion semantics in a single projection with the exception of the _id field.\n- It also accepts JSON string to use more powerful projection operators ($, $elemMatch or $slice)\n\n```python\nfrom typing import Dict, Any\n\nfrom mongo_queries_manager import mqm\n\nmongodb_query: Dict[str, Any] = mqm(string_query=\"fields=-_id,-price\")\n#{\n#   'filter': {},\n#   'sort': None,\n#   'projection': {'_id': 0, 'price': 0},\n#   'skip': 0,\n#   'limit': 0\n#}\n\n\nmongodb_query: Dict[str, Any] = mqm(string_query=\"fields=_id,price\")\n\n#{\n#   'filter': {},\n#   'sort': None,\n#   'projection': {'_id': 1, 'price': 1},\n#   'skip': 0,\n#   'limit': 0\n#}\n\n\nmongodb_query: Dict[str, Any] = mqm(\n    string_query='fields={\"games\": {\"$elemMatch\":{\"score\": {\"$gt\": 5}}}},joined,lastLogin')\n\n#{\n#   'filter': {},\n#   'sort': None,\n#   'projection': {'games': {'$elemMatch': {'score': {'$gt': 5}}}, 'joined': 1, 'lastLogin': 1}},\n#   'skip': 0,\n#   'limit': 0\n#}\n```\n\n#### Range filter:\n- Useful to filter fields to return in each records by range.\n- No error was handle by this library for range filter\n\n```python\nfrom typing import Dict, Any\n\nfrom mongo_queries_manager import mqm\n\nquery_result: Dict[str, Any] = mqm(string_query=\"price\u003e5\u0026price\u003c5\")\n\n# {\n# 'filter':\n# {\n#   'price': {'$gt': 5.0, '$lt': 5.0},\n#   },\n#   'sort': None,\n#   'projection': None,\n#   'skip': 0,\n#   'limit': 0\n# }\n```\n\n#### Custom caster:\n- Used to define custom type\n- Optional parameter\n\n```python\nfrom typing import Dict, Any, List\n\nfrom mongo_queries_manager import mqm\n\n\ndef parse_custom_list(custom_list: str) -\u003e List[str]:\n    return custom_list.split(';')\n\n\nquery_result: Dict[str, Any] = mqm(string_query=\"price=string(5)\u0026name=John\u0026in_stock=custom_list(1;2;3;4)\u0026\"\n                                                \"in_stock_string=custom_list(string(1);string(2);string(3);string(4))\",\n                                   casters={'string': str, 'custom_list': parse_custom_list})\n\n#{\n# 'filter':\n# {\n#   'price': '5',\n#   'name': 'John',\n#   'in_stock': {'$in': [1, 2, 3, 4]},\n#   'in_stock_string': {'$in': ['1', '2', '3', '4']}\n#   },\n#   'sort': None,\n#   'projection': None,\n#   'skip': 0,\n#   'limit': 0\n#}\n```\n\n\n## Contribution\n\n### Install all development dependencies\n\n```shell\n# Initialize a new virtual environment\npoetry shell\n\n# Install dev dependencies\npoetry install --with format,lint,type,tools,tests -E dateparser\n\n# Run tests\nnox\n\n# Pre commit (format / lint / type before commit)\npre-commit install\npre-commit run --all-files\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomic31%2Fmongodbqueriesmanager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcomic31%2Fmongodbqueriesmanager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomic31%2Fmongodbqueriesmanager/lists"}