{"id":16589410,"url":"https://github.com/bluerelay/windyquery","last_synced_at":"2025-03-16T21:30:30.807Z","repository":{"id":62589160,"uuid":"170042490","full_name":"bluerelay/windyquery","owner":"bluerelay","description":"Python 3.6+ Asyncio PostgreSQL query builder and model","archived":false,"fork":false,"pushed_at":"2023-02-19T15:22:58.000Z","size":160,"stargazers_count":67,"open_issues_count":2,"forks_count":3,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-02-27T14:12:20.226Z","etag":null,"topics":["activerecord","asynchronous","orm","postgresql","python","query-builder"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bluerelay.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-11T00:07:44.000Z","updated_at":"2024-11-28T16:34:15.000Z","dependencies_parsed_at":"2022-11-03T17:01:57.453Z","dependency_job_id":null,"html_url":"https://github.com/bluerelay/windyquery","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluerelay%2Fwindyquery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluerelay%2Fwindyquery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluerelay%2Fwindyquery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluerelay%2Fwindyquery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bluerelay","download_url":"https://codeload.github.com/bluerelay/windyquery/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243830915,"owners_count":20354850,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activerecord","asynchronous","orm","postgresql","python","query-builder"],"created_at":"2024-10-11T23:08:42.229Z","updated_at":"2025-03-16T21:30:30.447Z","avatar_url":"https://github.com/bluerelay.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# windyquery - A non-blocking Python PostgreSQL query builder\n\nWindyquery is a non-blocking PostgreSQL query builder with Asyncio.\n\n### Installation\n```\n$ pip install windyquery\n```\n\n### Connection\n```python\nimport asyncio\n\nfrom windyquery import DB\n\n# create DB connection for CRUD operatons\ndb = DB()\nasyncio.get_event_loop().run_until_complete(db.connect('db_name', {\n    'host': 'localhost',\n    'port': '5432',\n    'database': 'db_name',\n    'username': 'db_user_name',\n    'password': 'db_user_password'\n}, default=True))\n\nasyncio.get_event_loop().run_until_complete(db.connect('other_db_name', {\n    'host': 'localhost',\n    'port': '5432',\n    'database': 'other_db_name',\n    'username': 'db_user_name',\n    'password': 'db_user_password'\n}, default=False))\n\n# switch connections between different databases\ndb.connection('other_db_name')\n\n# the default connection can also be changed directly\ndb.default = 'other_db_name'\n\n# close DB connection\nasyncio.get_event_loop().run_until_complete(db.stop())\n```\n\n### CRUD examples\nA DB instance can be used to constuct a SQL. The instance is a coroutine object.\nIt can be scheduled to run by all [asyncio](https://docs.python.org/3/library/asyncio-task.html) mechanisms.\n\n#### Build a SQL and execute it\n```python\nasync def main(db):\n    # SELECT id, name FROM users\n    users = await db.table('users').select('id', 'name')\n    print(users[0]['name'])\n\nasyncio.run(main(db))\n```\n\n#### SELECT\n```python\n# SELECT name AS username, address addr FROM users\nawait db.table('users').select('name AS username', 'address addr')\n\n# SELECT * FROM users WHERE id = 1 AND name = 'Tom'\nawait db.table('users').select().where('id', 1).where('name', 'Tom')\n\n# SELECT * FROM users WHERE id = 1 AND name = 'Tom'\nawait db.table('users').select().where('id', '=', 1).where('name', '=', 'Tom')\n\n# SELECT * FROM users WHERE id = 1 AND name = 'Tom'\nawait db.table('users').select().where('id = ? AND name = ?', 1, 'Tom')\n\n# SELECT * FROM users WHERE id IN (1, 2)\nawait db.table('cards').select().where(\"id\", [1, 2])\n\n# SELECT * FROM users WHERE id IN (1, 2)\nawait db.table('cards').select().where(\"id\", 'IN', [1, 2])\n\n# SELECT * FROM users WHERE id IN (1, 2)\nawait db.table('cards').select().where(\"id IN (?, ?)\", 1, 2)\n\n# SELECT * FROM users ORDER BY id, name DESC\nawait db.table('users').select().order_by('id', 'name DESC')\n\n# SELECT * FROM users GROUP BY id, name\nawait db.table('users').select().group_by('id', 'name')\n\n# SELECT * FROM users LIMIT 100 OFFSET 10\nawait db.table('users').select().limit(100).offset(10)\n\n# SELECT users.*, orders.total FROM users\n#   JOIN orders ON orders.user_id = users.id\nawait db.table('users').select('users.*', 'orders.total').\\\n    join('orders', 'orders.user_id', '=', 'users.id')\n    \n# SELECT users.*, orders.total FROM users\n#   JOIN orders ON orders.user_id = users.id AND orders.total \u003e 100\nawait db.table('users').select('users.*', 'orders.total').\\\n    join('orders', 'orders.user_id = users.id AND orders.total \u003e ?', 100)\n```\n\n#### INSERT\n```python\n# INSERT INTO users(id, name) VALUES\n#   (1, 'Tom'),\n#   (2, 'Jerry'),\n#   (3, DEFAULT)\nawait db.table('users').insert(\n    {'id': 1, 'name': 'Tom'},\n    {'id': 2, 'name': 'Jerry'},\n    {'id': 3, 'name': 'DEFAULT'}\n)\n\n# INSERT INTO users(id, name) VALUES\n#   (1, 'Tom'),\n#   (2, 'Jerry'),\n#   (3, DEFAULT)\n#   RETRUNING id, name\nawait db.table('users').insert(\n    {'id': 1, 'name': 'Tom'},\n    {'id': 2, 'name': 'Jerry'},\n    {'id': 3, 'name': 'DEFAULT'}\n).returning('id', 'name')\n\n# INSERT INTO users(id, name) VALUES\n#   (1, 'Tom'),\n#   (2, 'Jerry'),\n#   (3, DEFAULT)\n#   RETRUNING *\nawait db.table('users').insert(\n    {'id': 1, 'name': 'Tom'},\n    {'id': 2, 'name': 'Jerry'},\n    {'id': 3, 'name': 'DEFAULT'}\n).returning()\n\n# INSERT INTO users (id, name) VALUES\n#   (1, 'Tom')\n#   ON CONFLICT (id) DO NOTHING\nawait db.table('users').insert(\n    {'id': 1, 'name': 'Tom'},\n).on_conflict('(id)', 'DO NOTHING')\n\n# INSERT INTO users As u (id, name) VALUES\n#   (1, 'Tom')\n#   ON CONFLICT ON CONSTRAINT users_pkey\n#   DO UPDATE SET name = EXCLUDED.name || ' (formerly ' || u.name || ')'\nawait db.table('users AS u').insert(\n    {'id': 1, 'name': 'Tom'},\n).on_conflict(\n    'ON CONSTRAINT users_pkey',\n    \"DO UPDATE SET name = EXCLUDED.name || ' (formerly ' || u.name || ')'\"\n)        \n```\n\n#### UPDATE\n```python\n# UPDATE cards SET name = 'Tom' WHERE id = 9\nawait db.table('cards').where('id', 9).update({'name': 'Tom'})\n\n# UPDATE cards SET total = total + 1 WHERE id = 9\nawait db.table('cards').update('total = total + 1').where('id', 9)\n\n# UPDATE users SET name = 'Tom' WHERE id = 9 RETRUNING *\nawait db.table('users').update({'name': 'Tom'}).where('id', '=', 9).returning()\n\n# UPDATE users SET name = 'Tom' WHERE id = 9 RETRUNING id, name\nawait db.table('users').update({'name': 'Tom'}).where('id', '=', 9).returning('id', 'name')\n\n# UPDATE users SET name = orders.name\n#   FROM orders\n#   WHERE orders.user_id = users.id\nawait db.table('users').update('name = orders.name').\\\n    from_table('orders').\\\n    where('orders.user_id = users.id')\n\n# UPDATE users SET name = products.name, purchase = products.name, is_paid = TRUE\n#   FROM orders\n#   JOIN products ON orders.product_id = products.id\n#   WHERE orders.user_id = users.id\nawait db.table('users').update('name = product.name, purchase = products.name, is_paid = ?', True).\\\n    from_table('orders').\\\n    join('products', 'orders.product_id', '=', 'products.id').\\\n    where('orders.user_id = users.id')\n```\n\n#### DELETE\n```python\n# DELETE FROM users WHERE id = 1\nawait db.table('users').where('id', 1).delete()\n\n# DELETE FROM users WHERE id = 1 RETURNING id, name\nawait db.table('users').where('id', 1).delete().returning('id', 'name')\n```\n\n### Migration examples\nThe DB instance can also be used to migrate database schema.\n\n#### CREATE TABLE\n```python\n# CREATE TABLE users (\n#    id            serial PRIMARY KEY,\n#    group_id      integer references groups (id) ON DELETE CASCADE,\n#    created_at    timestamp not null DEFAULT NOW(),\n#    email         text not null unique,\n#    is_admin      boolean not null default false,\n#    address       jsonb,\n#    payday        integer not null,\n#    CONSTRAINT unique_email UNIQUE(group_id, email)\n#    check(payday \u003e 0 and payday \u003c 8)\n#)\nawait db.schema('TABLE users').create(\n    'id            serial PRIMARY KEY',\n    'group_id      integer references groups (id) ON DELETE CASCADE',\n    'created_at    timestamp not null DEFAULT NOW()',\n    'email         text not null unique',\n    'is_admin      boolean not null default false',\n    'address       jsonb',\n    'payday        integer not null',\n    'CONSTRAINT unique_email UNIQUE(group_id, email)',\n    'check(payday \u003e 0 and payday \u003c 8)',\n)\n\n# CREATE TABLE accounts LIKE users\nawait db.schema('TABLE accounts').create(\n    'like users'\n)\n\n# CREATE TABLE IF NOT EXISTS accounts LIKE users\nawait db.schema('TABLE IF NOT EXISTS accounts').create(\n    'like users'\n)\n```\n\n#### Modify TABLE\n```python\n# ALTER TABLE users\n#   ALTER   id TYPE bigint,\n#   ALTER   name SET DEFAULT 'no_name',\n#   ALTER   COLUMN address DROP DEFAULT,\n#   ALTER   \"user info\" SET NOT NULL,\n#   ALTER   CONSTRAINT check(payday \u003e 1 and payday \u003c 6),\n#   ADD     UNIQUE(name, email) WITH (fillfactor=70),\n#   ADD     FOREIGN KEY (group_id) REFERENCES groups (id) ON DELETE SET NULL,\n#   DROP    CONSTRAINT IF EXISTS idx_email CASCADE\nawait db.schema('TABLE users').alter(\n    'alter  id TYPE bigint',\n    'alter  name SET DEFAULT \\'no_name\\'',\n    'alter  COLUMN address DROP DEFAULT',\n    'alter  \"user info\" SET NOT NULL',\n    'add    CONSTRAINT check(payday \u003e 1 and payday \u003c 6)',\n    'add    UNIQUE(name, email) WITH (fillfactor=70)',\n    'add    FOREIGN KEY (group_id) REFERENCES groups (id) ON DELETE SET NULL',\n    'drop   CONSTRAINT IF EXISTS idx_email CASCADE',\n)\n\n# ALTER TABLE users RENAME TO accounts\nawait db.schema('TABLE users').alter('RENAME TO accounts')\n\n# ALTER TABLE users RENAME email TO email_address\nawait db.schema('TABLE users').alter('RENAME email TO email_address')\n\n# ALTER TABLE users RENAME CONSTRAINT idx_name TO index_name\nawait db.schema('TABLE users').alter('RENAME CONSTRAINT idx_name TO index_name')\n\n# ALTER TABLE users ADD COLUMN address text\nawait db.schema('TABLE users').alter('ADD COLUMN address text')\n\n# ALTER TABLE users DROP address\nawait db.schema('TABLE users').alter('DROP address')\n\n# CREATE INDEX idx_email ON users (name, email)\nawait db.schema('INDEX idx_email ON users').create('name', 'email')\n\n# CREATE UNIQUE INDEX unique_name ON users(name) WHERE soft_deleted = FALSE\nawait db.schema('UNIQUE INDEX unique_name ON users').create('name',).where('soft_deleted', False)\n\n# DROP INDEX idx_email CASCADE\nawait db.schema('INDEX idx_email').drop('CASCADE')\n\n# DROP TABLE users\nawait db.schema('TABLE users').drop()\n```\n\n### Raw\nThe `raw` method can be used to execute any form of SQL. Usually the `raw` method is used to execute complex hard-coded (versus dynamically built) queries. It's also very common to use `raw` method to run migrations.\n\nThe input to `raw` method is not validated, so it is not safe from SQL injection.\n\n#### RAW for complex SQL\n```python\nawait db.raw('SELECT ROUND(AVG(group_id),1) AS avg_id, COUNT(1) AS total_users FROM users WHERE id in ($1, $2, $3)', 4, 5, 6)\n\nawait db.raw(\"SELECT * FROM (VALUES (1, 'one'), (2, 'two'), (3, 'three')) AS t (num, letter)\")\n\nawait db.raw(\"\"\"\n    INSERT INTO user (id, name)\n        SELECT $1, $2 WHERE NOT EXISTS (SELECT id FROM users WHERE id = $1)\n\"\"\", 1, 'Tom')\n```\n\n#### RAW for migration\n```python\nawait db.raw(\"\"\"\n    CREATE TABLE users(\n        id                       INT NOT NULL,\n        created_at               DATE NOT NULL,\n        first_name               VARCHAR(100) NOT NULL,\n        last_name                VARCHAR(100) NOT NULL,\n        birthday_mmddyyyy        CHAR(10) NOT NULL,\n    )\n\"\"\")\n```\n\n### WITH Clause using VALUES Lists\nThe Postgres [VALUES](https://www.postgresql.org/docs/12/queries-values.html) provides a way to generate a \"constant table\" from a list of values. Together with the [WITH](https://www.postgresql.org/docs/12/queries-with.html) clause, a small set of data can be loaded into the DB and queried like a table.\n\n#### SELECT using WITH VALUES\n```python\n# WITH \"my_values\" (\"text_col\", \"bool_col\", \"num_col\", \"dict_col\", \"datetime_col\", \"null_col\", \"null_col2\") AS\n#   (VALUES \n#     ('Tom', TRUE, 2, '{\"id\": 1}'::jsonb, '2021-07-20 10:00:00+00:00'::timestamptz, NULL, NULL)\n#   )\n# SELECT * FROM \"my_values\"\nresult = await db.with_values('my_values', {\n    'text_col': 'Tom',\n    'bool_col': True,\n    'num_col': 2,\n    'dict_col': {'id': 1},\n    'datetime_col': datetime.now(),\n    'null_col': 'null',\n    'null_col2': None\n}).table('my_values').select()\nresult[0]['text_col']      # 'Tom'\nresult[0]['bool_col']      # True\nresult[0]['num_col']       # 2\nresult[0]['dict_col']      # '{\"id\": 1}'\nresult[0]['datetime_col']  # datetime.datetime(2021, 7, 20, 10, 0, tzinfo=datetime.timezone.utc)\nresult[0]['null_col']      # None\nresult[0]['null_col2']     # None\n\n# join other tables\n# WITH \"workers\" (\"task_id\", \"name\") AS \n#   (VALUES \n#     (1, 'Tom'), \n#     (2, 'Jerry')\n#   ) \n# SELECT\n#   \"workers\".\"name\" AS \"worker_name\",\n#   \"tasks\".\"name\" AS \"task_name\"\n# FROM \"workers\"\n# JOIN \"tasks\" ON \"workers\".\"task_id\" = \"tasks\".\"id\"\nawait db.with_values('workers', {\n    'task_id': 1,\n    'name': 'Tom'\n}, {\n    'task_id': 2,\n    'name': 'Jerry'\n}).table('workers').select(\n    'workers.name AS worker_name',\n    'tasks.name AS task_name'\n).join('tasks', 'workers.task_id = tasks.id').order_by('tasks.id')\n\n# multiple WITH VALUES\n# WITH \"workers1\" (\"task_id\", \"name\") AS\n#   (VALUES\n#     (1, 'Tom'),\n#     (2, 'Jerry')\n#   ), \"workers2\" (\"task_id\", \"name\") AS\n#   (VALUES\n#     (1, 'Topsy'), \n#     (2, 'Nibbles')\n#   )\n# SELECT\n#   \"workers1\".\"name\" AS \"primary_worker_name\",\n#   \"workers2\".\"name\" AS \"secondary_worker_name\",\n#   \"tasks\".\"name\" AS \"task_name\"\n# FROM \"tasks\"\n# JOIN \"workers1\" ON \"workers1\".\"task_id\" = \"tasks\".\"id\"\n# JOIN \"workers2\" ON \"workers2\".\"task_id\" = \"tasks\".\"id\"\nawait db.with_values('workers1', {\n    'task_id': 1,\n    'name': 'Tom'\n}, {\n    'task_id': 2,\n    'name': 'Jerry'\n}).with_values('workers2', {\n    'task_id': 1,\n    'name': 'Topsy'\n}, {\n    'task_id': 2,\n    'name': 'Nibbles'\n}).table('tasks').select(\n    'workers1.name AS primary_worker_name',\n    'workers2.name AS secondary_worker_name',\n    'tasks.name AS task_name'\n).join('workers1', 'workers1.task_id = tasks.id').\\\n    join('workers2', 'workers2.task_id = tasks.id')\n```\n\n#### UPDATE using WITH VALUES\n```python\n# WITH \"workers\" (\"task_id\", \"name\") AS\n#   (VALUES\n#     (1, 'Tom'), \n#     (2, 'Jerry')\n#   )\n# UPDATE \"tasks\" \n# SET\n#   \"name\" = \"tasks\".\"name\" || ' (worked by ' || \"workers\".\"name\" || ')'\n# FROM \"workers\"\n# WHERE\n#   \"workers\".\"task_id\" = \"tasks\".\"id\"\n# RETURNING\n#   \"workers\".\"name\" AS \"worker_name\",\n#   \"tasks\".\"name\" AS \"task_name\"\nawait db.with_values('workers', {\n    'task_id': 1,\n    'name': 'Tom'\n}, {\n    'task_id': 2,\n    'name': 'Jerry'\n}).table('tasks').update(\"name = tasks.name || ' (worked by ' || workers.name || ')'\").\\\n    from_table('workers').\\\n    where('workers.task_id = tasks.id').\\\n    returning(\n        'workers.name AS worker_name',\n        'tasks.name AS task_name'\n    )\n```\n\n#### RAW using WITH VALUES\n```python\n# WITH \"workers\" (\"task_id\", \"name\") AS\n#   (VALUES\n#     (1, 'Tom'), \n#     (2, 'Jerry')\n#   )\n# SELECT * FROM tasks WHERE EXISTS(\n#   SELECT 1 FROM workers\n#   JOIN task_results ON workers.task_id = task_results.task_id\n#   WHERE workers.task_id = tasks.id\n# )\nawait db.with_values('workers', {\n    'task_id': 1,\n    'name': 'Tom'\n}, {\n    'task_id': 2,\n    'name': 'Jerry'\n}).raw(\"\"\"\nSELECT * FROM tasks WHERE EXISTS(\n    SELECT 1 FROM workers\n    JOIN task_results ON workers.task_id = task_results.task_id\n    WHERE workers.task_id = tasks.id\n)\n\"\"\")\n```\n\n### JSONB examples\nMethods are created to support jsonb data type for some simple use cases.\n\n#### Create a table with jsonb data type\n```python\n# CREATE TABLE users (\n#    id     serial PRIMARY KEY,\n#    data   jsonb\n#)\nawait db.schema('TABLE users').create(\n    'id     serial PRIMARY KEY',\n    'data   jsonb',\n)\n```\n\n#### Select jsonb field\n```python\n# SELECT data-\u003ename AS name, data-\u003e\u003ename AS name_text FROM users\nrows = await db.table('users').select('data', 'data-\u003ename AS name', 'data-\u003e\u003ename AS name_text')\n# rows[0]['data'] == '{\"name\":\"Tom\"}'\n# rows[0]['name'] == '\"Tom\"'\n# rows[0]['name_text'] == 'Tom'\n\n# SELECT data-\u003ename AS name FROM users WHERE data-\u003e\u003ename LIKE 'Tom%'\nawait db.table('users').select('data-\u003ename AS name').where('data-\u003e\u003ename', 'LIKE', 'Tom%')\n\n# SELECT data-\u003ename AS name FROM users WHERE data-\u003ename = '\"Tom\"'\nawait db.table('users').select('data-\u003ename AS name').where(\"data-\u003ename\", 'Tom')\n```\n\n#### Insert jsonb field\n```python\n# INSERT INTO users (data) VALUES\n#   ('{\"name\": \"Tom\"}'),\n#   ('{\"name\": \"Jerry\"}')\n#   RETURNING *\nawait db.table('users').insert(\n    {'data': {'name': 'Tom'}},\n    {'data': {'name': 'Jerry'}},\n).returning()\n```\n\n#### Update jsonb field\n```python\n# UPDATE SET data = '{\"address\": {\"city\": \"New York\"}}'\nawait db.table('users').update({'data': {'address': {'city': 'New York'}}})\n\n# UPDATE SET data = jsonb_set(data, '{address,city}', '\"Chicago\"')\nawait db.table('users').update({'data-\u003eaddress-\u003ecity': 'Chicago'})\n```\n\n### Migrations\nWindyquery has a preliminary support for database migrations. The provided command-line script is called `wq`. \n\n#### Generate a migration file\nA migration file can be created by,\n```bash\n# this creates a timestamped migration file, e.g. \"20210705233408_create_my_table.py\"\n$ wq make_migration --name=create_my_table\n```\n\nBy default, the new file is add to `database/migrations/` under the current working directory. If the diretory does not exist, it will be created first. The file contains an empty function to be filled by the user,\n```python\nasync def run(db):\n    # TODO: add code here\n    pass\n```\n\nSome sample migration templates are provided at [here](https://github.com/bluerelay/windyquery/blob/master/windyquery/scripts/migration_templates.py). They can be automatically inserted in the generated file by specifying the `--template` parameter,\n```bash\n# the generated file is pre-filled with some code template,\n# async def run(db):\n#     await db.schema('TABLE my_table').create(\n#         'id      serial PRIMARY KEY',\n#         'name    text not null unique',\n#     )\n$ wq make_migration --name=create_my_table --template=\"create table\"\n\n# create a migration file that contains all avaiable templates\n$ wq make_migration --name=create_my_table --template=all\n```\n\n#### Run migrations\nTo run all of the outstanding migrations, use the `migrate` sub-command,\n```bash\n$ wq migrate --host=localhost --port=5432 --database=my-db --username=my-name --password=my-pass\n\n# alternatively, the DB config can be provided by using environment variables\n$ DB_HOST=localhost DB_PORT=5432 DB_DATABASE=my-db DB_USERNAME=my-name DB_PASSWORD=my-pass wq migrate\n```\n\n#### Use custom directory and database table\nThe `wq` command requires a directory to save the migration files, and a database table to store executed migrations. By default, the migration directory is `database/migrations/` under the current working directroy, and the database table is named `migrations`. They are created automatically if they do not already exist.\nThe directory and table name can be customized by using `--migration_dir` and `--migration_table` parameters,\n```bash\n# creates the migrations file in \"my_db_work/migrations/\" of the current directory\n$ wq make_migration --name=create_my_table --migrations_dir=\"my_db_work/migrations\"\n\n# looks for outstanding migrations in \"my_db_work/migrations/\" and stores finished migrations in my_migrations table in DB\n$ wq migrate --host=localhost --port=5432 --database=my-db --username=my-name --password=my-pass --migrations_dir=\"my_db_work/migrations\" --migrations_table=my_migrations\n```\n\n### Syntax checker\nA very important part of windyquery is to validate the inputs of the various builder methods. It defines a [Validator](https://github.com/bluerelay/windyquery/blob/master/windyquery/validator/__init__.py) class, which is used to reject input strings not following the proper syntax.\nAs a result, it can be used separately as a syntax checker for other DB libraries. For example, it is very common for REST API to support filtering or searching parameters specified by the users,\n```python\n......\n# GET /example-api/users?name=Tom\u0026state=AZ;DROP%20TABLE%20Students\nurl_query = \"name=Tom\u0026state=AZ;DROP TABLE Students\"\nwhere = url_query.replace(\"\u0026\", \" AND \")\n\nfrom windyquery.validator import Validator\nfrom windyquery.validator import ValidationError\nfrom windyquery.ctx import Ctx\n\ntry:\n    ctx = Ctx()\n    validator = Validator()\n    where = validator.validate_where(where, ctx)\nexcept ValidationError:\n    abort(400, f'Invalid query parameters: {url_query}')\n\nconnection = psycopg2.connect(**dbConfig)\ncursor = connection.cursor()\ncursor.execute(f'SELECT * FROM users WHERE {where}')\n......\n```\nPlease note,\n- Except `raw`, all windyquery's own builder methods, such as `select`, `update`, `where`, and so on, already implicitly use these validation functions. They may be useful when used alone, for example, to help other DB libraries validate SQL snippets;\n- These validation functions only cover a very small (though commonly used) subset of SQL grammar of Postgres.\n\n### Listen for a notification\nPostgres implements [LISTEN/NOTIFY](https://www.postgresql.org/docs/12/sql-listen.html) for interprocess communications.\nIn order to listen on a channel, use the DB.listen() method. It returns an awaitable object, which resolves to a dict when a notification fires.\n```python\nfrom windyquery.exceptions import ListenConnectionClosed\n\n# method 1: manually call start() and stop()\nlistener = db.listen('my_table')\nawait listener.start()\ntry:\n    for _ in range(100):\n        result = await listener\n        # or result = await listener.next()\n        print(result) \n        # {\n        #     'channel': 'my_table',\n        #     'payload': 'payload fired by the notifier',\n        #     'listener_pid': 7321,\n        #     'notifier_pid': 7322\n        # }\nexcept ListenConnectionClosed as e:\n    print(e)\nfinally:\n    await listener.stop()\n\n# method 2: use with statement\nasync with db.listen('my_table') as listener:\n    for _ in range(100):\n        result = await listener\n        print(result)\n```\n\n### RRULE\nWindyquery has a rrule function that can \"expand\" a rrule string into it occurrences (a list of datetimes) by using [dateutil](https://github.com/dateutil/dateutil). A values CTE is prepared from the rrule occurrences, which can be further used by other querries.\n\n#### A simple rrule example\n```python\nrruleStr = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;COUNT=5\n\"\"\"\n\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-03-03 10:00:00+00:00'::timestamptz),\n#   ('2021-03-04 10:00:00+00:00'::timestamptz),\n#   ('2021-03-05 10:00:00+00:00'::timestamptz),\n#   ('2021-03-06 10:00:00+00:00'::timestamptz),\n#   ('2021-03-07 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr}).table('my_rrules').select()\n```\n\n#### More than one rrules\n```python\nrruleStr1 = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;COUNT=5\n\"\"\"\n\nrruleStr2 = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;INTERVAL=10;COUNT=3\nRRULE:FREQ=DAILY;INTERVAL=5;COUNT=3\n\"\"\"\n\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-03-03 10:00:00+00:00'::timestamptz),\n#   ('2021-03-04 10:00:00+00:00'::timestamptz),\n#   ('2021-03-05 10:00:00+00:00'::timestamptz),\n#   ('2021-03-06 10:00:00+00:00'::timestamptz),\n#   ('2021-03-07 10:00:00+00:00'::timestamptz),\n#   ('2021-03-03 10:00:00+00:00'::timestamptz),\n#   ('2021-03-08 10:00:00+00:00'::timestamptz),\n#   ('2021-03-13 10:00:00+00:00'::timestamptz),\n#   ('2021-03-23 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\n)\nawait db.rrule('my_rrules', {\n        'rrule': rruleStr1\n    }, {\n        'rrule': rruleStr2\n    }).table('my_rrules').select()\n\n# the rrule field can also take a list of mulitple rrules.\n# the previous example is equivalent to\nawait db.rrule('my_rrules', {\n        'rrule': [rruleStr1, rruleStr2]\n    }).table('my_rrules').select()\n```\n\n#### Use exrule\n```python\nrruleStr = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;COUNT=5\n\"\"\"\n\nexruleStr = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;BYWEEKDAY=SA,SU\n\"\"\"\n\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-03-03 10:00:00+00:00'::timestamptz),\n#   ('2021-03-04 10:00:00+00:00'::timestamptz),\n#   ('2021-03-05 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'exrule': exruleStr}).table('my_rrules').select()\n```\n\n#### Use rdate\n```python\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-05-03 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rdate': '20210503T100000Z'}).table('my_rrules').select()\n\nrruleStr = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;COUNT=5\n\"\"\"\n\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-03-03 10:00:00+00:00'::timestamptz),\n#   ('2021-03-04 10:00:00+00:00'::timestamptz),\n#   ('2021-03-05 10:00:00+00:00'::timestamptz),\n#   ('2021-03-06 10:00:00+00:00'::timestamptz),\n#   ('2021-03-07 10:00:00+00:00'::timestamptz),\n#   ('2021-05-03 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rdate': '20210503T100000Z'}).table('my_rrules').select()\n\n# similary to rrule, the rdate field can take a list of date strings\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-03-03 10:00:00+00:00'::timestamptz),\n#   ('2021-03-04 10:00:00+00:00'::timestamptz),\n#   ('2021-03-05 10:00:00+00:00'::timestamptz),\n#   ('2021-03-06 10:00:00+00:00'::timestamptz),\n#   ('2021-03-07 10:00:00+00:00'::timestamptz),\n#   ('2021-05-03 10:00:00+00:00'::timestamptz),\n#   ('2021-06-03 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rdate': ['20210503T100000Z','20210603T100000Z']}).table('my_rrules').select()\n```\n\n#### Use exdate\n```python\nrruleStr = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;COUNT=5\n\"\"\"\n\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-03-03 10:00:00+00:00'::timestamptz),\n#   ('2021-03-05 10:00:00+00:00'::timestamptz),\n#   ('2021-03-06 10:00:00+00:00'::timestamptz),\n#   ('2021-03-07 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'exdate': '20210304T100000Z'}).table('my_rrules').select()\n\n# similary to rrule, the exdate field can take a list of date strings\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-03-03 10:00:00+00:00'::timestamptz),\n#   ('2021-03-05 10:00:00+00:00'::timestamptz),\n#   ('2021-03-07 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'exdate': ['20210304T100000Z','20210306T100000Z']}).table('my_rrules').select()\n```\n\n#### Use after, before, and between\n```python\nrruleStr = \"\"\"\nDTSTART:20210715T100000Z\nRRULE:FREQ=DAILY;COUNT=5\n\"\"\"\n\n# rrule_after returns the first recurrence after the given datetime dt.\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-07-17 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rrule_after': {'dt': '20210716T100000Z'}}]}).table('my_rrules').select()\n\n# if the inc keyword is True dt is included if it is an occurrence.\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-07-16 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rrule_after': {'dt': '20210716T100000Z', 'inc': True}}]}).table('my_rrules').select()\n\n# rrule_before returns the last recurrence before the given datetime dt.\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-07-15 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rrule_before': {'dt': '20210716T100000Z'}}]}).table('my_rrules').select()\n\n# if the inc keyword is True dt is included if it is an occurrence.\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-07-16 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rrule_before': {'dt': '20210716T100000Z', 'inc': True}}]}).table('my_rrules').select()\n\n# rrule_between returns all the occurrences of the rrule between after and before.\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-07-17 10:00:00+00:00'::timestamptz)\n#   ('2021-07-18 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rrule_between': {'after': '20210716T100000Z', 'before': '20210719T100000Z'}}]}).table('my_rrules').select()\n\n# if the inc keyword is True after and/or before are included if they are occurrences.\n# WITH my_rrules (\"rrule\") AS \n# (VALUES\n#   ('2021-07-16 10:00:00+00:00'::timestamptz)\n#   ('2021-07-17 10:00:00+00:00'::timestamptz)\n#   ('2021-07-18 10:00:00+00:00'::timestamptz)\n#   ('2021-07-19 10:00:00+00:00'::timestamptz)\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rrule_between': {'after': '20210716T100000Z', 'before': '20210719T100000Z', 'inc': True}}]}).table('my_rrules').select()\n```\n\n#### Join rrule with other tables\n```python\nimport datetime\n\nrruleStr1 = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;COUNT=5\n\"\"\"\n\nrruleStr2 = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;INTERVAL=10;COUNT=3\nRRULE:FREQ=DAILY;INTERVAL=5;COUNT=3\n\"\"\"\n\n# WITH task_rrules (\"task_id\", \"rrule\") AS \n# (VALUES\n#   (1, '2021-03-03 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-04 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-05 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-06 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-07 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-03 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-08 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-13 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-23 10:00:00+00:00'::timestamptz)\n# )\n# SELECT task_rrules.rrule, tasks.name\n# FROM task_rrules\n# JOIN tasks ON tasks.id = task_rrules.task_id\n# WHERE\n#   rrule \u003e '2021-03-05 10:00:00+00:00' AND\n#   rrule \u003c '2021-03-08 10:00:00+00:00'\nawait db.rrule('task_rrules', {\n        'task_id': 1, 'rrule': rruleStr1\n    }, {\n        'task_id': 2, 'rrule': rruleStr2\n    }).table('task_rrules').\n    join('tasks', 'tasks.id', '=', 'task_rrules.task_id').\n    where('rrule \u003e ? AND rrule \u003c ?',\n        datetime.datetime(2021, 3, 5, 10, 0,\n                tzinfo=datetime.timezone.utc),\n        datetime.datetime(2021, 3, 8, 10, 0,\n                tzinfo=datetime.timezone.utc),\n    ).select('task_rrules.rrule', 'tasks.name')\n```\n\n#### Using rrule in update\n```python\nimport datetime\n\nrruleStr1 = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;COUNT=5\n\"\"\"\n\nrruleStr2 = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;INTERVAL=10;COUNT=3\nRRULE:FREQ=DAILY;INTERVAL=5;COUNT=3\n\"\"\"\n\n# WITH task_rrules (\"task_id\", \"rrule\") AS \n# (VALUES\n#   (1, '2021-03-03 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-04 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-05 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-06 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-07 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-03 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-08 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-13 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-23 10:00:00+00:00'::timestamptz)\n# )\n# UPDATE tasks SET result = 'done'\n# FROM task_rrules\n# WHERE task_rrules.task_id = tasks.id\nawait db.rrule('task_rrules', {\n        'task_id': 1, 'rrule': rruleStr1\n    }, {\n        'task_id': 2, 'rrule': rruleStr2\n    }).table('tasks').update(\"result = 'done'\").\n    from_table('task_rrules').\n    where('task_rrules.task_id = tasks.id')\n```\n\n#### Using rrule with raw method\n```python\nimport datetime\n\nrruleStr1 = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;COUNT=5\n\"\"\"\n\nrruleStr2 = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY;INTERVAL=10;COUNT=3\nRRULE:FREQ=DAILY;INTERVAL=5;COUNT=3\n\"\"\"\n\n# WITH task_rrules (\"task_id\", \"rrule\") AS \n# (VALUES\n#   (1, '2021-03-03 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-04 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-05 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-06 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-07 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-03 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-08 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-13 10:00:00+00:00'::timestamptz),\n#   (2, '2021-03-23 10:00:00+00:00'::timestamptz)\n# )\n# DELETE FROM tasks\n# WHERE EXISTS(\n#   SELECT 1 FROM task_rrules\n#   WHERE\n#     task_id = tasks.id AND\n#     rrule \u003e '2021-03-20 10:00:00+00:00'\n# )\n# RETURNING id, task_id\nawait db.rrule('task_rrules', {\n        'task_id': 1, 'rrule': rruleStr1\n    }, {\n        'task_id': 3, 'rrule': rruleStr2\n    }).raw(\"\"\"\n        DELETE FROM tasks\n        WHERE EXISTS(\n            SELECT 1 FROM task_rrules\n            WHERE \n                task_id = tasks.id AND\n                rrule \u003e $1\n        )\n        RETURNING id, task_id\n    \"\"\", datetime.datetime(2021, 3, 20, 10, 0,\n                tzinfo=datetime.timezone.utc))\n```\n\n#### Using a slice to limit the occurrences\n```python\nimport datetime\n\nrruleStr = \"\"\"\nDTSTART:20210303T100000Z\nRRULE:FREQ=DAILY\n\"\"\"\n\n# WITH my_rrules (\"task_id\", \"rrule\") AS \n# (VALUES\n#   (1, '2021-03-03 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-04 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-05 10:00:00+00:00'::timestamptz),\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rrule_slice': slice(3)}).table('my_rrules').select()\n\n# WITH my_rrules (\"task_id\", \"rrule\") AS \n# (VALUES\n#   (1, '2021-03-13 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-15 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-17 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-19 10:00:00+00:00'::timestamptz),\n#   (1, '2021-03-21 10:00:00+00:00'::timestamptz),\n# )\n# SELECT * FROM my_rrules\nawait db.rrule('my_rrules', {'rrule': rruleStr, 'rrule_slice': slice(10,20,2)}).table('my_rrules').select()\n```\n\n\n### Tests\nWindyquery includes [tests](https://github.com/bluerelay/windyquery/tree/master/windyquery/tests). These tests are also served as examples on how to use this library.\n\n#### Running tests\nInstall pytest to run the included tests,\n```bash\npip install -U pytest\n```\n\nSet up a postgres server with preloaded data. This can be done by using [docker](https://docs.docker.com/install/) with the [official postgre docker image](https://hub.docker.com/_/postgres),\n```bash\ndocker run --rm --name windyquery-test -p 5432:5432 -v ${PWD}/windyquery/tests/seed_test_data.sql:/docker-entrypoint-initdb.d/seed_test_data.sql -e POSTGRES_USER=windyquery-test -e POSTGRES_PASSWORD=windyquery-test -e POSTGRES_DB=windyquery-test -d postgres:12-alpine\n```\n\nNote: to use existing postgres server, it must be configured to have the correct user, password, and database needed in tests/conftest.py. Data needed by tests is in tests/seed_test_data.sql.\n\nTo run the tests,\n```bash\npytest\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbluerelay%2Fwindyquery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbluerelay%2Fwindyquery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbluerelay%2Fwindyquery/lists"}