{"id":19574070,"url":"https://github.com/acode/keyql","last_synced_at":"2025-04-27T05:33:40.397Z","repository":{"id":42206856,"uuid":"188716357","full_name":"acode/KeyQL","owner":"acode","description":"A language and specification for building data queries using key-value pairs","archived":false,"fork":false,"pushed_at":"2022-12-30T17:43:18.000Z","size":871,"stargazers_count":59,"open_issues_count":6,"forks_count":6,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-11-01T20:36:21.070Z","etag":null,"topics":["api","faas","keyql"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/acode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-26T18:10:35.000Z","updated_at":"2024-10-16T08:49:42.000Z","dependencies_parsed_at":"2023-01-31T12:31:11.310Z","dependency_job_id":null,"html_url":"https://github.com/acode/KeyQL","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acode%2FKeyQL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acode%2FKeyQL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acode%2FKeyQL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acode%2FKeyQL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/acode","download_url":"https://codeload.github.com/acode/KeyQL/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224062258,"owners_count":17249274,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","faas","keyql"],"created_at":"2024-11-11T06:38:00.657Z","updated_at":"2024-11-11T06:38:02.184Z","avatar_url":"https://github.com/acode.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# KeyQL\n\n![KeyQL Logo](/images/keyql-logo.png)\n\n![travis-ci build](https://travis-ci.org/FunctionScript/KeyQL.svg?branch=master)\n![npm version](https://badge.fury.io/js/keyql.svg)\n\nKeyQL is a language, specification and package for querying datasets using key-value pairs.\nIt is heavily inspired by the simplicity and ease-of-use of [Django](https://djangoproject.com)\nand similar ORMs.\nThe provided Node.js package can be used to filter large JSON datasets from within any\ncodebase, but the primary purpose of KeyQL is to be used with [FunctionScript](https://github.com/acode/FunctionScript)\nAPIs, where JSON or HTTP Query Parameter key-value pairs can be used to encode\nquery requests to underlying datasets.\n\nKeyQL is meant for easy querying of JSON datasets, spreadsheet data,\ninformation retrieved from APIs such as [Airtable](https://airtable.com) and\nmore. It can be used to add robust querying capabilities to existing APIs\nwithout a massive architectural lift and shift.\n\nThe motivation for KeyQL differs from that of GraphQL. KeyQL is intended to provide a\nsimple querying interface to existing imperative APIs and relatively flat\ndatasets. The operators (comparators) are the most important feature and are\nmeant to be easily interpretable by even the newest developer. KeyQL and GraphQL\ncan, in theory, coexist within a single codebase or API implementation.\nKeyQL is **not intended** to be used to define an entire backend architecture\nand provides no opinions on the graph-based structure of output data\n(you do not define schemas with it).\n\n# Quick Example\n\nA quick example of using KeyQL with a [FunctionScript](https://github.com/FunctionScript/FunctionScript)\nAPI would look like:\n\n**Filename:** `/dataset.json`\n```json\n[\n  {\n    \"id\": 1,\n    \"fields\": {\n      \"name\": \"Alice\",\n      \"birthdate\": \"12/01/1988\",\n      \"pets\": 2\n    }\n  },\n  {\n    \"id\": 2,\n    \"fields\": {\n      \"name\": \"Bernard\",\n      \"birthdate\": \"11/11/1972\",\n      \"pets\": 5\n    }\n  },\n  {\n    \"id\": 3,\n    \"fields\": {\n      \"name\": \"Christine\",\n      \"birthdate\": \"01/05/1991\",\n      \"pets\": 0\n    }\n  }\n]\n```\n\n**Filename:** `/functions/__main__.js`\n```javascript\nconst KeyQL = require('keyql');\nconst dataset = require('../dataset.json');\n// Searching through the \"fields\" object in each row\nconst kqlDataset = new KeyQL(dataset, row =\u003e row.fields);\n\n/**\n* Query a dataset based on an Array of Objects\n* @param {object.keyql.query} where A list of fields to query for\n* @returns {array} result The result list\n*/\nmodule.exports = async (where = {}) =\u003e {\n\n  return kqlDataset.query()\n    .select([where]) // Wrap in array if provided a raw object\n    .values();\n\n};\n```\n\nAn HTTP POST request containing:\n\n```json\n{\n  \"where\": {\n    \"pets__gt\": 3\n  }\n}\n```\n\nWould return:\n\n```json\n[\n  {\n    \"id\": 2,\n    \"fields\": {\n      \"name\": \"Bernard\",\n      \"birthdate\": \"11/11/1972\",\n      \"pets\": 5\n    }\n  }\n]\n```\n\n# Table of Contents\n\n1. [Introduction](#introduction)\n1. [Specification](#specification)\n   1. [Writing Queries](#writing-queries)\n   1. [Supported Operators](#supported-operators)\n1. [Installation and Usage](#installation-and-usage)\n   1. [Methods](#methods)\n1. [Comparison to GraphQL](#comparison-to-graphql)\n1. [Acknowledgements](#acknowledgements)\n   1. [Roadmap](#roadmap)\n\n# Introduction\n\nBy adhering to the KeyQL specification, your developers and users will have a\nsignificantly easier time learning how to work with your APIs and datasets.\n\nFor example, you may have an HTTP API built on [Autocode](https://autocode.com/).\nIn this project, you have set up a `users/select` endpoint and want to query your\nuser dataset for **every username that contains `\"ke\"`** in a case-insensitive\nfashion.\n\n```\nHTTP POST https://$user.api.stdlib.com/project@dev/users/select\n{\n  \"query\": {\n    \"username__icontains\": \"ke\"\n  }\n}\n```\n\nWith the intended response being something like:\n```json\n[\n  {\n    \"username\": \"Kelly\",\n    \"profile_image\": \"boop.jpg\"\n  },\n  {\n    \"username\": \"Kevin\",\n    \"profile_image\": \"snoot.jpg\"\n  }\n]\n```\n\nThe KeyQL specification removes the cognitive overhead of choosing how to\nstructure your query requests.\n\n# Specification\n\nThe KeyQL specification is heavily inspired by [Django](https://www.djangoproject.com/)'s ORM\nand over five years of work manipulating datasets on both the front and back-end\nof web projects, primarily working with JSON and SQL queries. It's the culmination\nof best practices learned implementing [DataCollection.js](https://github.com/keithwhor/DataCollection.js) and\n[Nodal](https://github.com/keithwhor/nodal)'s ORM.\n\n## Writing Queries\n\nWriting KeyQL Queries is as simple as preparing a JSON Object. For example,\nin a dataset that has records that look like...\n\n```javascript\n// Example dataset in JavaScript\n[\n  {\n    first_name: 'Dolores',\n    last_name: 'Abernathy',\n    is_host: true,\n    eye_color: 'blue',\n    hair_color: 'blonde',\n    location_in_park: null,\n    age: 250\n  }\n]\n```\n\nYou could write a query against it that returns...\n\n### Query: All entries with `first_name` = `Dolores`\n\n```json\n[\n  {\n    \"first_name\": \"Dolores\"\n  }\n]\n```\n\n### Query: `first_name` = `Dolores` AND `eye_color` in `blue`, `green`\n\n```json\n[\n  {\n    \"first_name\": \"Dolores\",\n    \"eye_color__in\": [\"blue\", \"green\"]\n  }\n]\n```\n\n### Query: `first_name` = `Dolores` OR `first_name` = `Teddy`\n\n```json\n[\n  {\n    \"first_name\": \"Dolores\"\n  },\n  {\n    \"first_name\": \"Teddy\"\n  }\n]\n```\n\n## Supported Operators\n\nAll operators in KeyQL queries are preceded by a `__` delimiter. To reiterate\nfrom the previous section, this means you can query the field `first_name` with;\n\n```javascript\n\"first_name\" // (default to \"is\" operator)\n\"first_name__is\"\n\"first_name__startswith\"\n\"first_name__gte\"\n```\n\n### Full List of Supported Operators\n\nThe following table assumes that `queryValue` is the value you're searching for\nprovided a specified key, and `entryValue` is the matching entry in a dataset.\n\n| Operator | Behavior |\n| -------- | -------- |\n| is | Finds all matching entries. Returns `entryValue === queryValue` (exact match, type included). |\n| not | Finds all non-matching entries. Returns `entryValue !== queryValue` (exact match, type included). |\n| gt | Finds all entries **greater than** specified value. Returns `entryValue \u003e queryValue`. |\n| gte | Finds all entries **greater than or equal to** specified value. Returns `entryValue \u003e= queryValue`. |\n| lt | Finds all entries **less than** specified value. Returns `entryValue \u003c queryValue`. |\n| lte | Finds all entries **less than or equal to** specified value. Returns `entryValue \u003c= queryValue`. |\n| contains | Finds all entries **containing** the **exact** provided value. Works when `entryValue` is a `string` or an `array`. |\n| icontains | Finds all entries **containing** the provided value, **case-insensitive**. Works when `entryValue` is a `string` or an `array`. |\n| startswith | Finds all entries **starting with** the **exact** provided value. Works when `entryValue` is a `string`.\n| istartswith | Finds all entries **starting with** the provided value, **case-insensitive**. Works when `entryValue` is a `string`. |\n| endswith | Finds all entries **ending with** the **exact** provided value. Works when `entryValue` is a `string`. |\n| iendswith | Finds all entries **ending with** the provided value, **case-insensitive**. Works when `entryValue` is a `string`. |\n| is_null | Finds all entries where `entryValue === null`, **`queryValue` is ignored**. |\n| is_true | Finds all entries where `entryValue === true`, **`queryValue` is ignored**. |\n| is_false | Finds all entries where `entryValue === false`, **`queryValue` is ignored**. |\n| not_null | Finds all entries where `entryValue !== null`, **`queryValue` is ignored**. |\n| not_true | Finds all entries where `entryValue !== true`, **`queryValue` is ignored**. |\n| not_false | Finds all entries where `entryValue !== false`, **`queryValue` is ignored**. |\n| in | Finds all entries **within** the provided value, intended to match when `queryValue` is an `array` but works with `string` input. |\n| not_in | Finds all entries **not in** the provided value, intended to match when `queryValue` is an `array` but works with `string` input. |\n| recency_lt | Finds all entries where `DATE(entryValue)` is recent within less than `queryValue` in number of seconds. i.e. `\"field__recency__lt\": 3600` would look for entries that have `field` as a date/timestamp within the past hour (exclusive). ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| recency_lte | Finds all entries where `DATE(entryValue)` is recent within less than or equal to `queryValue` in number of seconds. i.e. `\"field__recency__lte\": 3600` would look for entries that have `field` as a date/timestamp within the past hour (inclusive). ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| recency_gt | Finds all entries where `DATE(entryValue)` has a recency greater than `queryValue` in number of seconds. i.e. `\"field__recency__gt\": 3600` would look for entries that have `field` as a date/timestamp outside the past hour (exclusive). ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| recency_gte | Finds all entries where `DATE(entryValue)` has a recency greater than or equal to `queryValue` in number of seconds. i.e. `\"field__recency__gte\": 3600` would look for entries that have `field` as a date/timestamp outside the past hour (inclusive). ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| upcoming_lt | Finds all entries where `DATE(entryValue)` is going to occur within less than `queryValue` in number of seconds. i.e. `\"field__upcoming_lt\": 3600` would look for entries that have `field` as a date/timestamp within the next hour (exclusive). ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| upcoming_lte | Finds all entries where `DATE(entryValue)` is going to occur within less than or equal to `queryValue` in number of seconds. i.e. `\"field__upcoming_lte\": 3600` would look for entries that have `field` as a date/timestamp within the next hour (inclusive). ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| upcoming_gt | Finds all entries where `DATE(entryValue)` is going to occur within greater than `queryValue` in number of seconds. i.e. `\"field__upcoming_gt\": 3600` would look for entries that have `field` as a date/timestamp outside the next hour (exclusive). ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| upcoming_gte | Finds all entries where `DATE(entryValue)` is going to occur within greater than or equal to `queryValue` in number of seconds. i.e. `\"field__upcoming_gte\": 3600` would look for entries that have `field` as a date/timestamp outside the next hour (inclusive). ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| date_lt | Finds all entries where `DATE(entryValue)` is less than `DATE(queryValue)`, i.e. '12-06-1988' \u003c '01-01-2019'.  ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| date_lte | Finds all entries where `DATE(entryValue)` is less than or equal to `DATE(queryValue)`, i.e. '12-06-1988' \u003c= '12-06-1988'. ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| date_gt | Finds all entries where `DATE(entryValue)` is greater than `DATE(queryValue)`, i.e. '12-06-1988' \u003e '01-01-1980'. ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n| date_gte | Finds all entries where `DATE(entryValue)` is greater than or equal to `DATE(queryValue)`, i.e. '12-06-1988' \u003e= '12-06-1988'. ISO8601 Timestamps suggested, if no timezone entered UTC will be assumed. |\n\n# Installation and Usage\n\nThe KeyQL implementation provided as part of this GitHub repository is intended\nfor use in the Node.js ecosystem, using the package `keyql`. Right now, it can\nbe used to automatically filter JSON datasets (arrays of objects) based on a specified\nquery. In the future, we intend to migrate the [Nodal](https://github.com/keithwhor/nodal) Query Composer (ORM)\nto be able to automatically generate SQL queries from a provided KeyQL statement.\n\nYou can install the package simply using [Node.js](https://nodejs.org) (10 or higher) and NPM:\n\n```shell\n$ npm i keyql --save\n```\n\nAnd use it in your Node.js project with:\n\n```javascript\nconst KeyQL = require('keyql');\n\nlet dataset = [/* my dataset */]; // Your array of objects\n\nconst myDataset = new KeyQL(dataset);\nmyDataset.query()\n  .select({value__gte: 5})\n  .values(); // gets all records where \"value\" is \u003e 5\n```\n\n## Node.js Examples\n\n```javascript\nconst KeyQL = require('keyql');\n\nconst kqlDataset = new KeyQL(\n  [\n    {\n      \"id\": 1,\n      \"fields\": {\n        \"name\": \"Alice\",\n        \"birthdate\": \"12/01/1988\",\n        \"pets\": 2\n      }\n    },\n    {\n      \"id\": 2,\n      \"fields\": {\n        \"name\": \"Bernard\",\n        \"birthdate\": \"11/11/1972\",\n        \"pets\": 5\n      }\n    },\n    {\n      \"id\": 3,\n      \"fields\": {\n        \"name\": \"Christine\",\n        \"birthdate\": \"01/05/1991\",\n        \"pets\": 0\n      }\n    }\n  ],\n  row =\u003e row.fields // mapping\n);\n\n// Basic query\nkqlDataset.query()\n  .select([{pets__gt: 3}])\n  .values(); // gives Bernard\n\n// OR query is adding additional array elements\nkqlDataset.query()\n  .select([{pets__gt: 3}, {pets__lt: 2}])\n  .values(); // gives Bernard, Christine\n\n// AND query is additional parameters in a single object\nkqlDataset.query()\n  .select([{name__in: ['Bernard', 'Christine'], pets: 5}])\n  .values(); // gives Bernard\n\n// Chaining will continue to filter datasets\nkqlDataset.query()\n  .select([{name__in: ['Bernard', 'Christine']}])\n  .select([{pets__lt: 5})\n  .values(); // gives Christine\n\n// Updating can modify the base dataset\nkql.query()\n  .select([{birthdate__date_gt: '01-01-1980'}])\n  .update({pets: 100}); // Sets Alice and Christine to have 100 pets\n```\n\n## KeyQL\n\nThe main KeyQL constructor. Used to wrap datasets to be able to query them.\n\n### KeyQL#constructor\n\n```\nconstructor (dataset = [], mapFunction = v =\u003e v)\n```\n\nInitializes a KeyQL Dataset\n\n- **`dataset`** is an `array` of objects you wish to parse through\n- **`mapFunction`** gives us information on how to query each object in a dataset\n\nBy default, **`mapFunction`** is a no-op. This works when your dataset looks like:\n\n```json\n[\n  {\"id\": 1, \"name\": \"Jane\", \"age\": 27},\n  {\"id\": 2, \"name\": \"Stewart\", \"age\": 43}\n]\n```\n\nHowever, your dataset may not be this straightforward. Some APIs return nested field\nvalues. For example, in the case of something like:\n\n```json\n[\n  {\n    \"id\": 1,\n    \"fields\": {\"name\": \"Jane\", \"age\": 27}\n  },\n  {\n    \"id\": 2,\n    \"fields\": {\"name\": \"Stewart\", \"age\": 43}\n  }\n]\n```\n\nWe would provide the **`mapFunction`** as `v =\u003e v.fields` instead.\n\n### KeyQL#keys\n\n```\nkeys ()\n```\n\nReturns the keys for the Dataset, extracted from the first object provided.\n\n### KeyQL#rows\n\n```\nrows ()\n```\n\nReturns all rows in the Dataset as initially provided (**`mapFunction` applied**).\n\n### KeyQL#dataset\n\n```\ndataset ()\n```\n\nReturns all entries in the Dataset as initially provided (**no `mapFunction` applied**).\n\n### KeyQL#changeset\n\n```\nchangeset ()\n```\n\nReturns all entries in the dataset that have been updated as a result of `KeyQLQueryCommand#update`\nmethod calls. To create a copy of your dataset with all new changes committed\n(reset the updated rows tracker), use `KeyQL#commit`\n\n### KeyQL#commit\n\n```\ncommit ()\n```\n\nReturns a copy of your KeyQL instance with the same dataset, but the changeset\nwill have been reset.\n\n### KeyQL#query\n\n```\nquery ()\n```\n\nInstantiates a `KeyQLQueryCommand` instance, which will create an immutable\nhistory of all query commands.\n\n## KeyQLQueryCommand\n\nCreated from `KeyQL#query`, an immutable record of a query history. Can be\nchained indefinitely without overwriting previous `KeyQLQueryCommand` data.\n\nFor example:\n\n```javascript\nlet kqlDataset = new KeyQL([/* my dataset */]);\n\nlet q1 = kqlDataset.query().select({first_name: 'Tim'});\nlet q2 = q1.select({age__gt: 20});\n\nq1 === q2; // false\nq1.values(); // Everybody named \"Tim\"\nq2.values(); // Everybody named \"Tim\" with age \u003e 20\n```\n\n### KeyQLQueryCommand#select\n\nReturns a new `KeyQLQueryCommand` instance with a `select` command added.\nUsed to select values given a `KeyQLQuery`.\n\n```\nselect (keyQLQuery = [])\n```\n\n- **`keyQLQuery`** is an `array` of `objects` intended to be used as the query\n\n### KeyQLQueryCommand#limit\n\nReturns a new `KeyQLQueryCommand` instance with a `limit` command added.\nUsed to select values given a `KeyQLLimit`.\n\n```\nselect (keyQLLimit = {offset: 0, count: 0})\n```\n\n- **`keyQLLimit`** is an `object` containing the `offset` and `count` of records to return\n\n### KeyQLQueryCommand#values\n\nExecutes a query and returns a subset of your primary `dataset` based\non previous `KeyQLQueryCommand`s in the chain.\n\n```\nvalues ()\n```\n\nWill return an `array` of `objects` from your primary `dataset`.\n\n### KeyQLQueryCommand#update\n\nExecutes a query and returns a subset of your primary `dataset` based\non previous `KeyQLQueryCommand`s in the chain. Updates all values with the fields\nprovided.\n\n```\nupdate (fields = {})\n```\n\n- **`fields`** is an `object` containing key-value pairs you wish to update for match entries\n\nWill update and return an `array` of `objects` from your primary `dataset`.\n\n# Comparison to GraphQL\n\nKeyQL is not meant to act as a stand-in or replacement for GraphQL. You can\nthink of it more like a lightweight cousin: a midpoint between the wild-west of\nloosely opinionated SOAP and REST requests and the highly-structured, opinionated\nand complex world of GraphQL.\n\nWhereas GraphQL provides an interface and opinions around manipulating large,\ngraph-structured datasets and can define an entire backend architecture,\nKeyQL takes a more minimalistic and piecemeal approach -- providing a simple\nstructure for writing queries using JSON.\n\n# Acknowledgements\n\nThanks for checking out KeyQL. There's a lot more to come as the API is improved!\nThere have been a number of helpful supporters and contributors along the way,\nand both KeyQL and [FunctionScript](https://github.com/FunctionScript/FunctionScript)\nwould not be possible without any of them.\n\n### Core Contributors\n\n- [**Keith Horwood**](https://twitter.com/keithwhor)\n- [**Jacob Lee**](https://twitter.com/hacubu)\n- [**Steve Meyer**](https://twitter.com/notoriaga)\n- [**Yusuf Musleh**](https://twitter.com/yusuf-musleh)\n\n### Thanks to Airtable\n\nA special thanks to [Airtable](https://airtable.com) for providing a wonderful\nproduct and service. It has allowed our team to focus more on making backend tooling\nand development accessible to a larger number of developers and developers-to-be.\nWithout it, this project would be unlikely to exist in present form.\n\n[![Airtable Logo](/images/airtable-logo-300.png)](https://airtable.com)\n\n## Roadmap\n\n- **(High Priority)** Support type coercion of `entryValue` and `queryValue`\n- **(Low Priority)** PostgreSQL Support (re: [Nodal](https://github.com/keithwhor/nodal))\n\nKeyQL is (c) 2021 Polybit Inc.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Facode%2Fkeyql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Facode%2Fkeyql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Facode%2Fkeyql/lists"}