{"id":31542648,"url":"https://github.com/adamhl8/filterql","last_synced_at":"2025-10-04T11:59:57.319Z","repository":{"id":309926299,"uuid":"1038057277","full_name":"adamhl8/filterql","owner":"adamhl8","description":"A tiny query language for filtering structured data","archived":false,"fork":false,"pushed_at":"2025-09-20T22:37:47.000Z","size":269,"stargazers_count":273,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-21T00:16:50.336Z","etag":null,"topics":["data","filter","filtering","filterql","language","query","query-language","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adamhl8.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-14T14:49:54.000Z","updated_at":"2025-09-20T22:37:50.000Z","dependencies_parsed_at":"2025-09-12T22:22:21.602Z","dependency_job_id":"081d3a12-aa44-4d0e-a3f0-b974fb3cf19f","html_url":"https://github.com/adamhl8/filterql","commit_stats":null,"previous_names":["adamhl8/filterql"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/adamhl8/filterql","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamhl8%2Ffilterql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamhl8%2Ffilterql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamhl8%2Ffilterql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamhl8%2Ffilterql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adamhl8","download_url":"https://codeload.github.com/adamhl8/filterql/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamhl8%2Ffilterql/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278308618,"owners_count":25965654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","filter","filtering","filterql","language","query","query-language","typescript"],"created_at":"2025-10-04T11:59:52.424Z","updated_at":"2025-10-04T11:59:57.312Z","avatar_url":"https://github.com/adamhl8.png","language":"TypeScript","readme":"\u003ch1 align=\"center\"\u003e\u003cimg align=\"top\" style=\"color:#36BCF7; width:38px; height:38px;\" src=\"https://raw.githubusercontent.com/adamhl8/filterql/refs/heads/main/assets/logo.svg\"\u003e FilterQL\u003c/h1\u003e\n\nA tiny query language for filtering structured data 🚀\n\n\u003c!-- https://readme-typing-svg.demolab.com/demo/?font=JetBrains+Mono\u0026size=16\u0026duration=3000\u0026pause=7500\u0026vCenter=true\u0026width=690\u0026height=25\u0026lines=(genre+%3D%3D+Action+%7C%7C+genre+%3D%3D+Comedy)+%26%26+rating+%3E%3D+8.5+%7C+SORT+rating+desc --\u003e\n\n[![Typing SVG](\u003chttps://readme-typing-svg.demolab.com?font=JetBrains+Mono\u0026size=16\u0026duration=3000\u0026pause=7500\u0026vCenter=true\u0026width=690\u0026height=25\u0026lines=(genre+%3D%3D+Action+%7C%7C+genre+%3D%3D+Comedy)+%26%26+rating+%3E%3D+8.5+%7C+SORT+rating+desc\u003e)](https://git.io/typing-svg)\n\n---\n\nIn addition to the [Overview](#overview) below, there are three main sections of this README:\n\n- **[Queries](#queries)** - How to write FilterQL queries\n- **[TypeScript Library](#typescript-library)** - Usage of the FilterQL TypeScript library\n- **[FilterQL language specification](#language-specification)**\n\n## Overview\n\nDefine a query schema and create a FilterQL instance:\n\n```ts\nimport { FilterQL } from \"filterql\"\n\n// The schema determines what fields are allowed in the query\nconst schema = {\n  title: { type: \"string\", alias: \"t\" },\n  year: { type: \"number\", alias: \"y\" },\n  monitored: { type: \"boolean\", alias: \"m\" },\n  rating: { type: \"number\" },\n  genre: { type: \"string\" },\n}\n\nconst filterql = new FilterQL({ schema })\n\nconst movies = [\n  { title: \"The Matrix\", year: 1999, monitored: true, rating: 8.7, genre: \"Action\" },\n  { title: \"Inception\", year: 2010, monitored: true, rating: 8.8, genre: \"Thriller\" },\n  { title: \"The Dark Knight\", year: 2008, monitored: false, rating: 9.0, genre: \"Action\" },\n]\n\n// Filter movies by genre\nconst actionMovies = filterql.query(movies, \"genre == Action\")\n\n// Field aliases and multiple comparisons\nconst recentGoodMovies = filterql.query(movies, \"y \u003e= 2008 \u0026\u0026 rating \u003e= 8.5\")\n\n// Sort the filtered data by using the built-in SORT operation\nconst recentGoodMovies = filterql.query(movies, \"year \u003e= 2008 | SORT rating desc\")\n\n// Filter using boolean shorthand\nconst monitoredMovies = filterql.query(movies, \"monitored\")\n```\n\n---\n\n\u003c!-- toc --\u003e\n\n- [Queries](#queries)\n  - [Logical Operators](#logical-operators)\n  - [Comparison Operators](#comparison-operators)\n  - [Boolean Fields](#boolean-fields)\n  - [Quoted Values](#quoted-values)\n  - [Empty Value Checks](#empty-value-checks)\n  - [Match-All](#match-all)\n  - [Operations](#operations)\n- [TypeScript Library](#typescript-library)\n  - [Installation](#installation)\n  - [Example](#example)\n  - [Schemas](#schemas)\n  - [Handling data](#handling-data)\n  - [Options](#options)\n  - [Custom Operations](#custom-operations)\n  - [API Reference](#api-reference)\n- [Language Specification](#language-specification)\n  - [Grammar](#grammar)\n  - [Comparison Operators](#comparison-operators-1)\n  - [Logical Operators](#logical-operators-1)\n  - [Match-All](#match-all-1)\n  - [Operations](#operations-1)\n  - [Syntax Rules](#syntax-rules)\n  - [Implementation](#implementation)\n\n\u003c!-- tocstop --\u003e\n\n---\n\n## Queries\n\nThe most basic query is a single comparison: `\u003cfield\u003e \u003ccomparison operator\u003e \u003cvalue\u003e`\n\n```\ntitle == Interstellar\n```\n\nYou can also use the alias for a field:\n\n```\nt == Interstellar\n```\n\nCombine multiple comparisons using logical operators for more complex queries:\n\n```\ntitle == Interstellar \u0026\u0026 year == 2014\n```\n\n### Logical Operators\n\nThe following logical operators can be used in queries:\n\n- `()` (parentheses for grouping)\n- `!` (not)\n- `\u0026\u0026` (and)\n- `||` (or)\n\n\u003e [!TIP]\n\u003e Note that these operators are listed in order of precedence. This is important because many queries will likely require parentheses to do what you want.\n\nFor example:\n\n`genre == Action || genre == Thriller \u0026\u0026 rating \u003e= 8.5` means \"genre must be Action or, genre must be Thriller and rating must be at least 8.5.\" This probably isn't what you want.\n\n`(genre == Action || genre == Thriller) \u0026\u0026 rating \u003e= 8.5` means \"genre must be Action or Thriller, and rating must be at least 8.5.\"\n\n### Comparison Operators\n\nThe following comparison operators can be used in comparisons:\n\n- `==` (equals)\n- `!=` (not equals)\n- `*=` (contains)\n- `^=` (starts with)\n- `$=` (ends with)\n- `~=` (matches regex)\n- `\u003e=` (greater than or equal)\n- `\u003c=` (less than or equal)\n\n\u003e [!TIP]\n\u003e Comparisons are case-sensitive. To make them case-insensitive, prefix the comparison operator with `i`.\n\n```\ntitle i== interstellar\n```\n\n### Boolean Fields\n\nFor boolean fields, you can use the field name without any comparison to check for `true`:\n\n`downloaded` is equivalent to `downloaded == true`\n\n`!downloaded` is equivalent to `!(downloaded == true)`\n\n### Quoted Values\n\nIf your comparison value has spaces, you must enclose it in double quotes:\n\n```\ntitle == \"The Dark Knight\"\n```\n\nInside a quoted value, double quotes must be escaped:\n\n```\ntitle == \"A title with \\\"quotes\\\"\"\n```\n\nValues ending with `)` _as part of the value_ (not a closing parenthesis) must be quoted:\n\n```\ntitle == \"(a title surrounded by parentheses)\"\n```\n\n### Empty Value Checks\n\nSometimes the data you're filtering might have empty values (`\"\"`, `undefined`, `null`). You can filter for empty values by comparing to an empty string:\n\nGet all entries that _don't_ have a rating:\n\n```\nrating == \"\"\n```\n\nGet all entries that have a rating:\n\n```\nrating != \"\"\n```\n\n### Match-All\n\nIf you want to get _all_ of the entries, use `*` (the data is not filtered):\n\n```\n*\n```\n\nThis is mainly useful when you don't want to filter the data but want to apply operations to it.\n\n```\n* | SORT rating desc\n```\n\n### Operations\n\nAfter filtering, you can apply operations to transform the data: `\u003cfilter\u003e | \u003coperation\u003e [arg]...`\n\n```\nyear \u003e= 2000 | SORT rating desc | LIMIT 10\n```\n\n- Operations are applied in the order they are specified.\n- The same operation can be applied multiple times.\n\n#### Built-in Operations\n\nThere are currently two built-in operations:\n\n- `SORT`: Sorts the data by the specified field.\n  - `SORT \u003cfield\u003e [direction]`\n  - `direction` can be `asc` or `desc` (default: `asc`).\n- `LIMIT`: Limits the number of entries returned.\n  - `LIMIT \u003cnumber\u003e`\n\nIf you have any suggestions for other operations, please let me know by opening an issue!\n\n\u003e [!TIP]\n\u003e You can also define **[custom operations](#custom-operations)**.\n\n## TypeScript Library\n\n### Installation\n\n```bash\nbun add filterql\n# or: npm install filterql\n```\n\n### Example\n\nLet's say you're building a CLI tool that fetches some data to be filtered by a query the user provides:\n\n```ts\nimport { FilterQL } from \"filterql\"\n\n// data is an array of objects\nconst data = await (await fetch(\"https://api.example.com/movies\")).json()\n\nconst query = process.argv[2] // first argument\n\nconst schema = {\n  title: { type: \"string\", alias: \"t\" },\n  year: { type: \"number\", alias: \"y\" },\n  monitored: { type: \"boolean\", alias: \"m\" },\n  rating: { type: \"number\" },\n  genre: { type: \"string\" },\n}\n\nconst filterql = new FilterQL({ schema })\nconst filteredMovies = filterql.query(data, query)\n\nconsole.log(filteredMovies)\n```\n\nAnd then the user might use your CLI tool like this:\n\n```sh\nmovie-cli '(genre == Action || genre == Comedy) \u0026\u0026 year \u003e= 2000 \u0026\u0026 rating \u003e= 8.5'\n```\n\n### Schemas\n\nThe schema given to the `FilterQL` constructor determines what fields and value types are allowed in queries.\n\n\u003e [!IMPORTANT]\n\u003e The type of data the `FilterQL` methods accept is `Record\u003cstring, unknown\u003e[]`. This means that **FilterQL does not care about extra properties/keys in the _data_**.\n\u003e\n\u003e In other words, a schema's keys can be a subset of the data's keys.\n\u003e\n\u003e Similarly, **the schema is _not_ used to validate the data**. It is only used to validate the values given in the _query_.\n\u003e\n\u003e See the [Handling data](#handling-data) section.\n\nEach field has a `type` and an (optional) `alias`.\n\n```ts\nconst schema = {\n  title: { type: \"string\", alias: \"t\" },\n  year: { type: \"number\", alias: \"y\" },\n  monitored: { type: \"boolean\" },\n}\n\nconst filterql = new FilterQL({ schema })\n```\n\nField types determine validation behavior:\n\n- `string`: The value must be coercible to a string (this is always the case)\n- `number`: The value must be coercible to a number\n- `boolean`: The value must be `true` or `false`\n\nWhen a comparison value can't be coerced to the field's type, an error is thrown. For example, consider a query like `year = foo`.\n\n- The comparison value of `foo` can't be coerced to a number, so an error is thrown.\n\n### Handling data\n\nIt's important to note that query comparisons are only evaluated against certain data types.\n\n- Specifically, a data value must be one of `string`, `number`, or `boolean`, `undefined`, or `null` to be evaluated.\n\nFor example, say we have the following data:\n\n```ts\nconst people = [\n  {\n    name: \"Bob\",\n    age: 30,\n    address: {\n      street: \"123 Main St\",\n      city: \"Anytown\",\n    },\n    roles: [\"admin\", \"user\"],\n  },\n  // more people...\n]\n```\n\nPassing in data like this is perfectly valid, but because the `address` and `roles` properties are not one of the comparable types, they can't be filtered on. e.g. a query like `roles == admin` won't return any results.\n\nIf you want to filter on nested data like this, you should transform the data into a flat structure before passing it to FilterQL.\n\nFor example, say you wanted to query for people who have the `\"admin\"` role. You could transform the data like this:\n\n```ts\n// query: 'roles_admin == true'\n{\n  name: \"Bob\",\n  age: 30,\n  address_street: \"123 Main St\",\n  address_city: \"Anytown\",\n  roles_admin: true,\n  roles_user: true,\n}\n```\n\nOr maybe something like this, where the elements/properties are joined as a string:\n\n```ts\n// query: 'roles *= admin'\n{\n  name: \"Bob\",\n  age: 30,\n  address: \"123 Main St, Anytown\",\n  roles: \"admin, user\",\n}\n```\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003ci\u003eWhy not support nested data structures?\u003c/i\u003e\u003c/summary\u003e\n\nSupporting nested data structures would require a more complex syntax, which goes against FilterQL's general principle that the query language should be relatively simple and easy to understand.\n\nBy effectively \"forcing\" data to be flattened (if you want to filter on those elements/properties), \u003ci\u003ethat extra complexity/work is the developer's responsibility, not the person who's writing the query\u003c/i\u003e.\n\n\u003c/details\u003e\n\n### Options\n\n```ts\nconst filterql = new FilterQL({\n  schema,\n  options: {\n    allowUnknownFields: false,\n  },\n})\n```\n\nThe `FilterQL` constructor accepts an optional `options` object with the following properties:\n\n`allowUnknownFields` (default: `false`): By default, an error is thrown if a query contains a field that's not in the schema. If `true`, unknown fields are allowed.\n\n- This could be useful in situations where the schema can't be determined ahead of time. i.e. the keys of the data are unknown or may change.\n\n### Custom Operations\n\n\u003e [!TIP]\n\u003e Take a look at the built-in operations in [src/operation-evaluator/operations.ts](./src/operation-evaluator/operations.ts) to see how they're implemented.\n\nLet's say you want to create a custom operation called `ROUND` that takes a field name as the first argument and rounds the number value of that field.\n\nA query might look something like `year \u003e= 2000 | SORT rating desc | ROUND rating`.\n\nYou can define custom operations by providing a `customOperations` object to the `FilterQL` constructor.\n\n- The keys are the names of the custom operations and **must** be all uppercase.\n- The values are functions that return the transformed data.\n\n```ts\nimport type { OperationMap } from \"filterql\"\n\nconst customOperations: OperationMap = {\n  ROUND: (data, args, { resolveField }) =\u003e {\n    const field = resolveField(args[0]) // the first argument might be the alias of the field\n    if (!field) throw new Error(`Unknown field '${args[0]}' for operation 'ROUND'`)\n    return data.map((item) =\u003e Math.round(item[field]))\n  },\n}\n\nconst filterql = new FilterQL({ schema: mySchema, customOperations })\n```\n\nThree arguments are provided to a given operation function:\n\n- `data`: The data after its been filtered (and transformed by any previous operations).\n- `args`: A string array containing any arguments passed to the operation.\n- `operationHelpers`: An object containing the FilterQL instance `schema`, `options`, and a `resolveField` function.\n  - The `resolveField` function takes a string and returns the full field name if it exists in the schema. Use it to support field aliases in operations. e.g. `\"t\" -\u003e \"title\"` or `\"title\" -\u003e \"title\"` (full field name resolves to itself).\n\n#### Overriding built-in operations\n\nBuilt-in operations can be overridden by providing a custom operation with the same name:\n\n```ts\nconst customOperations: OperationMap = {\n  SORT: (data, args, { resolveField }) =\u003e {\n    // your custom SORT implementation\n  },\n}\n```\n\n### API Reference\n\nIn addition to the primary `query` method, the `parse`, `applyFilter`, and `applyOperations` methods are available. These may be useful in cases where you want to filter and apply operations separately.\n\n```ts\nclass FilterQL {\n  constructor({\n    schema: Schema,\n    options?: FilterQLOptions,\n    customOperations?: OperationMap\n  })\n\n  /** Filter and apply operations to a data array with the given query */\n  query\u003cT extends Record\u003cstring, unknown\u003e\u003e(data: T[], query: string): T[]\n\n  /**\n   * Parse a query string into an `ASTNode` containing the `FilterNode` and `OperationNode[]`\n   *\n   * You can use the returned `ASTNode` with the `applyFilter` and `applyOperations` methods\n   */\n  parse(query: string): ASTNode\n\n  /** Apply the given `FilterNode` to a data array */\n  applyFilter\u003cT extends DataObject\u003e(data: T[], filter: FilterNode): T[]\n\n  /** Apply the given `OperationNode[]` to a data array */\n  applyOperations\u003cT extends DataObject\u003e(data: T[], operations: OperationNode[]): T[]\n}\n```\n\n## Language Specification\n\n\u003e [!NOTE]\n\u003e Implementations in other languages are more than welcome!\n\n### Grammar\n\nFilterQL follows this grammar:\n\n```\nquery := filter ( \"|\" operation )*\nfilter := expr\noperation := operation_name arg*\nexpr := and_expr ( \"||\" and_expr )*\nand_expr := term ( \"\u0026\u0026\" term )*\nterm := \"!\" term | \"(\" expr \")\" | comparison\ncomparison := field operator value | field | \"*\"\n```\n\n### Comparison Operators\n\n| Operator | Description           | Example                 |\n| -------- | --------------------- | ----------------------- |\n| `==`     | Equals                | `title == Interstellar` |\n| `!=`     | Not equals            | `title != \"The Matrix\"` |\n| `*=`     | Contains              | `title *= \"Matrix\"`     |\n| `^=`     | Starts with           | `title ^= The`          |\n| `$=`     | Ends with             | `title $= Knight`       |\n| `~=`     | Regular expression    | `title ~= \".*Matrix.*\"` |\n| `\u003e=`     | Greater than or equal | `year \u003e= 2000`          |\n| `\u003c=`     | Less than or equal    | `rating \u003c= 8.0`         |\n\nAny comparison operator can be prefixed with `i` (used to make a comparison case-insensitive):\n\n```\ntitle i== \"the matrix\"\n```\n\n### Logical Operators\n\n| Operator | Description            | Example                               | Precedence (in order from highest to lowest) |\n| -------- | ---------------------- | ------------------------------------- | -------------------------------------------- |\n| `()`     | Parentheses (grouping) | `(year \u003e= 2000 \u0026\u0026 year \u003c= 2010)`      | Highest precedence                           |\n| `!`      | NOT                    | `!title *= Matrix`                    | Right associative                            |\n| `\u0026\u0026`     | AND                    | `monitored \u0026\u0026 year \u003e= 2000`           | Left associative                             |\n| `\\|\\|`   | OR                     | `genre == Action \\|\\| genre == Drama` | Left associative                             |\n\n### Match-All\n\nThe `*` character can be used in place of a comparison. When evaluated, it _always_ matches.\n\nExamples:\n\n`* | SORT year`: Matches all entries and then sorts by `year`.\n\n`title == Matrix || * | SORT year`: This is effectively the same as the above query because the `|| *` matches all entries.\n\n### Operations\n\nOperations can be chained after the filter expression using the pipe operator (`|`). Each operation consists of an uppercase operation name followed by zero or more arguments:\n\n```\ntitle == \"Matrix\" | SORT year\nrating \u003e= 8.5 | SORT rating desc | LIMIT 10\n```\n\n### Syntax Rules\n\n- Tokens/words in queries are terminated by whitespace (` `, `\\t`, `\\n`, `\\r`)\n  - e.g. a query like `title==Matrix` would be tokenized as _one_ token with a value of `\"title==Matrix\"`\n- Queries are terminated by end of input\n- An empty/blank query is equivalent to `*`\n- Fields can be used without a comparison operator: `monitored` is equivalent to `monitored == true`\n- Fields and values can have operators \"attached\" to them that are automatically split off during tokenization:\n  - Fields can have one or more leading operators (`!`, `(`) and trailing operators (`)`) attached. e.g. `!(monitored)` becomes tokens: `[\"!\", \"(\", \"monitored\", \")\"]`\n  - Values can have one or more trailing operators (`)`) attached. e.g. `(title == Matrix)` becomes tokens: `[\"(\", \"title\", \"==\", \"Matrix\", \")\"]`\n    - As a consequence, values ending with `)` _as part of the value_ must be quoted: `field == \"(value)\"`\n  - This allows for natural syntax like `!(field == value)` without requiring spaces around operators\n- Values are either **unquoted** or **quoted**\n  - Values requiring whitespace must be enclosed in double quotes: `\"The Matrix\"`\n  - Double quotes (`\"`) are the only valid quotes\n  - Double quotes inside quoted values are escaped with a backslash (`\\\"`): `\"a value with \\\"quotes\\\"\"`\n    - This is the only supported escape sequence\n  - Empty quoted values are valid: `\"\"`\n- Values are _always_ preceded by a comparison operator, which is how they can be differentiated from fields\n- Operation names must be all uppercase\n\n### Implementation\n\nThis repository serves as a reference implementation for the language. There are many non-syntax rules/considerations that are not covered in this section of the README. See the source code for implementation details.\n","funding_links":[],"categories":["Built with TypeScript"],"sub_categories":["Libraries"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamhl8%2Ffilterql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadamhl8%2Ffilterql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamhl8%2Ffilterql/lists"}