{"id":13484907,"url":"https://github.com/gajus/liqe","last_synced_at":"2025-09-06T17:41:23.657Z","repository":{"id":57106743,"uuid":"425372488","full_name":"gajus/liqe","owner":"gajus","description":"Lightweight and performant Lucene-like parser, serializer and search engine.","archived":false,"fork":false,"pushed_at":"2025-06-11T20:43:53.000Z","size":512,"stargazers_count":656,"open_issues_count":13,"forks_count":23,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-08-18T03:49:51.125Z","etag":null,"topics":["filter","lucene","parser","search","serializer"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gajus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"gajus","patreon":"gajus"}},"created_at":"2021-11-06T23:40:38.000Z","updated_at":"2025-08-11T06:26:45.000Z","dependencies_parsed_at":"2024-01-16T23:30:10.480Z","dependency_job_id":"bfb83876-1d41-4575-803f-79d9a78f76a3","html_url":"https://github.com/gajus/liqe","commit_stats":{"total_commits":219,"total_committers":3,"mean_commits":73.0,"dds":"0.013698630136986356","last_synced_commit":"d70ec84041ace8016d099c08ff5c56149b228fda"},"previous_names":[],"tags_count":82,"template":false,"template_full_name":null,"purl":"pkg:github/gajus/liqe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gajus%2Fliqe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gajus%2Fliqe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gajus%2Fliqe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gajus%2Fliqe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gajus","download_url":"https://codeload.github.com/gajus/liqe/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gajus%2Fliqe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273939559,"owners_count":25194960,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["filter","lucene","parser","search","serializer"],"created_at":"2024-07-31T17:01:38.381Z","updated_at":"2025-09-06T17:41:23.615Z","avatar_url":"https://github.com/gajus.png","language":"TypeScript","funding_links":["https://github.com/sponsors/gajus","https://patreon.com/gajus"],"categories":["TypeScript"],"sub_categories":[],"readme":"# liqe\n\n[![Coveralls](https://img.shields.io/coveralls/gajus/liqe.svg?style=flat-square)](https://coveralls.io/github/gajus/liqe)\n[![NPM version](http://img.shields.io/npm/v/liqe.svg?style=flat-square)](https://www.npmjs.org/package/liqe)\n[![Canonical Code Style](https://img.shields.io/badge/code%20style-canonical-blue.svg?style=flat-square)](https://github.com/gajus/canonical)\n[![Twitter Follow](https://img.shields.io/twitter/follow/kuizinas.svg?style=social\u0026label=Follow)](https://twitter.com/kuizinas)\n\nLightweight and performant Lucene-like parser, serializer and search engine.\n\n* [Motivation](#motivation)\n* [Usage](#usage)\n* [Query Syntax](#query-syntax)\n  * [Liqe syntax cheat sheet](#liqe-syntax-cheat-sheet)\n  * [Keyword matching](#keyword-matching)\n  * [Number matching](#number-matching)\n  * [Range matching](#range-matching)\n  * [Wildcard matching](#wildcard-matching)\n  * [Boolean operators](#boolean-operators)\n* [Serializer](#serializer)\n* [AST](#ast)\n* [Utilities](#utilities)\n* [Compatibility with Lucene](#compatibility-with-lucene)\n* [Recipes](#recipes)\n  * [Handling syntax errors](#handling-syntax-errors)\n  * [Highlighting matches](#highlighting-matches)\n* [Development](#development)\n* [Tutorials](#tutorials)\n\n## Motivation\n\nOriginally built Liqe to enable [Roarr](https://github.com/gajus/roarr) log filtering via [cli](https://github.com/gajus/roarr-cli#filtering-logs). I have since been polishing this project as a hobby/intellectual exercise. I've seen it being adopted by [various](https://github.com/gajus/liqe/network/dependents) CLI and web applications that require advanced search. To my knowledge, it is currently the most complete Lucene-like syntax parser and serializer in JavaScript, as well as a compatible in-memory search engine.\n\nLiqe use cases include:\n\n* parsing search queries\n* serializing parsed queries\n* searching JSON documents using the Liqe query language (LQL)\n\nNote that the [Liqe AST](#ast) is treated as a public API, i.e., one could implement their own search mechanism that uses Liqe query language (LQL).\n\n## Usage\n\n```ts\nimport {\n  filter,\n  highlight,\n  parse,\n  test,\n} from 'liqe';\n\nconst persons = [\n  {\n    height: 180,\n    name: 'John Morton',\n  },\n  {\n    height: 175,\n    name: 'David Barker',\n  },\n  {\n    height: 170,\n    name: 'Thomas Castro',\n  },\n];\n```\n\nFilter a collection:\n\n```ts\nfilter(parse('height:\u003e170'), persons);\n// [\n//   {\n//     height: 180,\n//     name: 'John Morton',\n//   },\n//   {\n//     height: 175,\n//     name: 'David Barker',\n//   },\n// ]\n```\n\nTest a single object:\n\n```ts\ntest(parse('name:John'), persons[0]);\n// true\ntest(parse('name:David'), persons[0]);\n// false\n```\n\nHighlight matching fields and substrings:\n\n```ts\nhighlight(parse('name:john'), persons[0]);\n// [\n//   {\n//     path: 'name',\n//     query: /(John)/,\n//   }\n// ]\nhighlight(parse('height:180'), persons[0]);\n// [\n//   {\n//     path: 'height',\n//   }\n// ]\n```\n\n## Query Syntax\n\nLiqe uses Liqe Query Language (LQL), which is heavily inspired by Lucene but extends it in various ways that allow a more powerful search experience.\n\n### Liqe syntax cheat sheet\n\n```rb\n# search for \"foo\" term anywhere in the document (case insensitive)\nfoo\n\n# search for \"foo\" term anywhere in the document (case sensitive)\n'foo'\n\"foo\"\n\n# search for \"foo\" term in `name` field\nname:foo\n\n# search for \"foo\" term in `full name` field\n'full name':foo\n\"full name\":foo\n\n# search for \"foo\" term in `first` field, member of `name`, i.e.\n# matches {name: {first: 'foo'}}\nname.first:foo\n\n# search using regex\nname:/foo/\nname:/foo/o\n\n# search using wildcard\nname:foo*bar\nname:foo?bar\n\n# boolean search\nmember:true\nmember:false\n\n# null search\nmember:null\n\n# search for age =, \u003e, \u003e=, \u003c, \u003c=\nheight:=100\nheight:\u003e100\nheight:\u003e=100\nheight:\u003c100\nheight:\u003c=100\n\n# search for height in range (inclusive, exclusive)\nheight:[100 TO 200]\nheight:{100 TO 200}\n\n# boolean operators\nname:foo AND height:=100\nname:foo OR name:bar\n\n# unary operators\nNOT foo\n-foo\nNOT foo:bar\n-foo:bar\nname:foo AND NOT (bio:bar OR bio:baz)\n\n# implicit AND boolean operator\nname:foo height:=100\n\n# grouping\nname:foo AND (bio:bar OR bio:baz)\n```\n\n### Keyword matching\n\nSearch for word \"foo\" in any field (case insensitive).\n\n```rb\nfoo\n```\n\nSearch for word \"foo\" in the `name` field.\n\n```rb\nname:foo\n```\n\nSearch for `name` field values matching `/foo/i` regex.\n\n```rb\nname:/foo/i\n```\n\nSearch for `name` field values matching `f*o` wildcard pattern.\n\n```rb\nname:f*o\n```\n\nSearch for `name` field values matching `f?o` wildcard pattern.\n\n```rb\nname:f?o\n```\n\nSearch for phrase \"foo bar\" in the `name` field (case sensitive).\n\n```rb\nname:\"foo bar\"\n```\n\n### Number matching\n\nSearch for value equal to 100 in the `height` field.\n\n```rb\nheight:=100\n```\n\nSearch for value greater than 100 in the `height` field.\n\n```rb\nheight:\u003e100\n```\n\nSearch for value greater than or equal to 100 in the `height` field.\n\n```rb\nheight:\u003e=100\n```\n\n### Range matching\n\nSearch for value greater or equal to 100 and lower or equal to 200 in the `height` field.\n\n```rb\nheight:[100 TO 200]\n```\n\nSearch for value greater than 100 and lower than 200 in the `height` field.\n\n```rb\nheight:{100 TO 200}\n```\n\n### Wildcard matching\n\nSearch for any word that starts with \"foo\" in the `name` field.\n\n```rb\nname:foo*\n```\n\nSearch for any word that starts with \"foo\" and ends with \"bar\" in the `name` field.\n\n```rb\nname:foo*bar\n```\n\nSearch for any word that starts with \"foo\" in the `name` field, followed by a single arbitrary character.\n\n```rb\nname:foo?\n```\n\nSearch for any word that starts with \"foo\", followed by a single arbitrary character and immediately ends with \"bar\" in the `name` field.\n\n```rb\nname:foo?bar\n```\n\n### Boolean operators\n\nSearch for phrase \"foo bar\" in the `name` field AND the phrase \"quick fox\" in the `bio` field.\n\n```rb\nname:\"foo bar\" AND bio:\"quick fox\"\n```\n\nSearch for either the phrase \"foo bar\" in the `name` field AND the phrase \"quick fox\" in the `bio` field, or the word \"fox\" in the `name` field.\n\n```rb\n(name:\"foo bar\" AND bio:\"quick fox\") OR name:fox\n```\n\n## Serializer\n\nSerializer allows to convert Liqe tokens back to the original search query.\n\n```ts\nimport {\n  parse,\n  serialize,\n} from 'liqe';\n\nconst tokens = parse('foo:bar');\n\n// {\n//   expression: {\n//     location: {\n//       start: 4,\n//     },\n//     quoted: false,\n//     type: 'LiteralExpression',\n//     value: 'bar',\n//   },\n//   field: {\n//     location: {\n//       start: 0,\n//     },\n//     name: 'foo',\n//     path: ['foo'],\n//     quoted: false,\n//     type: 'Field',\n//   },\n//   location: {\n//     start: 0,\n//   },\n//   operator: {\n//     location: {\n//       start: 3,\n//     },\n//     operator: ':',\n//     type: 'ComparisonOperator',\n//   },\n//   type: 'Tag',\n// }\n\nserialize(tokens);\n// 'foo:bar'\n```\n\n## AST\n\n```ts\nimport {\n  type BooleanOperatorToken,\n  type ComparisonOperatorToken,\n  type EmptyExpression,\n  type FieldToken,\n  type ImplicitBooleanOperatorToken,\n  type ImplicitFieldToken,\n  type LiteralExpressionToken,\n  type LogicalExpressionToken,\n  type RangeExpressionToken,\n  type RegexExpressionToken,\n  type TagToken,\n  type UnaryOperatorToken,\n} from 'liqe';\n```\n\nThere are 11 AST tokens that describe a parsed Liqe query.\n\nIf you are building a serializer, then you must implement all of them for the complete coverage of all possible query inputs. Refer to the [built-in serializer](./src/serialize.ts) for an example.\n\n## Utilities\n\n```ts\nimport {\n  isSafeUnquotedExpression,\n} from 'liqe';\n\n/**\n * Determines if an expression requires quotes.\n * Use this if you need to programmatically manipulate the AST\n * before using a serializer to convert the query back to text.\n */\nisSafeUnquotedExpression(expression: string): boolean;\n```\n\n## Compatibility with Lucene\n\nThe following Lucene abilities are not supported:\n\n* [Fuzzy Searches](https://lucene.apache.org/core/2_9_4/queryparsersyntax.html#Fuzzy%20Searches)\n* [Proximity Searches](https://lucene.apache.org/core/2_9_4/queryparsersyntax.html#Proximity%20Searches)\n* [Boosting a Term](https://lucene.apache.org/core/2_9_4/queryparsersyntax.html#Boosting%20a%20Term)\n\n## Recipes\n\n### Handling syntax errors\n\nIn case of a syntax error, Liqe throws `SyntaxError`.\n\n```ts\nimport {\n  parse,\n  SyntaxError,\n} from 'liqe';\n\ntry {\n  parse('foo bar');\n} catch (error) {\n  if (error instanceof SyntaxError) {\n    console.error({\n      // Syntax error at line 1 column 5\n      message: error.message,\n      // 4\n      offset: error.offset,\n      // 1\n      offset: error.line,\n      // 5\n      offset: error.column,\n    });\n  } else {\n    throw error;\n  }\n}\n```\n\n### Highlighting matches\n\nConsider using [`highlight-words`](https://github.com/tricinel/highlight-words) package to highlight Liqe matches.\n\n## Development\n\n### Compiling Parser\n\nIf you are going to modify parser, then use `npm run watch` to run compiler in watch mode.\n\n### Benchmarking Changes\n\nBefore making any changes, capture the current benchmark on your machine using `npm run benchmark`. Run benchmark again after making any changes. Before committing changes, ensure that performance is not negatively impacted.\n\n\n## Tutorials\n\n* [Building advanced SQL search from a user text input](https://contra.com/p/WobOBob7-building-advanced-sql-search-from-a-user-text-input)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgajus%2Fliqe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgajus%2Fliqe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgajus%2Fliqe/lists"}