https://github.com/isqua/syntax-sandbox
Early prototype of custom query language editor
https://github.com/isqua/syntax-sandbox
codemirror codemirror6 lezer lezer-grammar query-language
Last synced: 11 months ago
JSON representation
Early prototype of custom query language editor
- Host: GitHub
- URL: https://github.com/isqua/syntax-sandbox
- Owner: isqua
- License: gpl-3.0
- Created: 2023-09-01T17:52:42.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-07-17T19:28:52.000Z (11 months ago)
- Last Synced: 2025-07-17T22:55:15.868Z (11 months ago)
- Topics: codemirror, codemirror6, lezer, lezer-grammar, query-language
- Language: TypeScript
- Homepage: https://isqua.github.io/syntax-sandbox/
- Size: 441 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# syntax-sandbox
This repository contains my experiment on creating a query language for [the CodeMirror](https://codemirror.net/). It is a demo, not a reusable package.
The application demonstrates how to create custom Query Language for CodeMirror using [the Lezer Parser System](https://lezer.codemirror.net/). It supports autocomplete, syntax highlight, validation, and custom UI decorations.
My goal was to create query language that is unbound from an application domain. So the grammar part defines just a syntax for queries, but does not contains any properties or values. Thus, domain specific data can be defined independently of grammar.
## Architecture
At the application level users of the component describe available properties and values. Please see the example in [src/data](./src/data/properties.ts). They can also define a `Suggest` and/or `Validator` classes to implement some custom logic. Then they need to instantiate a data model with defined properties and pass it to the `Editor`, see [src/main.ts](./src/main.ts). Optionally, they can also create a `Decorator` to add some specific UI features for different properties and values: [src/decorator](./src/decorator/decorator.ts).
Then, the `Editor` renders a CodeMirror instance with some plugins. In the [src/language](./src/language/language.ts) multiple CodeMirror plugins are instantiated using defined data model and decorator. The parser is generated using Lezer from the [QueryLanguage.grammar](./src/language/grammar/QueryLanguage.grammar). The mapper from syntax tree to the MongoDB-like JSON is implemented in [src/language/query](./src/language/query/query.ts).
The `Editor` emits change event with the result of parsing. The result is shaped as MongoDB-like JSON to show the query tree.
What can be improved:
- Asyncronous suggest for data-intensive applications
- More operators, for example for numeric fields
- Better UX for different kind of values, for example datepicker for date and time
## Commands
Start:
```
npm run dev
```
Rebuild grammar:
```
npm run grammar
```
Run tests:
```
npm test
```
## File Structure
1. [main.ts](./src/main.ts) is an entrypoint. It finds the root DOM element and render the app.
1. **Domain Specific Data**
1. [data](./src/data) contains some data and declarations that are related to a specific app (e.g. task manager)
1. [properties](./src/data/properties.ts) describes possible properties and its values available for QL
1. [persons](./src/data/persons.ts) contains some imaginable “users” that could be used for filtering by an author
1. [decorator](./src/decorator) defines how to replace query tokens with rich HTML widgets, e.g. it replaces usernames of authors with their names
1. **Universal query language:** [language](./src/language) contains a set of CodeMirror plugins that implement work with the query language defined in Grammar
1. [grammar](./src/language/grammar) contains everything about parsing a text to an AST
1. [QueryLanguage.grammar](./src/language/grammar/QueryLanguage.grammar) grammar definition in terms of [The Lezer Parser System](https://lezer.codemirror.net/). The parser itself is autogenerated based on the grammar file.
1. [query](./src/language/query/query.ts) a helper to transform AST to query tree, it helps to visualize the parsing result and [test](./src/language/query/query.spec.ts) the parser
1. [highlighter](./src/language/highlighter) highlights different semantic tokens with different colors
1. [linter](./src/language/linter) shows errors for improper tokens
1. [decorator](./src/language/decorator) shows user-defined widgets for some tokens
1. [autocomplete](./src/language/autocomplete) suggests related tokens to a user
1. [language](./src/language/language.ts) combines them all