Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/nicholaswmin/flomaster
fast, native syntax highlighting [wip]
https://github.com/nicholaswmin/flomaster
highlight-api parser syntax-highlighting tokenizer
Last synced: 19 days ago
JSON representation
fast, native syntax highlighting [wip]
- Host: GitHub
- URL: https://github.com/nicholaswmin/flomaster
- Owner: nicholaswmin
- Created: 2024-11-22T09:16:07.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2024-11-27T03:16:36.000Z (about 1 month ago)
- Last Synced: 2024-12-15T16:50:29.639Z (23 days ago)
- Topics: highlight-api, parser, syntax-highlighting, tokenizer
- Language: JavaScript
- Homepage: https://nicholaswmin.github.io/flomaster/
- Size: 80.1 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# flomaster
[](https://github.com/nicholaswmin/flomaster/actions/workflows/test.yml)
[sandbox][snd]
## why
The current process, used by every (top 3 at least)
syntax highlighters:
1. [Lexical tokenization][tok] of raw source code
2. Wrapping *each* token in a DOM element
3. CSS selection
Step 2 is an awkward mechanism which triggers [layout reflow][rfl][^1] in
the main thread, hence contributing to "jank" and [First-Meaningful Paint][fmp]
delays. Additionally, it enlarges the DOM further exacerbating the issue.
This prototype attempts an alternative method using position offsets,
so it (should) avoid these issues altogether.
Simply put: it's *faster*.
## todos
- [x] get a non-fancy [tokenizer][tok] to iterate upon:
- [x] return shape: `[{ token: 'keyword', start: 5, end: 10 }]`
- [x] add minimal tests.
- [ ] the tokenizer falls over some regex,
see notes in test.
- [ ] fix/ensure errors are correctly printable in browser
- [ ] many cases that need ironing-out. Fix the presets.
- [ ] needs precedence fixes, i.e `forEach` is colored half
for `for` and the rest separate.
- [ ] start benchmarks asap, this is more than enough
- This is perfect: https://gist.github.com/addyosmani/c053f68aead473d7585b45c9e8dce31e
- what about other browsers
- [ ] fix the online sandbox
- [x] a decent color mapper
- [ ] add a small benchmark
- decide on the parameters *before* you start
- [ ] review if it makes sense
## test
> tests require Node `v22+`
```bash
node --run test
```
## misc
serve demo sandbox locally
```bash
$ node --run demo
```
## license
> MIT License
>
> Copyright (c) 2024 Nicholas Kyriakides
>
> Permission is hereby granted, free of charge, to any person obtaining
> a copy of this software and documentation files (the "Software"), to
> deal in the Software without restriction, including without limitation
> the rights to use, copy, modify, merge, publish, distribute, sublicense,
> and/or sell copies of the Software, and to permit persons to whom the
> Software is furnished to do so, subject to the following conditions:
>
> The above copyright notice and this permission notice shall be
> included in all copies or substantial portions of the Software.
[^1]: this is distinct from *forced synchronous layout reflow*, the reflow
that is triggered by accessing certain geometric APIs. Both are types
of reflow.
[tok]: https://en.wikipedia.org/wiki/Lexical_analysis#Tokenization
[fmp]: https://developer.mozilla.org/en-US/docs/Glossary/First_meaningful_paint
[dom]: https://en.wikipedia.org/wiki/Document_Object_Model
[rfl]: https://developer.mozilla.org/en-US/docs/Glossary/Reflow
[bnk]: https://www.chromium.org/blink/
[snd]: https://nicholaswmin.github.io/flomaster/demo.html
[dcs]: https://nicholaswmin.github.io/flomaster
[pdc]: https://github.com/nicholaswmin/flomaster/tree/main/.github/docs