Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/deltachat/message-parser
Parsing of Links, Email adresses, simple text formatting (markdown subset), user mentions, hashtags and more in DeltaChat messages.
https://github.com/deltachat/message-parser
deltachat deltachat-desktop markdown parser
Last synced: about 20 hours ago
JSON representation
Parsing of Links, Email adresses, simple text formatting (markdown subset), user mentions, hashtags and more in DeltaChat messages.
- Host: GitHub
- URL: https://github.com/deltachat/message-parser
- Owner: deltachat
- License: other
- Created: 2021-06-20T16:50:04.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2024-04-09T14:01:53.000Z (7 months ago)
- Last Synced: 2024-04-13T20:35:46.659Z (7 months ago)
- Topics: deltachat, deltachat-desktop, markdown, parser
- Language: Rust
- Homepage: https://deltachat.github.io/message-parser/
- Size: 627 KB
- Stars: 11
- Watchers: 11
- Forks: 2
- Open Issues: 18
-
Metadata Files:
- Readme: README.MD
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# DeltaChat Message Parser
Parsing of Links, Email adresses, simple text formatting (markdown subset), user mentions, hashtags and more in DeltaChat messages.
The specification can be found in [spec.md](./spec.md).
WASM Demo:
## The idea behind it
Have the same rich message parsing on all platforms.
The basic idea is that core can use this library to convert messages to an AST format,
that can then be displayed by the UIs how they see fit, for desktop it will be converted to react elements.
> Desktop already uses this package (minus the markdown, because it does not make sense to only have markdown only on one platform) as wasm module (see `./message_parser_wasm`), later this will probably be integrated into deltachat core.
## Coding Principles
- **many** test cases
- aim to be fast - so also benchmarks to make sure the lib stays fast enough
## Recomendations
If used for message parsing, don't parse messages that are over `10 000` chars in size to ensure performance stays excellent. (the lib could and should support more than that and should aim to be fast enough for it, but on slow devices or transpiled to wasm or asmjs limiting it makes sense to avoid laggy/freezed interface)
### Benchmarking:
```
cargo install cargo-criterion
```
benchmark:
```
cargo criterion
```
docs about benchmarking: https://bheisler.github.io/criterion.rs/book/criterion_rs.html
### Changing CPU power settings for consistent results
These days most CPUs change their performance according to some rules to save power.
To produce consistent benchmark results, CPU performance must not change between benchmarks. There are various ways to achieve this.
If you've got a laptop, the first step might be connecting the AC adapter to ensure your laptop won't go on power saving mode and thus changing the
CPU frequency. The next step is to change CPU frequency to a constant value *under the maximum frequency CPU can handle*. Because the CPUs usually can't handle the maximum possible frequency on all cores.
On Linux, you can set the CPU frequency using `cpupower` utility:
```sh
cpupower frequency-set --min 3500 --max 3500 # this to set maximum and minimum to the same value
cpupower frequency-set -f 3500 # set frequency explicitly if the kernel module is available
```
## References
- Older discussion on introducing markdown into deltachat: https://github.com/deltachat/interface/pull/20
- Feature request for message markdown in the forum: https://support.delta.chat/t/markdown-support-in-chat/159
## Emoji Helpers
Additionally to message parsing this crate also contains some useful functions for working with emojis.
- `parser::is_emoji::emoji` (rust only) - nom parser that eats one emoji
- idea: could potentially be used by core to filter reactions to only emojis
- `parser::is_emoji::get_first_emoji(text)` - get first emoji if text begins with an emoji
- idea: can be used by UI to get the first emoji of a chat name to display it as text avatar
- `parser::is_emoji::count_emojis_if_only_contains_emoji(text)` - counts emojis in texts that contain only emojis
- useful for jumbomoji logic (if you send a small message with just emojis the emojis get displayed larger).
- this function does not fail on too long strings, so to keep good performance check the length beforehand and if it is too long the message would not be big anyway so you don't need to call this function.