https://github.com/simonw/sqlite-fts5-trigram
Trigram tokenizer module for SQLite FTS5
https://github.com/simonw/sqlite-fts5-trigram
fts5 sqlite trigram
Last synced: 8 months ago
JSON representation
Trigram tokenizer module for SQLite FTS5
- Host: GitHub
- URL: https://github.com/simonw/sqlite-fts5-trigram
- Owner: simonw
- License: other
- Created: 2020-09-19T23:14:51.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2021-02-22T04:07:54.000Z (almost 5 years ago)
- Last Synced: 2024-10-18T07:53:59.389Z (over 1 year ago)
- Topics: fts5, sqlite, trigram
- Language: C
- Homepage: https://sqlite.org/forum/forumpost/ca90da691a?t=h
- Size: 8.79 KB
- Stars: 12
- Watchers: 3
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# sqlite-fts5-trigram
Trigram tokenizer module for SQLite FTS5
Code by Dan Kennedy, shared on [this forum thread](https://sqlite.org/forum/forumpost/ca90da691a?t=h).
See also [my weeknotes](https://simonwillison.net/2020/Sep/26/weeknotes-software-carpentry-sqlite/) describing this repository.
**This repository is now obsolete** because this shipped as a feature in December 2020 in [SQLite 3.34.0](https://www.sqlite.org/releaselog/3_34_0.html).
Example usage:
```
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> c = sqlite3.connect(":memory:")
>>> c
>>> c.enable_load_extension(True)
>>> c.load_extension("ftstri.so")
>>> c
>>> c.execute("CREATE VIRTUAL TABLE dict USING fts5(word, tokenize=tri);")
>>> c.execute('INSERT INTO dict values ("simon")')
>>> c.execute('INSERT INTO dict values ("cleo")')
>>> c.execute('INSERT INTO dict values ("natalie")')
>>> c.execute('select * from dict(?)', ['simon']).fetchall()
[('simon',)]
>>> c.execute('select * from dict(?)', ['sim']).fetchall()
[('simon',)]
>>> c.execute('select * from dict(?)', ['imo']).fetchall()
[('simon',)]
>>> c.execute("select highlight(dict, 0, '(', ')') from dict(?)", ['sim']).fetchall()
[('(sim)on',)]
>>> c.execute("select highlight(dict, 0, '(', ')') from dict(?)", ['simon']).fetchall()
[('(simon)',)]
>>> c.execute("select highlight(dict, 0, '(', ')') from dict(?)", ['mon']).fetchall()
[('si(mon)',)]
>>> c.execute("select highlight(dict, 0, '(', ')') from dict(?)", ['cl']).fetchall()
[]
>>> c.execute("select highlight(dict, 0, '(', ')') from dict(?)", ['cleo']).fetchall()
[('(cleo)',)]
>>> c.execute("select highlight(dict, 0, '(', ')') from dict(?)", ['cle']).fetchall()
[('(cle)o',)]
>>> c.execute("select highlight(dict, 0, '(', ')') from dict(?)", ['cleop']).fetchall()
[]
>>> c.execute("select highlight(dict, 0, '(', ')') from dict(?)", ['nat']).fetchall()
[('(nat)alie',)]
```