https://github.com/groonga/groonga-tokenizer-sample
It is a sample groonga tokenizer
https://github.com/groonga/groonga-tokenizer-sample
Last synced: about 1 year ago
JSON representation
It is a sample groonga tokenizer
- Host: GitHub
- URL: https://github.com/groonga/groonga-tokenizer-sample
- Owner: groonga
- License: other
- Created: 2013-08-07T13:51:39.000Z (almost 13 years ago)
- Default Branch: master
- Last Pushed: 2014-01-24T05:04:56.000Z (over 12 years ago)
- Last Synced: 2025-02-10T11:45:17.844Z (over 1 year ago)
- Language: C
- Size: 128 KB
- Stars: 1
- Watchers: 10
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: COPYING
Awesome Lists containing this project
README
# Sample groonga tokenizer
## Install
Install libgroonga-dev.
Build this tokenizer.
% ./configure
% make
% sudo make install
## Usage
Register `tokenizers/sample`:
% groonga DB
> register tokenizers/sample
Now, you can use `TokenSample` that tokenizes the content by a
character:
> tokenizer TokenSample "abc d e f"
[
[
0,
0.0,
0.0
],
[
{
"value": "a",
"position": 0
},
{
"value": "b",
"position": 1
},
{
"value": "c",
"position": 2
},
{
"value": " ",
"position": 3
},
{
"value": "d",
"position": 4
},
{
"value": " ",
"position": 5
},
{
"value": "e",
"position": 6
},
{
"value": " ",
"position": 7
},
{
"value": "f",
"position": 8
}
]
]
## License
Public domain. You can copy and modify this project freely.