https://github.com/jiegec/tantivy-jieba

An adapter that bridges between tantivy and jieba-rs.
https://github.com/jiegec/tantivy-jieba

bridges crates jieba-rs tantivy

Last synced: 4 days ago
JSON representation

An adapter that bridges between tantivy and jieba-rs.

• Host: GitHub
• URL: https://github.com/jiegec/tantivy-jieba
• Owner: jiegec
• License: mit
• Created: 2019-02-13T19:58:40.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2024-08-18T20:33:03.000Z (3 months ago)
• Last Synced: 2024-10-26T22:48:32.216Z (18 days ago)
• Topics: bridges, crates, jieba-rs, tantivy
• Language: Rust
• Homepage:
• Size: 30.3 KB
• Stars: 46
• Watchers: 5
• Forks: 10
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# tantivy-jieba

[![Crates.io version][crate-img]][crate]

[![docs.rs][docs-img]][docs]

[![Changelog][changelog-img]][changelog]

[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fjiegec%2Ftantivy-jieba.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Fjiegec%2Ftantivy-jieba?ref=badge_shield)

An adapter that bridges between tantivy and jieba-rs.

## Usage

Add dependency `tantivy-jieba` to your `Cargo.toml`.

### Example

```rust

use tantivy::tokenizer::*;

let mut tokenizer = tantivy_jieba::JiebaTokenizer {};

let mut token_stream = tokenizer.token_stream("测试");

assert_eq!(token_stream.next().unwrap().text, "测试");

assert!(token_stream.next().is_none());

```

### Register tantivy tokenizer

```rust

use tantivy::schema::Schema;

use tantivy::tokenizer::*;

use tantivy::Index;

let tokenizer = tantivy_jieba::JiebaTokenizer {};

let index = Index::create_in_ram(schema);

index.tokenizers()

     .register("jieba", tokenizer);

```

See [examples/mod.rs](examples/mod.rs) for detailed example.

[crate-img]:     https://img.shields.io/crates/v/tantivy-jieba.svg

[crate]:         https://crates.io/crates/tantivy-jieba

[changelog-img]: https://img.shields.io/badge/changelog-online-blue.svg

[changelog]:     https://github.com/jiegec/tantivy-jieba/blob/master/CHANGELOG.md

[docs-img]:      https://docs.rs/tantivy-jieba/badge.svg

[docs]:          https://docs.rs/tantivy-jieba

## License

[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fjiegec%2Ftantivy-jieba.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fjiegec%2Ftantivy-jieba?ref=badge_large)