https://github.com/opennmt/lua-sentencepiece
lua wrapper for sentencepiece library
https://github.com/opennmt/lua-sentencepiece
Last synced: 3 months ago
JSON representation
lua wrapper for sentencepiece library
- Host: GitHub
- URL: https://github.com/opennmt/lua-sentencepiece
- Owner: OpenNMT
- License: mit
- Created: 2017-12-05T14:48:12.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2018-06-23T07:13:19.000Z (over 7 years ago)
- Last Synced: 2025-07-14T10:22:52.952Z (6 months ago)
- Language: C++
- Size: 597 KB
- Stars: 4
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
This repository defines a Lua wrapper for [sentencepiece](https://github.com/google/sentencepiece).
## Installation
1\. Install Lua development libraries (**optional if [Torch](http://torch.ch/docs/getting-started.html) is installed**):
On Ubuntu, they can be installed with apt-get:
```bash
sudo apt-get install lua5.1 liblua5.1-dev
```
2\. Install [sentencepiece](https://github.com/google/sentencepiece)
3\. Install the wrapper:
```bash
luarocks make lua-sentencepiece-scm-1.rockspec
```
## Test and use
```
$(which luajit lua | head -n 1) test.lua
```
The module creates a `sentencepiece` namespace with 2 functions:
* `sentencepiece.encode(model, string)`: encodes the string (convert into pieces table)
* `sentencepiece.decode(model, table)`: builds a string from the table of piece