https://github.com/chakrit/node-icu-wordsplit

`npm install icu-wordsplit` -- Break string into words in many many languages. Simple icu boundary analysis module bindings for node.js
https://github.com/chakrit/node-icu-wordsplit

Last synced: about 20 hours ago
JSON representation

`npm install icu-wordsplit` -- Break string into words in many many languages. Simple icu boundary analysis module bindings for node.js

• Host: GitHub
• URL: https://github.com/chakrit/node-icu-wordsplit
• Owner: chakrit
• Created: 2012-06-09T15:57:59.000Z (over 12 years ago)
• Default Branch: master
• Last Pushed: 2017-06-20T14:41:11.000Z (over 7 years ago)
• Last Synced: 2024-10-10T22:39:39.708Z (about 1 month ago)
• Language: JavaScript
• Homepage:
• Size: 42 KB
• Stars: 29
• Watchers: 3
• Forks: 6
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGES.md

Awesome Lists containing this project

README

        
# node-icu-wordsplit

First, install libicu from source (so you have all the needed headers and binaries)

```sh

wget http://download.icu-project.org/files/icu4c/50.1/icu4c-50_1-src.tgz

tar -xvzf icu4c-50_1-src.tgz

cd icu/source

./configure

make

make install # or sudo make install

ldconfig # refresh linker cache

```

This will take a while. After it finishes you should have all the required ICU

headers and binaries in your default build path ready to be used.

Afterwards, switch to your project folder and

    npm install icu-wordsplit

And you should now be able to `require('icu-wordsplit')` in your code.

## API

Right now the only function call available is exported from the module.

#### splitWords ( [locale], string )

Splits the specified string into words using ICU-defined word boundary.

    var splitWords = require('icu-wordsplit');

    var results = splitWords('The quick brown fox jumps over the lazy dog.');

    console.log(results);

    results = splitWords('th_TH', 'แยกคำภาษาไทยก็ทำได้นะจ้ะ');

    console.log(results);

The first argument is the locale. You *must* specify an ICU-compatible locale name here.

However, this argument is now optional as you can use `en_US` to split most

strings without problem.

The second argument is the string to which to split.

The function will returns an array of words. Whitespaces and some common punctuations

may be automatically removed from the list.

## LICENSE

BSD

## TODO / CONTRIBUTE

All contributions welcome!

Compile and run tests with

```sh

make test

```

Here's something to do:

* More tests with more languages.

* Add more ICU functionality (like locale auto-detection.)

* Fix gyp and C++ build problems once and for all.