https://github.com/jdeng/mmseg

MMSEG simple word segmenter in C++ 11
https://github.com/jdeng/mmseg

Last synced: 12 days ago
JSON representation

MMSEG simple word segmenter in C++ 11

• Host: GitHub
• URL: https://github.com/jdeng/mmseg
• Owner: jdeng
• License: mit
• Created: 2014-07-19T18:35:35.000Z (over 10 years ago)
• Default Branch: master
• Last Pushed: 2014-07-19T20:11:18.000Z (over 10 years ago)
• Last Synced: 2024-12-18T22:02:13.241Z (28 days ago)
• Language: C
• Size: 1.07 MB
• Stars: 17
• Watchers: 4
• Forks: 4
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
mmseg

=====

MMSEG simple word segmenter in C++ 11 

  * Based on http://yongsun.me/2013/06/simple-implementation-of-mmseg-with-python/

  * Data files from mmseg4j https://code.google.com/p/mmseg4j/

  * Compile with: g++ -Ofast -march=native -funroll-loops -DMMSEG_MAIN -x c++ -o mmseg -std=c++11 mmseg.h

Usage

===

  	MMSeg mmseg;

    mmseg.load("words.dic", "chars.dic");

  	std::u16string s = MMSeg::from_utf8(MMSeg::trim(line));

    for (auto& w: mmseg.segment(s)) 

	    std::cout << MMSeg::to_utf8(w) << "  "; 

  	std::cout << std::endl;