icut

A fast Thai tokenization library


License
MIT
Install
pip install icut==0.0.4

Documentation

icut

A fast Thai word tokenization

ICU can do very fast tokenization. We use preprocessing and postprocessing after icu, allowing customization while keeping the speed.

Now only just ICU.