Versions in this module Expand all Collapse all v0 v0.7.3 Feb 22, 2017 Changes in this version + func ICUDisabled() bool + type Tokenizer struct + func NewTokenizer(s []byte) (*Tokenizer, error) + func (t *Tokenizer) Destroy() + func (t *Tokenizer) Next() []byte + func (t *Tokenizer) Tokens() []string