package
Version:
v0.0.0-...-c5bc608
Opens a new window with list of versions in this module.
Published: Jul 13, 2022
License: MIT
Opens a new window with license information.
Imports: 1
Opens a new window with list of imports.
Imported by: 0
Opens a new window with list of known importers.
Documentation
¶
func NewTokenizer(source string) *tokenizer
const (
ErrorToken TokenType = iota
IdentifierToken
OpenCurlyBraceToken
CloseCurlyBraceToken
OpenSquareBracketToken
CloseSquareBracketToken
ColonToken
SemicolonToken
CommaToken
WhitespaceToken
EOFToken
)
Source Files
¶
Click to show internal directories.
Click to hide internal directories.