Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Token ¶
A Token represents a known token with its literal representation
func NewToken ¶
NewToken returns a new Token associated with the given Type typ, the Literal literal and the Position pos
func (Token) IsAssignOperator ¶
IsAssignOperator returns true for tokens corresponding to assignment operators and delimiters; it returns false otherwise.
func (Token) IsKeyword ¶
IsKeyword returns true for tokens corresponding to keywords; it returns false otherwise.
func (Token) IsLiteral ¶
IsLiteral returns true for tokens corresponding to identifiers and basic type literals; it returns false otherwise.
func (Token) IsOperator ¶
IsOperator returns true for tokens corresponding to operators and delimiters; it returns false otherwise.
type Type ¶
type Type int
A Type represents a type of a known token
const ( ILLEGAL Type = iota // An illegal/unknown character EOF // end of input IDENT CONST GLOBAL INT STRING ASSIGN // = ADDASSIGN // += SUBASSIGN // -= MULASSIGN // *= DIVASSIGN // /= MODASSIGN // %= PLUS // + MINUS // - BANG // ! ASTERISK // * SLASH // / MODULO // % AND // & LOGICALAND // && PIPE // | LOGICALOR // || LT // < LTE // <= GT // > GTE // >= EQ // == NOTEQ // != SPACESHIP // <=> LSHIFT // << HASHROCKET // => NEWLINE // \n COMMA SEMICOLON HASH // # CAPTURE // & DOT // . COLON // : LPAREN // ( RPAREN // ) LBRACE // { RBRACE // } LBRACKET // [ RBRACKET // ] SCOPE // :: AT // @ QMARK // ? SYMBEG // : DEF SELF END IF THEN ELSE UNLESS TRUE FALSE RETURN NIL MODULE CLASS DO YIELD BEGIN RESCUE WHILE KEYWORD__FILE__ )
Recognized token types
func LookupIdent ¶
LookupIdent returns a keyword Type if ident is a keyword. If ident starts with an upper character it returns CONST. In any other case it returns IDENT
func (Type) IsAssignOperator ¶
IsAssignOperator returns true for tokens corresponding to assignment operators and delimiters; it returns false otherwise.
func (Type) IsKeyword ¶
IsKeyword returns true for tokens corresponding to keywords; it returns false otherwise.
func (Type) IsLiteral ¶
IsLiteral returns true for tokens corresponding to identifiers and basic type literals; it returns false otherwise.
func (Type) IsOperator ¶
IsOperator returns true for tokens corresponding to operators and delimiters; it returns false otherwise.
func (Type) String ¶
String returns the string corresponding to the token tok. For operators, delimiters, and keywords the string is the actual token character sequence (e.g., for the token ADD, the string is "+"). For all other tokens the string corresponds to the token constant name (e.g. for the token IDENT, the string is "IDENT").
Source Files