Documentation
¶
Overview ¶
Package scanner implements a scanner for yacc[0] source text with actions written in Go. It takes a []byte as source which can then be tokenized through repeated calls to the Scan method.
Changelog ¶
2014-12-18: Support %precedence for bison compatibility[1].
2014-11-17: Allow dashes in symbol names for bison compatibility[1].
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Scanner ¶
type Scanner struct {
Col int // Starting column of the last scanned token.
Errors []error // List of accumulated errors.
FName string // File name (reported) of the scanned source.
Line int // Starting line of the last scanned token.
NCol int // Starting column (reported) for the next scanned token.
NLine int // Starting line (reported) for the next scanned token.
// contains filtered or unexported fields
}
A Scanner holds the scanner's internal state while processing a given text.
func (*Scanner) Mode ¶
Mode allows to switch the scanner mode from scanning yacc tokens to scanning rule action tokens (Go tokens). Initially the yacc mode is active.
func (*Scanner) Scan ¶
Scan works like ScanRaw but recognizes also yacc's C_IDENTIFIER token (in yacc mode).
func (*Scanner) ScanRaw ¶
ScanRaw scans the next token and returns the token and its value if applicable. The source end is indicated by EOF.
If the returned token is IDENT, INT, FLOAT, IMAG, CHAR, STRING or COMMENT, lval has has the corresponding value - not the string representation of the value. However, numeric literals which overflow the corresponding Go predeclared types are returned as string.
If the returned token is DLR_NUM or DLR_TAG_DLR, num has the corresponding number (int) or lval has the corresponding tag (string).
If the returned token is DLR_TAG_NUM, lval has the corresponding tag (string) and num has the corresponding number.
If the returned token is ILLEGAL, the literal string is the offending character or number/string/char literal.
type Token ¶
type Token int
const ( // Special tokens ILLEGAL Token EOF COMMENT // Identifiers and basic type literals // (these tokens stand for classes of literals) IDENT // main INT // 12345 FLOAT // 123.45 IMAG // 123.45i CHAR // 'a' STRING // "abc" // Operators and delimiters ADD // + SUB // - MUL // * QUO // / REM // % AND // & OR // | XOR // ^ SHL // << SHR // >> AND_NOT // &^ ADD_ASSIGN // += SUB_ASSIGN // -= MUL_ASSIGN // *= QUO_ASSIGN // /= REM_ASSIGN // %= AND_ASSIGN // &= OR_ASSIGN // |= XOR_ASSIGN // ^= SHL_ASSIGN // <<= SHR_ASSIGN // >>= AND_NOT_ASSIGN // &^= LAND // && LOR // || ARROW // <- INC // ++ DEC // -- EQL // == LSS // < GTR // > ASSIGN // = NOT // ! NEQ // != LEQ // <= GEQ // >= DEFINE // := ELLIPSIS // ... LPAREN // ( LBRACK // [ LBRACE // { COMMA // , PERIOD // . RPAREN // ) RBRACK // ] RBRACE // } SEMICOLON // ; COLON // : // Keywords BREAK CASE CHAN CONST CONTINUE DEFAULT DEFER ELSE FALLTHROUGH FOR FUNC GO GOTO IF IMPORT INTERFACE MAP PACKAGE RANGE RETURN SELECT STRUCT SWITCH GO_TYPE VAR DLR_DLR // $$ DLR_NUM // $1 DLR_TAG_DLR // $<tag>$ DLR_TAG_NUM // $<tag>2 // yacc mode tokens C_IDENTIFIER // IDENTIFIER ':' ERR_VERBOSE // %error-verbose IDENTIFIER // [a-zA-Z_][a-zA-Z0-9_.-]* LCURL // %{ LEFT // %left MARK // %% NONASSOC // %nonassoc PREC // %prec RCURL // %} RIGHT // %right START // %start TOKEN // %token TYPE // %type UNION // %union PRECEDENCE // %precedence )
Source Files