token

package

v0.0.0-...-017d103 Latest Latest Go to latest Published: Jun 5, 2023 License: BSD-3-Clause Imports: 1 Imported by: 4

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

Documentation ¶

Overview ¶

Package token defines constants representing the lexical tokens of the bibtex language and basic operations on tokens (printing, predicates).

Index ¶

type Token

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Token ¶

type Token int

Token is the set of lexical tokens for bibtex.

const (
	Illegal Token = iota
	EOF
	TexComment // % foo

	Abbrev   // @STRING, @string
	Comment  // @COMMENT, @comment
	Preamble // @PREAMBLE, @pReAmble
	BibEntry // @article, @book, etc

	Ident       // author
	String      // "abc"
	BraceString // {abc}
	Number      // 2005

	DoubleQuote     // " - delimits a string
	StringLBrace    // {
	StringRBrace    // }
	StringSpace     // any consecutive whitespace '\n', '\r', '\t', ' ' in a bibtex string
	StringNBSP      // ~ - non-breakable space in LaTeX
	StringContents  // anything else inside a string
	StringHyphen    // - a hyphen counts as a token separator when parsing names
	StringMath      // $...$
	StringComma     // , - useful for parsing author names
	StringBackslash // \&, \$, \{, \_ - single character escape for special bibtex chars
	StringAccent    // \'{o} == ȯ, \ae == æ
	StringMacro     // \url, \(, \[, \, - TeX macro, either alphabetical or a single non-alphabetical char

	Assign // =
	LParen // (
	LBrace // {
	RParen // )
	RBrace // }
	Concat // #
	Comma  // ,

)

func (Token) IsCommand ¶

func (tok Token) IsCommand() bool

IsCommand returns true for tokens corresponding to commands. It returns false otherwise.

func (Token) IsLiteral ¶

func (tok Token) IsLiteral() bool

IsLiteral returns true for tokens corresponding to identifiers and basic type literals. It returns false otherwise.

func (Token) IsOperator ¶

func (tok Token) IsOperator() bool

IsOperator returns true for tokens corresponding to operators and delimiters. It returns false otherwise.

func (Token) IsStringLiteral ¶

func (tok Token) IsStringLiteral() bool

IsStringLiteral returns true for tokens corresponding to tokens within a string literal. It returns false otherwise.

func (Token) String ¶

func (tok Token) String() string

Source Files ¶

View all Source files

token.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL