Documentation ¶
Overview ¶
Package scanner is a subpackage of ptk that contains various implementations of the Scanner, along with related support types and code such as Char and Location. A scanner is an object that can be queried for the next character in a character stream, such as a file or a list of arguments; its return value is a Char, which packages together a Unicode rune and a location that may be used for error reporting.
Index ¶
- Constants
- Variables
- func LocationError(loc Location, err error) error
- type ArgJoiner
- type ArgLocation
- type ArgOption
- type ArgPos
- type ChainingScanner
- type Char
- type EncodingErrorHandler
- type EncodingErrorOption
- type FileLocation
- type FileOption
- type FilePos
- type FileScanner
- type LineDis
- type LineStyle
- type ListScanner
- type Location
- type MemoizingScanner
- type Scanner
- type TabStop
Constants ¶
const DefaultTabStop = 8
DefaultTabStop is the default tab stop for the scanner.
const (
EOF rune = unicode.MaxRune + iota + 1 // Signals end-of-file
)
Special runes used to signal end-of-file or other conditions in the scanners.
Variables ¶
var ( ErrSplitLocation = errors.New("Attempt to range file location through an incompatible location") ErrBadEncoding = errors.New("Invalid UTF-8 encoding") )
Simple errors that may be generated within the package.
var DOSLineStyle = &dosLineStyle{}
DOSLineStyle is a style for handling the use of carriage return and newline line endings, also known as DOS line endings.
var MacLineStyle = &macLineStyle{}
MacLineStyle is a style for handling the use of bare carriage return line endings, also known as Mac classic line endings.
var NoLineStyle = &noLineStyle{}
NoLineStyle is a style for handling the case when no newlines in the source should be recognized. All carriage returns and newlines will be substituted with spaces.
var UNIXLineStyle = &unixLineStyle{}
UNIXLineStyle is a style for handling the use of bare newline line endings, also known as UNIX line endings.
var UnknownLineStyle = &unknownLineStyle{}
UnknownLineStyle is a style for handling files where the line ending style is not yet known. It guesses the line ending style from the first line ending encountered, then switches to the appropriate line ending style.
Functions ¶
func LocationError ¶
LocationError wraps an error and includes a location.
Types ¶
type ArgJoiner ¶
type ArgJoiner string
ArgJoiner is an argument option that specifies the string that should logically be expected between each argument. By default, this is a single space (" ").
type ArgLocation ¶
ArgLocation is an implementation of Location that identifies the location of an element within a list of strings, typically command line arguments. It represents a full range, but tab stops and newlines are not treated specially.
func (ArgLocation) Incr ¶
func (l ArgLocation) Incr(c rune, tabstop int) Location
Incr increments the location by one character. It is passed the character (a rune) and the tabstop size (for handling tabs). It should return a new Location.
func (ArgLocation) String ¶
func (l ArgLocation) String() string
String constructs a string representation of the location.
type ArgOption ¶
type ArgOption interface {
// contains filtered or unexported methods
}
ArgOption is an option that may be passed to the NewArgumentScanner function.
type ArgPos ¶
type ArgPos struct { I int // The index of the argument in its list C int // The index of the character within the argument }
ArgPos specifies a particular character location within an argument list. It is a component of the ArgLocation type.
type ChainingScanner ¶
type ChainingScanner struct {
// contains filtered or unexported fields
}
ChainingScanner is a scanner that chains together several scanners. When one scanner returns EOF, the ChainingScanner skips it and proceeds to the next one. Note that if any call to Next returns an error, that error is returned immediately.
func NewChainingScanner ¶
func NewChainingScanner(streams []Scanner) *ChainingScanner
NewChainingScanner constructs and returns a Scanner implementation that returns characters from each of the provided scanners in turn. When one scanner returns EOF, that EOF is skipped and the chaining scanner begins drawing characters from the next one. If any scanner returns an error, that error is returned immediately.
func (*ChainingScanner) Next ¶
func (ccs *ChainingScanner) Next() (Char, error)
Next returns the next character from the stream as a Char, which will include the character's location. If an error was encountered, that will also be returned.
type Char ¶
type Char struct { Rune rune // The rune read from the source Loc Location // The location of the rune within the stream }
Char represents a character retrieved from the source input stream. It bundles together a rune and a location.
type EncodingErrorHandler ¶
type EncodingErrorHandler interface { // Handle handles the reported encoding error. If it returns // non-nil, the scanner will report an error. Handle(e error) error }
EncodingErrorHandler is an interface for an encoding error handler. A scanner will call the Handle method of the error handler, which must return either nil or an error (which may be the same error).
type EncodingErrorOption ¶
type EncodingErrorOption struct {
// contains filtered or unexported fields
}
EncodingErrorOption is the type that stores the encoding error handler that the file scanner should use.
func EncodingError ¶
func EncodingError(enc EncodingErrorHandler) EncodingErrorOption
EncodingError is an option that may be passed to either NewFileScanner or NewArgumentScanner. It is used to specify an EncodingErrorHandler to use to handle encoding errors.
type FileLocation ¶
type FileLocation struct { File string // Name of the file B FilePos // The beginning of the range E FilePos // The end of the range }
FileLocation is an implementation of Location that identifies the location of an element within a file. It represents a full range, and has some additional utilities to simplify handling advancement within the file, including of tab stops.
func (FileLocation) Incr ¶
func (l FileLocation) Incr(c rune, tabstop int) Location
Incr increments the location by one character. It is passed the character (a rune) and the tabstop size (for handling tabs). It should return a new Location.
func (FileLocation) String ¶
func (l FileLocation) String() string
String constructs a string representation of the location.
type FileOption ¶
type FileOption interface {
// contains filtered or unexported methods
}
FileOption is an option that may be passed to the NewFileScanner function.
func LineEndings ¶
func LineEndings(ls LineStyle) FileOption
LineEndings is a file scanner option that may be used to set the preferred line ending style. A line ending style is an instance of LineStyle that controls how the scanner recognizes newlines. The scanner always converts line endings into single newlines.
type FilePos ¶
type FilePos struct { L int // The line number of the position (1-indexed) C int // The column number of the position (1-indexed) }
FilePos specifies a particular character location within a file. It is a component of the FileLocation type.
type FileScanner ¶
type FileScanner struct {
// contains filtered or unexported fields
}
FileScanner is an implementation of Scanner that wraps io.Reader objects. FileScanner also contains logic to handle diverse line ending styles through the use of LineStyle objects; line endings will be converted into single newlines if this support is enabled.
func NewFileScanner ¶
func NewFileScanner(r io.Reader, loc Location, options ...FileOption) *FileScanner
NewFileScanner constructs a new instance of the FileScanner.
func (*FileScanner) Next ¶
func (s *FileScanner) Next() (Char, error)
Next returns the next character from the stream as a Char, which will include the character's location. If an error was encountered, that will also be returned.
type LineStyle ¶
type LineStyle interface { // Handle checks to see if a line ending sequence has been // encountered. It returns a LineDis value, which indicates // the disposition of the character; and a LineStyle object to // use next time around. Handle(chs []rune) (LineDis, LineStyle) }
LineStyle represents a line ending style. This controls how lines are handled when read from the source input stream, and is used for maintaining the location that is then attached to runes.
type ListScanner ¶
type ListScanner struct {
// contains filtered or unexported fields
}
ListScanner is a scanner that returns characters from a simple list. It is intended for testing in cases where MockScanner is not a good fit.
func NewListScanner ¶
func NewListScanner(chars []Char, err error) *ListScanner
NewListScanner constructs and returns a Scanner implementation that returns characters from a list of characters. The last character should be a EOF; this character will be returned with the error passed in. This scanner is intended for testing.
func (*ListScanner) Next ¶
func (lcs *ListScanner) Next() (Char, error)
Next returns the next character from the stream as a Char, which will include the character's location. If an error was encountered, that will also be returned.
type Location ¶
type Location interface { // String constructs a string representation of the location. String() string // Thru creates a new Location that ranges from the beginning // of this location to the beginning of another Location. Thru(other Location) (Location, error) // ThruEnd is similar to Thru, except that it creates a new // Location that ranges from the beginning of this location to // the ending of another location. ThruEnd(other Location) (Location, error) // Incr increments the location by one character. It is // passed the character (a rune) and the tabstop size (for // handling tabs). It should return a new Location. Incr(c rune, tabstop int) Location }
Location is an interface for location data. Each token and node should have attached location data that reports its location. This aids in finding the location of errors.
func LocationOf ¶
LocationOf attempts to retrieve the location of an error. If the location is not available, it returns nil.
type MemoizingScanner ¶
type MemoizingScanner struct {
// contains filtered or unexported fields
}
MemoizingScanner is a scanner that wraps another scanner and records the characters that it returns. Once the wrapped scanner returns an EOF, the MemoizingScanner will loop back around and simply return the same characters over and over.
func NewMemoizingScanner ¶
func NewMemoizingScanner(src Scanner) *MemoizingScanner
NewMemoizingScanner constructs and returns a Scanner implementation that wraps another and simply returns all the characters it returns. After the EOF character is returned--which is passed through unchanged--the scanner is replayed over and over. If the source scanner returns an error, that error is reported immediately.
func (*MemoizingScanner) Next ¶
func (mcs *MemoizingScanner) Next() (Char, error)
Next returns the next character from the stream as a Char, which will include the character's location. If an error was encountered, that will also be returned.
type Scanner ¶
type Scanner interface { // Next returns the next character from the stream as a Char, // which will include the character's location. If an error // was encountered, that will also be returned. Next() (Char, error) }
Scanner presents a stream of characters. The basic scanner does not provide backtracking or character push-back. The FileScanner is one implementation of Scanner.
func NewArgumentScanner ¶
NewArgumentScanner constructs and returns a Scanner implementation that returns characters drawn from a provided list of argument strings. This is intended for use with arguments taken from the command line, but could be useful in other contexts as well. The strings are logically joined by spaces; to use a different joiner, pass that as an option.