Documentation
¶
Index ¶
Constants ¶
const ( TokenRoleNone = TokenRole(iota) TokenRoleOperator TokenRoleKeyword TokenRoleNumber TokenRoleString TokenRoleComment )
const ( TokenRoleCustom1 = TokenRole((1 << 16) + iota) TokenRoleCustom2 TokenRoleCustom3 TokenRoleCustom4 TokenRoleCustom5 TokenRoleCustom6 TokenRoleCustom7 TokenRoleCustom8 TokenRoleCustom9 TokenRoleCustom10 TokenRoleCustom11 TokenRoleCustom12 TokenRoleCustom13 TokenRoleCustom14 TokenRoleCustom15 TokenRoleCustom16 )
Variables ¶
var FailedResult = Result{}
FailedResult represents a failed parse.
Functions ¶
This section is empty.
Types ¶
type ComputedToken ¶ added in v0.3.0
type ComputedToken struct {
// Offset is the token's start position,
// defined relative to the computation's start position.
Offset uint64
Length uint64
Role TokenRole
}
ComputedToken is a token recognized by a computation.
type Edit ¶
type Edit struct {
// contains filtered or unexported fields
}
Edit represents a change to a document.
func NewDeleteEdit ¶ added in v0.3.0
func NewInsertEdit ¶ added in v0.3.0
type EmptyState ¶ added in v0.3.0
type EmptyState struct{}
EmptyState represents an empty state. An empty state is considered equal to every other empty state.
func (EmptyState) Equals ¶ added in v0.3.0
func (s EmptyState) Equals(other State) bool
type Func ¶ added in v0.3.0
type Func func(TrackingRuneIter, State) Result
Func incrementally parses a document into tokens.
It returns the number of tokens consumed and a slice of tokens. The output MUST be deterministic based solely on the input args.
Each invocation of the function is cached and may be reused when reparsing the document after an edit.
The returned tokens must be sequential, non-overlapping, have non-zero length, and have positions within the range of consumed characters.
Every successful parse must consume at least one rune.
The state parameter allows the parse func to track state across invocations. The initial state is always EmptyState. The parse func must return a non-nil state, which will be passed back to the parse func on the next invocation.
func (Func) Map ¶ added in v0.3.0
Map maps a successful parse result to another parse result using mapFn.
func (Func) MapWithInput ¶ added in v0.3.0
func (f Func) MapWithInput(mapFn MapWithInputFn) Func
MapWithInput maps a successful parse to another parse result according to mapFn. The input iterator will output only runes consumed by the result before returning EOF.
func (Func) MaybeBefore ¶ added in v0.5.0
MaybeBefore produces a parse func that attempts `f` then `nextFn`, falling back to `nextFn` if `f` fails.
func (Func) Or ¶ added in v0.3.0
Or produces a parse func that returns the result of `f` if it succeeds, or the result of `nextFn` if `f` fails.
func (Func) Then ¶ added in v0.3.0
Then produces a parse func that succeeds if both `f` and `nextFn` succeed.
type MapWithInputFn ¶ added in v0.3.0
type MapWithInputFn func(Result, TrackingRuneIter, State) Result
MapWithInputFn maps a successful parse result to another parse result, using the original input (iter + state) that produced the first parse result.
type P ¶ added in v0.3.0
type P struct {
// contains filtered or unexported fields
}
P parses a document into tokens. It caches the results from the last parse so it can efficiently reparse a document after an edit (insertion/deletion).
func (*P) ReparseAfterEdit ¶ added in v0.3.0
ReparseAfterEdit parses a document after an edit (insertion/deletion), re-using cached results from previous computations when possible. This should be called *after* at least one invocation of ParseAll(). It must be called for *every* edit to the document, otherwise the tokens may not match the current state of the document.
func (*P) TokenAtPosition ¶ added in v0.7.0
TokenAtPosition returns the token containing a position. If no such token exists, it returns the Token zero value.
func (*P) TokensIntersectingRange ¶ added in v0.3.0
TokensIntersectingRange returns tokens that overlap the interval [startPos, endPos)
type Result ¶ added in v0.3.0
type Result struct {
NumConsumed uint64
ComputedTokens []ComputedToken
NextState State
}
Result represents the result of a single execution of a parse function.
func (Result) ShiftForward ¶ added in v0.3.0
ShiftForward shifts the result offsets forward by the specified number of positions.
type TokenRole ¶
type TokenRole int
TokenRole represents the role a token plays in a document, as interpreted in a particular syntax language.
type TrackingRuneIter ¶ added in v0.3.0
type TrackingRuneIter struct {
// contains filtered or unexported fields
}
TrackingReader tracks the number of runes read by a reader iter and all its clones. It updates a shared counter, so clones of this iter should NOT be used in other threads. Copying the struct produces a new, independent iterator.
func NewTrackingRuneIter ¶ added in v0.3.0
func NewTrackingRuneIter(reader text.Reader) TrackingRuneIter
NewTrackingRuneIter starts tracking an existing rune iter.
func (*TrackingRuneIter) Limit ¶ added in v0.6.0
func (iter *TrackingRuneIter) Limit(n uint64)
Limit sets the maximum number of runes this reader can produce.
func (*TrackingRuneIter) MaxRead ¶ added in v0.3.0
func (iter *TrackingRuneIter) MaxRead() uint64
MaxRead returns the maximum number of runes read by this iter and all its clones.
func (*TrackingRuneIter) NextRune ¶ added in v0.3.0
func (iter *TrackingRuneIter) NextRune() (rune, error)
NextRune returns the next rune from the underlying reader and advances the iterator.
func (*TrackingRuneIter) Skip ¶ added in v0.3.0
func (iter *TrackingRuneIter) Skip(n uint64) uint64
Skip advances the iterator by the specified number of positions or the end of the file, whichever comes first.
Source Files