The highest tagged major version is
Documentation
¶
Overview ¶
Package iterator adds various (record and Lesser) iterators utilities
Index ¶
- Variables
- func Equals(l1, l2 Lesser) bool
- func NewLesserChanFromIterator(it LesserIterator) chan Lesser
- func NewLesserPipe() (LesserWriter, LesserIterator)
- func NewRecordPipe() (RecordWriter, RecordIterator)
- type BtreeSyncedLesser
- func (tree *BtreeSyncedLesser) ClearLesser(addNodesToFreelist bool)
- func (tree *BtreeSyncedLesser) CloneLesser() *BtreeSyncedLesser
- func (tree *BtreeSyncedLesser) DeleteLesser(item Lesser) Lesser
- func (tree *BtreeSyncedLesser) DeleteMaxLesser() Lesser
- func (tree *BtreeSyncedLesser) DeleteMinLesser() Lesser
- func (tree *BtreeSyncedLesser) GetLesser(key Lesser) Lesser
- func (tree *BtreeSyncedLesser) HasLesser(key Lesser) bool
- func (tree *BtreeSyncedLesser) LenLesser() int
- func (tree *BtreeSyncedLesser) MaxLesser() Lesser
- func (tree *BtreeSyncedLesser) MinLesser() Lesser
- func (tree *BtreeSyncedLesser) ReplaceOrInsertLesser(item Lesser) Lesser
- type Lesser
- type LesserIterator
- func CombineLesserIterators(iterators ...LesserIterator) LesserIterator
- func LogLesserterator(it LesserIterator, pattern string) LesserIterator
- func NewBufferedRecordIteratorBTree(ri LesserIterator, bufferSize int) LesserIterator
- func NewLesserPipeFromChan(recChan chan Lesser) LesserIterator
- func SortedLesserIterators(LesserIterators []LesserIterator) (LesserIterator, error)
- type LesserIteratorClustered
- type LesserWriter
- type RecordIterator
- func CombineIterators(iterators ...RecordIterator) RecordIterator
- func DeduplicateRecordIterators(it RecordIterator) RecordIterator
- func JSONRecordIterator(new func() interface{}, r io.Reader) RecordIterator
- func LogRecordIterator(it RecordIterator, pattern string) RecordIterator
- func SortedRecordIterators(iterators []RecordIterator) (RecordIterator, error)
- type RecordWriter
Constants ¶
This section is empty.
Variables ¶
var ( // ErrIteratorStop is returned by RecordIterators where there are not more records to be found. ErrIteratorStop = errors.New("iterator stop") // ErrNotLesser is returned if records yeilded from a recorditerator can't be changed to Lessers ErrNotLesser = errors.New("record does not implement the Lesser interface and can't be sorted") )
Functions ¶
func Equals ¶
Equals checks if two items are equal to each other but ensuring neiher is less than the other.
func NewLesserChanFromIterator ¶
func NewLesserChanFromIterator(it LesserIterator) chan Lesser
NewLesserChanFromIterator returns a channels form an iterator; any iterator error will close the channel
func NewLesserPipe ¶
func NewLesserPipe() (LesserWriter, LesserIterator)
NewLesserPipe returns a pipe from writer to Iterator. Bandaid solution for cases where providing an iterator is not feasable and a writer iterface is required. Uses channels under the hood
func NewRecordPipe ¶
func NewRecordPipe() (RecordWriter, RecordIterator)
NewRecordPipe returns a pipe from writer to Iterator. Bandaid solution for cases where providing an iterator is not feasable and a writer iterface is required. Uses channels under the hood
Types ¶
type BtreeSyncedLesser ¶
BtreeSyncedLesser is an mutex protected version of github.com/google/btree with Lesser typed methods.
func NewBtreeSyncedLesser ¶
func NewBtreeSyncedLesser(degree int) *BtreeSyncedLesser
func (*BtreeSyncedLesser) ClearLesser ¶
func (tree *BtreeSyncedLesser) ClearLesser(addNodesToFreelist bool)
func (*BtreeSyncedLesser) CloneLesser ¶
func (tree *BtreeSyncedLesser) CloneLesser() *BtreeSyncedLesser
func (*BtreeSyncedLesser) DeleteLesser ¶
func (tree *BtreeSyncedLesser) DeleteLesser(item Lesser) Lesser
func (*BtreeSyncedLesser) DeleteMaxLesser ¶
func (tree *BtreeSyncedLesser) DeleteMaxLesser() Lesser
func (*BtreeSyncedLesser) DeleteMinLesser ¶
func (tree *BtreeSyncedLesser) DeleteMinLesser() Lesser
func (*BtreeSyncedLesser) GetLesser ¶
func (tree *BtreeSyncedLesser) GetLesser(key Lesser) Lesser
func (*BtreeSyncedLesser) HasLesser ¶
func (tree *BtreeSyncedLesser) HasLesser(key Lesser) bool
func (*BtreeSyncedLesser) LenLesser ¶
func (tree *BtreeSyncedLesser) LenLesser() int
func (*BtreeSyncedLesser) MaxLesser ¶
func (tree *BtreeSyncedLesser) MaxLesser() Lesser
func (*BtreeSyncedLesser) MinLesser ¶
func (tree *BtreeSyncedLesser) MinLesser() Lesser
func (*BtreeSyncedLesser) ReplaceOrInsertLesser ¶
func (tree *BtreeSyncedLesser) ReplaceOrInsertLesser(item Lesser) Lesser
type Lesser ¶
type Lesser interface {
Less(other interface{}) bool
}
Lesser is implemented by type which can be compared to each other and should answer i'm I/this/self less than the other record (argument 1)
type LesserIterator ¶
LesserIterator iterators is the function interface
func CombineLesserIterators ¶ added in v0.0.5
func CombineLesserIterators(iterators ...LesserIterator) LesserIterator
CombineLesserIterators will yeild the results from each of the consisting iterators the error ErrIteratorStop is expected to progress to the next iterator. To combine LesserIterators in sorted fashion use SortedLesserIterators()
func LogLesserterator ¶ added in v0.0.12
func LogLesserterator(it LesserIterator, pattern string) LesserIterator
LogLesserterator prints the contents of the record prior to returning it using the pattern as the fmt-directive where the first argument is the records second is the error as such. log.Printf(pattern, r, err)
func NewBufferedRecordIteratorBTree ¶
func NewBufferedRecordIteratorBTree(ri LesserIterator, bufferSize int) LesserIterator
NewBufferedRecordIteratorBTree creates a B+Tree of bufferSize from which records are emitted in sorted order. useful when sortable records are recieved out of order and should be emitted in (best effort) sorted order. Will block until the underlying LesserIterator has yeilded at least bufferSize items or returned an error, as such iterators can unblock with virtual errors such as ErrIteratorStall which will propagate up and can be discarded by the consumer.
func NewLesserPipeFromChan ¶
func NewLesserPipeFromChan(recChan chan Lesser) LesserIterator
NewLesserPipeFromChan turns a channel into an interator; will yeild ErrIteratorStop when the chan is closed
func SortedLesserIterators ¶
func SortedLesserIterators(LesserIterators []LesserIterator) (LesserIterator, error)
SortedLesserIterators combines a list of iterators; always yeilding the lowest value available from all iterators. To do this it keeps a local "peak cache" of the next value for each iterator. This means that iterators that produces data from volatile sources (e.g time) might be experience unexpected results.
func (LesserIterator) ToRecordIterator ¶
func (it LesserIterator) ToRecordIterator() RecordIterator
type LesserIteratorClustered ¶
LesserIteratorClustered iterators is the function interface
func NewBufferedClusterIteartor ¶
func NewBufferedClusterIteartor(ri LesserIterator, bufferSize int) LesserIteratorClustered
NewBufferedClusterIteartor works like NewBufferedRecordIteratorBTree but returns the cluster-id the records was returned from with the guarrantee that within each cluster; records are always in sorted order. Good for handling out of order records. NOTE: to give this guarrantee a cache of records must be keep in memory effectively creating an memory leak. The best way to avoid this memory leak to grow is to reset the iterator from time to time. Fututre iterators might include a GC based on inactivity
func (LesserIteratorClustered) ToLesserIterator ¶
func (it LesserIteratorClustered) ToLesserIterator() LesserIterator
type LesserWriter ¶
LesserWriter writes lesser records. Writing nil will close the pipe
type RecordIterator ¶
type RecordIterator func() (interface{}, error)
RecordIterator is a function which yeild any golang data struct each time called Where there are no more records; ErrIteratorStop should be returned and should not be treated as an error (compare it to io.EOF)
func CombineIterators ¶ added in v0.0.5
func CombineIterators(iterators ...RecordIterator) RecordIterator
CombineIterators will yeild the results from each of the consisting iterators the error ErrIteratorStop is expected to progress to the next iterator
func DeduplicateRecordIterators ¶ added in v0.0.9
func DeduplicateRecordIterators(it RecordIterator) RecordIterator
DeduplicateRecordIterators works like the unix command uniq where if two records are equal (ussing .Less(other) bool as comparison func) only the first are emitted. If not both records implement .Less(interface{})bool DeduplicateRecordIterators is a NOP (with overhead). Nil records are never considred equal
func JSONRecordIterator ¶
func JSONRecordIterator(new func() interface{}, r io.Reader) RecordIterator
JSONRecordIterator returns a RecordIterator based on a JSON stream of data, NewLine delimited @new - creator to allocate a new struct for each record; used to allow concurrent use of records yeilded @r - byte stream reader containing new line delimited json data
func LogRecordIterator ¶ added in v0.0.12
func LogRecordIterator(it RecordIterator, pattern string) RecordIterator
LogRecordIterator prints the contents of the record prior to returning it using the pattern as the fmt-directive where the first argument is the records second is the error as such. log.Printf(pattern, r, err)
func SortedRecordIterators ¶
func SortedRecordIterators(iterators []RecordIterator) (RecordIterator, error)
SortedRecordIterators combines a list of iterators; always yeilding the lowest value (if the records are of type Lesser) available from all iterators. To do this it keeps a local "peak cache" of the next value for each iterator. This means that iterators that produces data from volatile sources (e.g time) might be experience unexpected results.
func (RecordIterator) ToLesserIterator ¶
func (it RecordIterator) ToLesserIterator() LesserIterator
type RecordWriter ¶
type RecordWriter func(interface{}) error
RecordWriter writes records. Writing nil will close the pipe
Source Files
Directories