engine

package
v0.0.0-...-bc502ad Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 7, 2023 License: MIT Imports: 2 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type ConcurrentEngine

type ConcurrentEngine struct {
	Scheduler   Scheduler
	WorkerCount int
	ItemSaver   chan interface{}
}

func (*ConcurrentEngine) Run

func (e *ConcurrentEngine) Run(seeds ...Request)

type ParserResult

type ParserResult struct {
	Requests []Request
	Items    []interface{}
}

func NilParserFunc

func NilParserFunc(content []byte) ParserResult

type ReadyNotifier

type ReadyNotifier interface {
	WorkerReady(chan Request)
}

type Request

type Request struct {
	Url        string
	ParserFunc func([]byte) ParserResult
}

type Scheduler

type Scheduler interface {
	ReadyNotifier
	Submit(Request)
	WorkChan() chan Request
	Run()
}

type SimpleEngine

type SimpleEngine struct {
}

func (SimpleEngine) Run

func (e SimpleEngine) Run(seeds ...Request)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL