Documentation ¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AddEdgesIfDoNotExist ¶
func AddEdgesIfDoNotExist( currentNode string, neighborNodes []string, ) ( neighborsAdded []string, err error, )
adds edge to DB, returns new neighbors added (to crawl on)
func FilterPage ¶
func FilterPage(e *colly.HTMLElement) (*colly.HTMLElement, error)
filters down full page body to elements we want to focus on
func GetRandomNode ¶
func IsValidCrawlLink ¶
Types ¶
This section is empty.
Click to show internal directories.
Click to hide internal directories.