downloader

package
v1.3.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 15, 2020 License: Apache-2.0 Imports: 7 Imported by: 43

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	SurferDownloader = &Surfer{
		surf:    surfer.New(cookieJar),
		phantom: surfer.NewPhantom(config.PHANTOMJS, config.PHANTOMJS_TEMP, cookieJar),
	}
)

Functions

This section is empty.

Types

type Downloader

type Downloader interface {
	Download(*spider.Spider, *request.Request) *spider.Context
}

The Downloader interface. You can implement the interface by implement function Download. Function Download need to return Page instance pointer that has request result downloaded from Request.

type Surfer

type Surfer struct {
	// contains filtered or unexported fields
}

func (*Surfer) Download

func (self *Surfer) Download(sp *spider.Spider, cReq *request.Request) *spider.Context

Directories

Path Synopsis
surfer是一款Go语言编写的高并发web下载器,支持 GET/POST/HEAD 方法及 http/https 协议,同时支持固定UserAgent自动保存cookie与随机大量UserAgent禁用cookie两种模式,高度模拟浏览器行为,可实现模拟登录等功能。
surfer是一款Go语言编写的高并发web下载器,支持 GET/POST/HEAD 方法及 http/https 协议,同时支持固定UserAgent自动保存cookie与随机大量UserAgent禁用cookie两种模式,高度模拟浏览器行为,可实现模拟登录等功能。
agent
Package agent generates user agents strings for well known browsers and for custom browsers.
Package agent generates user agents strings for well known browsers and for custom browsers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL