goscrapy-cli

goscrapy-cli

goscrapy-cli is easy to use as a command line tool, which is helpful to create goscrapy project and spiders for your own project.

create a new project

goscrapy-cli create project -n newproj -o ./output_dir

create a new spider

goscrapy-cli create spider -n MyFirstSpider -o ./spiders -pkg spiders

after this, a new spider named MyFirstSpider will be created at ./spiders/myfirstspider.go

create a new pipeline

goscrapy-cli create pipeline -n demopipeline -o ./pipelines -pkg pipelines --item itemA --item itemB