parallel_downloader

package module

v0.0.0-...-985536d Latest Latest Go to latest Published: Dec 6, 2023 License: MIT Imports: 8 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/mlasalmo/parallel_downloader

Links

Open Source Insights

README ¶

Parallel downloader

Parallel downloader is a tool for downloading files in parallel.

Usage

go run cmd/main.go \
  -url https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2018-05.parquet \
  -output yellow_tripdata_2018-05.parquet

References

https://www.353solutions.com/c/znga/dld.html

Documentation ¶

Overview ¶

Package parallel_downloader provides a simple way to download files in parallel.

Index ¶

func CreateEmptyFile(path string, size int64) error
func DownloadChunk(url string, destFile *os.File, offset, size int64, retries int, ...)
func GetFileHandle(path string) *os.File

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func CreateEmptyFile ¶

func CreateEmptyFile(path string, size int64) error

CreateEmptyFile creates an empty file with the given size

func DownloadChunk ¶

func DownloadChunk(url string, destFile *os.File, offset, size int64, retries int, wg *sync.WaitGroup, ch chan<- error)

DownloadChunk downloads a portion of the file and writes it to the correct offset in the destination file

func GetFileHandle ¶

func GetFileHandle(path string) *os.File

GetFileHandle opens the file for writing and seeks to the specified offset

Types ¶

This section is empty.

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
cmd

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL