Documentation
¶
Index ¶
- func AppendFile(path string) pipeline.Program
- func Basename() pipeline.Program
- func Buffer(buf []byte) pipeline.Program
- func BufferAll() pipeline.Program
- func Column(col int) pipeline.Program
- func Concat() pipeline.Program
- func CountLines() pipeline.Program
- func Dirname() pipeline.Program
- func Do(req *http.Request, c *http.Client) pipeline.Program
- func EachLine(process func(string, *strings.Builder)) pipeline.Programdeprecated
- func Echo(s string) pipeline.Program
- func Exec(name string, arg ...string) pipeline.Program
- func ExecForEach(builder func(line string) (name string, arg []string)) pipeline.Program
- func File(path string) pipeline.Program
- func FilterLine(filter func(string) string) pipeline.Program
- func FindFiles(dir string) pipeline.Program
- func First(n int) pipeline.Program
- func Freq() pipeline.Program
- func Get(url string, c *http.Client) pipeline.Program
- func IfExists(path string) pipeline.Program
- func Join() pipeline.Program
- func Last(n int) pipeline.Program
- func ListFiles(pattern string) pipeline.Program
- func Match(s string) pipeline.Program
- func MatchRegexp(re *regexp.Regexp) pipeline.Program
- func Post(url string, c *http.Client) pipeline.Program
- func Reject(s string) pipeline.Program
- func RejectRegexp(re *regexp.Regexp) pipeline.Program
- func Replace(search, replace string) pipeline.Program
- func ReplaceRegexp(re *regexp.Regexp, replace string) pipeline.Program
- func SHA256Sum() pipeline.Program
- func SHA256Sums() pipeline.Program
- func Scanner(filter func(string, io.Writer)) pipeline.Program
- func Stdin() pipeline.Program
- func Stdout() pipeline.Program
- func Tee(writers ...io.Writer) pipeline.Program
- func Wait() pipeline.Program
- func WriteFile(path string) pipeline.Program
- type DoProgram
- type Pipeline
- func (p *Pipeline[T]) AppendFile(path string) T
- func (p *Pipeline[T]) Basename() T
- func (p *Pipeline[T]) Column(col int) T
- func (p *Pipeline[T]) Concat() T
- func (p *Pipeline[T]) CountLines() T
- func (p *Pipeline[T]) Dirname() T
- func (p *Pipeline[T]) Do(req *http.Request, c *http.Client) T
- func (p *Pipeline[T]) EachLine(process func(string, *strings.Builder)) Tdeprecated
- func (p *Pipeline[T]) Echo(s string) T
- func (p *Pipeline[T]) Exec(name string, arg ...string) T
- func (p *Pipeline[T]) ExecForEach(builder func(line string) (string, []string)) T
- func (p *Pipeline[T]) FilterLine(filter func(string) string) T
- func (p *Pipeline[T]) First(n int) T
- func (p *Pipeline[T]) Freq() T
- func (p *Pipeline[T]) Get(url string, c *http.Client) T
- func (p *Pipeline[T]) Join() T
- func (p *Pipeline[T]) Last(n int) T
- func (p *Pipeline[T]) Match(s string) T
- func (p *Pipeline[T]) MatchRegexp(re *regexp.Regexp) T
- func (p *Pipeline[T]) Pipe(program pipeline.Program) T
- func (p *Pipeline[T]) Post(url string, c *http.Client) T
- func (p *Pipeline[T]) Reject(s string) T
- func (p *Pipeline[T]) RejectRegexp(re *regexp.Regexp) T
- func (p *Pipeline[T]) Replace(search, replace string) T
- func (p *Pipeline[T]) ReplaceRegexp(re *regexp.Regexp, replace string) T
- func (p *Pipeline[T]) SHA256Sum() T
- func (p *Pipeline[T]) SHA256Sums() T
- func (p *Pipeline[T]) Scanner(filter func(string, io.Writer)) T
- func (p *Pipeline[T]) Tee(writers ...io.Writer) T
- func (p *Pipeline[T]) Wait() T
- func (p *Pipeline[T]) WithError(err error) T
- func (p *Pipeline[T]) WithReader(r io.Reader) T
- func (p *Pipeline[T]) WithStderr(w io.Writer) T
- func (p *Pipeline[T]) WithStdout(w io.Writer) T
- func (p *Pipeline[T]) WriteFile(path string) T
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AppendFile ¶
AppendFile appends the contents of the pipe to the file path, creating it if necessary, and returns the number of bytes successfully written, or an error.
func Basename ¶
Basename reads paths from the pipe, one per line, and removes any leading directory components from each. So, for example, /usr/local/bin/foo would become just foo. This is the complementary operation to [Pipe.Dirname].
If any line is empty, Basename will transform it to a single dot. Trailing slashes are removed. The behaviour of Basename is the same as filepath.Base (not by coincidence).
func Buffer ¶ added in v0.0.3
Buffer copies the stream in a buffered manner using the provided buffer
func Column ¶
Column produces column col of each line of input, where the first column is column 1, and columns are delimited by Unicode whitespace. Lines containing fewer than col columns will be skipped.
func Concat ¶
Concat reads paths from the pipe, one per line, and produces the contents of all the corresponding files in sequence. If there are any errors (for example, non-existent files), these will be ignored, execution will continue, and the pipe's error status will not be set.
This makes it convenient to write programs that take a list of paths on the command line. For example:
script.Args().Concat().Stdout()
The list of paths could also come from a file:
script.File("filelist.txt").Concat()
Or from the output of a command:
script.Exec("ls /var/app/config/").Concat().Stdout()
Each input file will be closed once it has been fully read. If any of the files can't be opened or read, Concat will simply skip these and carry on, without setting the pipe's error status. This mimics the behaviour of Unix cat(1).
func CountLines ¶
CountLines returns the number of lines of input, or an error.
func Dirname ¶
Dirname reads paths from the pipe, one per line, and produces only the parent directories of each path. For example, /usr/local/bin/foo would become just /usr/local/bin. This is the complementary operation to [Pipe.Basename].
If a line is empty, Dirname will transform it to a single dot. Trailing slashes are removed, unless Dirname returns the root folder. Otherwise, the behaviour of Dirname is the same as filepath.Dir (not by coincidence).
func Do ¶
Do performs the HTTP request req using the pipe's configured HTTP client, as set by [Pipe.WithHTTPClient], or http.DefaultClient otherwise. The response body is streamed concurrently to the pipe's output. If the response status is anything other than HTTP 200-299, the pipe's error status is set.
func EachLine
deprecated
EachLine calls the function process on each line of input, passing it the line as a string, and a *strings.Builder to write its output to.
Deprecated: use [Pipe.FilterLine] or [Pipe.FilterScan] instead, which run concurrently and don't do unnecessary reads on the input.
func Echo ¶
Echo sets the pipe's reader to one that produces the string s, detaching any existing reader without draining or closing it.
func Exec ¶
Exec runs cmdLine as an external command, sending it the contents of the pipe as input, and produces the command's standard output (see below for error output). The effect of this is to filter the contents of the pipe through the external command.
Error handling ¶
If the command had a non-zero exit status, the pipe's error status will also be set to the string “exit status X”, where X is the integer exit status. Even in the event of a non-zero exit status, the command's output will still be available in the pipe. This is often helpful for debugging. However, because [Pipe.String] is a no-op if the pipe's error status is set, if you want output you will need to reset the error status before calling [Pipe.String].
If the command writes to its standard error stream, this will also go to the pipe, along with its standard output. However, the standard error text can instead be redirected to a supplied writer, using [Pipe.WithStderr].
func ExecForEach ¶
ExecForEach renders cmdLine as a Go template for each line of input, running the resulting command, and produces the combined output of all these commands in sequence. See [Pipe.Exec] for error handling details.
This is mostly useful for substituting data into commands using Go template syntax. For example:
ListFiles("*").ExecForEach("touch {{.}}").Wait()
func FilterLine ¶
FilterLine sends the contents of the pipe to the function filter, a line at a time, and produces the result. filter takes each line as a string and returns a string as its output. See [Pipe.Filter] for concurrency handling.
func FindFiles ¶
FindFiles creates a pipe listing all the files in the directory dir and its subdirectories recursively, one per line, like Unix find(1). If dir doesn't exist or can't be read, the pipe's error status will be set.
Each line of the output consists of a slash-separated path, starting with the initial directory. For example, if the directory looks like this:
test/
1.txt
2.txt
the pipe's output will be:
test/1.txt test/2.txt
func First ¶
First produces only the first n lines of the pipe's contents, or all the lines if there are less than n. If n is zero or negative, there is no output at all.
func Freq ¶
Freq produces only the unique lines from the pipe's contents, each prefixed with a frequency count, in descending numerical order (most frequent lines first). Lines with equal frequency will be sorted alphabetically.
For example, we could take a common shell pipeline like this:
sort input.txt |uniq -c |sort -rn
and replace it with:
File("input.txt").Freq().Stdout()
Or to get only the ten most common lines:
File("input.txt").Freq().First(10).Stdout()
Like Unix uniq(1), Freq right-justifies its count values in a column for readability, padding with spaces if necessary.
func Get ¶
Get makes an HTTP GET request to url, sending the contents of the pipe as the request body, and produces the server's response. See [Pipe.Do] for how the HTTP response status is interpreted.
func IfExists ¶
IfExists tests whether path exists, and creates a pipe whose error status reflects the result. If the file doesn't exist, the pipe's error status will be set, and if the file does exist, the pipe will have no error status. This can be used to do some operation only if a given file exists:
IfExists("/foo/bar").Exec("/usr/bin/something")
func Join ¶
Join joins all the lines in the pipe's contents into a single space-separated string, which will always end with a newline.
func Last ¶
Last produces only the last n lines of the pipe's contents, or all the lines if there are less than n. If n is zero or negative, there is no output at all.
func ListFiles ¶
ListFiles creates a pipe containing the files or directories specified by path, one per line. path can be a glob expression, as for filepath.Match. For example:
ListFiles("/data/*").Stdout()
ListFiles does not recurse into subdirectories; use FindFiles instead.
func MatchRegexp ¶
MatchRegexp produces only the input lines that match the compiled regexp re.
func Post ¶
Post makes an HTTP POST request to url, using the contents of the pipe as the request body, and produces the server's response. See [Pipe.Do] for how the HTTP response status is interpreted.
func RejectRegexp ¶
RejectRegexp produces only lines that don't match the compiled regexp re.
func ReplaceRegexp ¶
ReplaceRegexp replaces all matches of the compiled regexp re with the string replace. $x variables in the replace string are interpreted as by [regexp#Regexp.Expand]; for example, $1 represents the text of the first submatch.
func SHA256Sum ¶
SHA256Sum returns the hex-encoded SHA-256 hash of the entire contents of the pipe, or an error.
func SHA256Sums ¶
SHA256Sums reads paths from the pipe, one per line, and produces the hex-encoded SHA-256 hash of each corresponding file, one per line. Any files that cannot be opened or read will be ignored.
func Stdout ¶
Stdout copies the pipe's contents to its configured standard output (using [Pipe.WithStdout]), or to os.Stdout otherwise, and returns the number of bytes successfully written, together with any error.
func Tee ¶
Tee copies the pipe's contents to each of the supplied writers, like Unix tee(1). If no writers are supplied, the default is the pipe's standard output.
Types ¶
type DoProgram ¶ added in v0.0.3
type DoProgram struct {
pipeline.BaseProgram
// contains filtered or unexported fields
}
type Pipeline ¶
func NewPipeline ¶
func (*Pipeline[T]) AppendFile ¶
AppendFile reads the input and appends it to the file path, creating it if necessary, and outputs the number of bytes successfully written
func (*Pipeline[T]) Basename ¶
func (p *Pipeline[T]) Basename() T
Basename reads each line as a file path and outputs each path with any leading directory components removed
func (*Pipeline[T]) Column ¶
Column reads each line and outputs column col, where columns are whitespace delimited and the first column is column 1
func (*Pipeline[T]) Concat ¶
func (p *Pipeline[T]) Concat() T
Concat reads each line as a file path and outputs the file contents
func (*Pipeline[T]) CountLines ¶
func (p *Pipeline[T]) CountLines() T
CountLines returns the number of lines of input, or an error.
func (*Pipeline[T]) Dirname ¶
func (p *Pipeline[T]) Dirname() T
Dirname reads each line as a file path and outputs each path with just the leading directory remaining
func (*Pipeline[T]) Do ¶
Get reads the input as the request body, sends the request and outputs the response
func (*Pipeline[T]) Exec ¶
Exec executes the command with name and arguments, using input as stdin and outputs the result
func (*Pipeline[T]) ExecForEach ¶ added in v0.0.4
Exec executes the command with name and arguments, using input as stdin and outputs the result
func (*Pipeline[T]) FilterLine ¶
FilterLine reads the input, calls the function filter on each line and outputs the result
func (*Pipeline[T]) First ¶
First produces only the first n lines of the input and outputs only the first n number of lines
func (*Pipeline[T]) Freq ¶
func (p *Pipeline[T]) Freq() T
Freq reads the input and outputs only the unique lines, each prefixed with a frequency count, in descending numerical order
func (*Pipeline[T]) Get ¶
Get reads the input as the request body, sends a GET request and outputs the response
func (*Pipeline[T]) Join ¶
func (p *Pipeline[T]) Join() T
Join reads all the lines and joins them into a single space-separated string
func (*Pipeline[T]) MatchRegexp ¶
MatchRegexp reads the input and outputs lines that match the compiled regexp re
func (*Pipeline[T]) Pipe ¶
Get reads the input as the request body, sends a POST request and outputs the response
func (*Pipeline[T]) Post ¶
Get reads the input as the request body, sends a POST request and outputs the response
func (*Pipeline[T]) Reject ¶
Reject reads the input and outputs lines that do not contain the string s
func (*Pipeline[T]) RejectRegexp ¶
RejectRegexp reads the input and outputs lines that do not match the compiled regexp re
func (*Pipeline[T]) Replace ¶
Replace reads the input and replaces all occurrences of the string search with the string replace
func (*Pipeline[T]) ReplaceRegexp ¶
ReplaceRegexp reads the input and replaces all matches of the compiled regexp re with the string replace
func (*Pipeline[T]) SHA256Sum ¶
func (p *Pipeline[T]) SHA256Sum() T
SHA256Sum reads the input and outputs the hex-encoded SHA-256 hash
func (*Pipeline[T]) SHA256Sums ¶
func (p *Pipeline[T]) SHA256Sums() T
SHA256Sum reads the input and outputs the hex-encoded SHA-256 hash of each line
func (*Pipeline[T]) Scanner ¶
Scanner reads the input into a scanner, calls the function filter on each line and outputs the result
func (*Pipeline[T]) Tee ¶
Tee reads the input and copies it to each of the supplied writers, like Unix tee(1)
func (*Pipeline[T]) WithReader ¶
WithReader sets the pipe's input reader to r
func (*Pipeline[T]) WithStderr ¶
WithStderr redirects the standard error output for commands
func (*Pipeline[T]) WithStdout ¶
WithStdout sets the pipe's standard output to the writer w
Source Files