redo

package module
v0.0.0-...-7d8d520 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 29, 2024 License: MIT Imports: 7 Imported by: 0

README

RedoGopher

Package redo is an ergonomic retry library for Go with decorrelated backoff.

PkgGoDev

Ergonomic?

The API is intended to be "ergonomic" in that it attempts to be intuitive to use and easy to integrate into existing code, without a lot of cognitive load.

To this end, it has the following features:

  • Declarative syntax to wrap existing functions.
  • Short, memorable retrier functions.
  • Support for functional options with sensible defaults as well as a RetryPolicy type to predeclare a set of options for re-use.

Supported Function Types

The following function types are supported:

Function Signature Retry Method(s)
func() error Fn
func()(OUT, error) FnOut
func(IN) error FnIn, FnInRefr
func(IN) (OUT, error) FnIO, FnIORefr
func(context.Context) error FnCtx
func(context.Context)(OUT, error) FnOutCtx
func(context.Context, IN) error FnInCtx, FnInCtxRefr
func(context.Context, IN) (OUT, error) FnIOCtx, FnIOCtxRefr

Retry Workflow

Functions are retried by invoking them with the appropriate package-level retry method. If the function fails, it will be run again after some delay. This process will continue until one of the following conditions occurs:

  • The function returns successfully with a nil error value.
  • The function exhausts its configured number of retries.
  • The function is halted by a function provided with HaltOn or Halt is used to manually return a fatal error.
  • The context is cancelled.
  • The refresh function, if used, fails, returning a *RefreshError.

In the case of context cancellation, context.Cause will be called on the context to get the underlying error, if set.

Example

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"andy.dev/redo"
)

var try = 0

func ReturnsString() (string, error) {
	try++
	if try != 2 {
		return "", fmt.Errorf("simulate an error")
	}
	return "my result", nil
}

func main() {
	policy := redo.Policy{
		InitialDelay: time.Second,
		MaxDelay:     2 * time.Minute,
		MaxTries:     5,
		Each: func(status redo.Status) {
			log.Printf("Returned error: %v (%+s)", status.Err, status)
		},
	}

	str, err := redo.FnOut(context.Background(), ReturnsString, redo.WithPolicy(policy))
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Got: %s", str)
}

Returned error: simulate an error (attempt 1/5 - next in 984ms)
Got: my result

Beta

The API is mostly complete; however semantics are still being experimented with. The number of unique retrier functions is too high, but this is required by the limitations of the current generic type inference algorithms. It might be possible to reduce them in the future (see the section on union interfaces below), but if that doesn't pan out I may consider splitting the package into context and contextless functions or removing one or the other class altogether to keep the API intuitive and uncluttered.

Union Interfaces

It would be nice to unify the -Ctx versions of retriers with those that don't require a context using a general interface union. Unfortunately,Go's type inference is not yet able to make sense of the following without explicit type parameters:

type FnOutT[OUT any] interface {
    func(context.Context) (OUT, error) | func() (OUT, error)
}

func FnOut[OUT any, F FnOutT[OUT]](ctx context.Context, fn F) (OUT, error) {
    var fa any = fn
    /* ... */
}

func ToRetry(ctx context.Context) (string, error){
    return "test", nil
}

func main(){
    // The following results in an error: "cannot infer OUT"
    str, err := FnOut(context.Background(), FnOut(ToRetry))

    // This is required instead, which defeats the point:
    str, err := FnOut[string](context.Background(), FnOut(ToRetry))
}

It's unclear if this will be supported any time soon, since support for type switching on union interfaces is complex and ongoing.

Refresh Functions
Refresh function signatures are a bit lengthy, and I may need to look into simplifying them.

Documentation

Overview

Package redo is an ergonomic retry library for Go.

It provides a set of generic retriers for functions of common signatures using a decorrelated soft exponential backoff delay to limit concurrent requests downstream.

Ergonomic?

The API is intended to be "ergonomic" in that it attempts to be intuitive to use and easy to integrate into existing code, without a lot of cognitive load.

To this end, it has the following features:

  • Declarative syntax to wrap existing code.
  • Short, memorable names for wrapping functions.
  • Support for functional options with sensible defaults as well as a Policy type to predeclare a set of options for re-use.

Supported Function Types

The following function types are supported: 

|           Function Signature           |   Retry Method(s)    |
|----------------------------------------|----------------------|
| func() error                           | Fn                   |
| func()(OUT, error)                     | FnOut                |
| func(IN) error                         | FnIn, FnInRefr       |
| func(IN) (OUT, error)                  | FnIO, FnIORefr       |
| func(context.Context) error            | FnCtx                |
| func(context.Context)(OUT, error)      | FnOutCtx             |
| func(context.Context, IN) error        | FnInCtx, FnInCtxRefr |
| func(context.Context, IN) (OUT, error) | FnIOCtx, FnIOCtxRefr |

Retry Workflow

Functions are retried by invoking them with the appropriate package-level retry method. If the function fails, it will be run again after some delay. This process will continue until one of the following conditions occurs:

  • The function returns with a nil error value.
  • The function exhausts its configured number of retries.
  • The function is halted by a HaltFn or Halt is used to manually return a fatal error.
  • The context is cancelled, or its deadline is exceeded.
  • The refresh function, if used, fails, returning a *RefreshError.

In the case of context cancellation, context.Cause will be called on the context as a convenience to get the underlying error. To disable this, see CtxCause.

Index

Examples

Constants

View Source 
const (
	DefaultInitialDelay = 1 * time.Second
	DefaultMaxDelay     = 20 * time.Minute
	DefaultMaxTries     = 10
)

Variables

This section is empty.

Functions

func Exhausted 

func Exhausted(e error) bool

Exhausted returns true if the error is the final result after all tries.

Example 
package main

import (
	"context"
	"fmt"

	"andy.dev/redo"
)

func someFunction() error {
	return fmt.Errorf("some error")
}

func main() {
	fnToRetry := func(ctx context.Context) error {
		if err := someFunction(); err != nil {
			fmt.Printf("there was a problem: %v\n", err)
			return err
		}
		return nil
	}

	err := redo.FnCtx(context.Background(), fnToRetry, redo.MaxTries(2))
	if err != nil {
		fmt.Println(err)
	}

	if redo.Exhausted(err) {
		fmt.Println("looks like that was it")
	}
}

Output:

there was a problem: some error
there was a problem: some error
some error
looks like that was it

func Fn 

func Fn(ctx context.Context,
	fn func() error,
	options ...Option,
) error

Fn is a retrier for functions with the signatures of: 

func() error

The error returned will be the ultimate error returned after all retries are complete or nil, in the case of a successful run. For more information on how functions will be retried and values returned, see the package documentation.

func FnCtx 

func FnCtx(
	ctx context.Context,
	fn func(context.Context) error,
	options ...Option,
) error

FnCtx is a retrier for functions with the following signature: 

func(context.Context) error

The error returned will be the ultimate error returned after all retries are complete or nil, in the case of a successful run. For more information on how functions will be retried and values returned, see the package documentation.

Example (WithCancelledContextCause) 
package main

import (
	"context"
	"errors"
	"fmt"
	"time"

	"andy.dev/redo"
)

func main() {
	ctx, cf := context.WithCancelCause(context.Background())
	go func() {
		time.Sleep(1 * time.Second)
		cf(errors.New("I've changed my mind"))
	}()

	fnToRetry := func(ctx context.Context) error {
		return errors.New("I'll fail forever")
	}

	err := redo.FnCtx(ctx, fnToRetry, redo.MaxTries(10))
	if err != nil {
		fmt.Println(err)
	}
}

Output:

I've changed my mind

func FnIO 

func FnIO[IN, OUT any](
	ctx context.Context,
	fn func(IN) (OUT, error),
	fnArg IN,
	options ...Option,
) (OUT, error)

FnIO is a retrier for functions with the signature of: 

func(IN)(OUT, ERROR)

Where IN is an input argument fnArg of any type and OUT is a return value of any type.

The function will be retried following the rules described in the package documentation, and will return the values of the first successful run or the final unsuccessful run. It is a combination of FnIn and FnOut.

func FnIOCtx 

func FnIOCtx[IN, OUT any](
	ctx context.Context,
	fn func(context.Context, IN) (OUT, error),
	fnArg IN,
	options ...Option,
) (OUT, error)

FnIO is a retrier for functions with the signature of: 

func(context.Context, IN)(OUT, ERROR)

Where IN is an input argument fnArg of any type and OUT is a return value of any type.

The function will be retried following the rules described in the package documentation, and will return the values of the first successful run or the final unsuccessful run. It is a combination of FnInCtx and FnOutCtx.

Example 
package main

import (
	"context"
	"fmt"

	"andy.dev/redo"
)

var fetchHttpCount = 0

func fetchHttp(_ context.Context, url string) ([]byte, error) {
	fetchHttpCount++
	if fetchHttpCount < 2 {
		return nil, fmt.Errorf("HTTP error fetching %s", url)
	}
	return []byte(`{"status":"success"}`), nil
}

func main() {
	val, err := redo.FnIOCtx(context.Background(), fetchHttp, "http://my.site.com", redo.MaxTries(3))
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Printf("%s", val)
}

Output:

{"status":"success"}

func FnIOCtxRefr 

func FnIOCtxRefr[IN, OUT any](
	ctx context.Context,
	fn func(context.Context, IN) (OUT, error),
	fnArg IN,
	refreshFn RefreshFn[IN],
	options ...Option,
) (OUT, error)

FnIOCtxRefr is a retrier for functions with the signature of: 

func(context.Context, IN)(OUT, ERROR)

Where IN is an input argument fnArg of any type and OUT is a return value of any type.The initial input value for fn is passed using the fnArg argument and will be refreshed using refreshFn for subsequent retries, if needed. It is a combination of FnInCtxRefr and FnOutCtx.

func FnIORefr 

func FnIORefr[IN, OUT any](
	ctx context.Context,
	fn func(IN) (OUT, error),
	fnArg IN,
	refreshFn RefreshFn[IN],
	options ...Option,
) (OUT, error)

FnIORefr is a retrier for functions with the signature of: 

func(IN)(OUT, ERROR)

Where IN is an input argument fnArg of any type and OUT is a return value of any type.The initial input value for fn is passed using the fnArg argument and will be refreshed using refreshFn for subsequent retries, if needed. It is a combination of FnInRefr and FnOut.

func FnIn 

func FnIn[IN any](
	ctx context.Context,
	fn func(IN) error,
	fnArg IN,
	options ...Option,
) error

FnIn is a retrier for functions with the signature of: 

func(IN) error

Where IN is an input argument fnArg of any type.

Note: fn is passed by value, separately from fnArg: 

FnIn(ctx, fnToRetry, <argument>) - CORRECT
FnIn(ctx, fnToRetry(<argument)) - INCORRECT

func FnInCtx 

func FnInCtx[IN any](
	ctx context.Context,
	fn func(context.Context, IN) error,
	fnArg IN,
	options ...Option,
) error

FnInCtx is a retrier for functions with the signature of: 

func(context.Context, IN) error

Where IN is an input argument fnArg of any type.

Note: fn is passed by value, separately from fnArg: 

FnInCtx(ctx, fnToRetry, <arg>) - CORRECT
FnInCtx(ctx, fnToRetry(arg)) - INCORRECT
Example 
package main

import (
	"context"
	"errors"
	"fmt"

	"andy.dev/redo"
)

func main() {
	fnToRetry := func(ctx context.Context, str string) error {
		try := redo.GetStatus(ctx).TryNumber
		fmt.Printf("try %d with arg: %q\n", try, str)
		if try < 3 {
			return errors.New("not yet")
		}
		return nil
	}

	err := redo.FnInCtx(context.Background(), fnToRetry, "my argument", redo.MaxTries(3))
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Printf("Success!")
}

Output:

try 1 with arg: "my argument"
try 2 with arg: "my argument"
try 3 with arg: "my argument"
Success!

func FnInCtxRefr 

func FnInCtxRefr[IN any](
	ctx context.Context,
	fn func(context.Context, IN) error,
	fnArg IN,
	refreshFn RefreshFn[IN],
	options ...Option,
) error

FnInCtxRefr is a retrier for functions with the signature of: 

func(context.Context, IN) error

Where IN is an input argument of any type. The initial value for this argument is passed using the fnArg argument and will be refreshed using refreshFn for subsequent retries, if needed.

func FnInRefr 

func FnInRefr[IN any](
	ctx context.Context,
	fn func(IN) error,
	refreshFn RefreshFn[IN],
	fnArg IN,
	options ...Option,
) error

FnInRefr is a retrier for functions with the signature of: 

func(IN) error

Where IN is an input argument of any type. The initial value for this argument is passed using the fnArg argument and will be refreshed using refreshFn for subsequent retries, if needed.

func FnOut 

func FnOut[OUT any](
	ctx context.Context,
	fn func() (OUT, error),
	options ...Option,
) (OUT, error)

FnOut is a retrier for functions with the signature of: 

func() (OUT, error)

Where OUT is a return value of any type.

The function will be retried following the rules described in the package documentation, and will return the values of the first successful run or the final unsuccessful run.

func FnOutCtx 

func FnOutCtx[OUT any](
	ctx context.Context,
	fn func(context.Context) (OUT, error),
	options ...Option,
) (OUT, error)

FnOutCtx is a retrier for functions with the signature of: 

func(context.Context) (OUT, error)

Where OUT is a return value of any type.

The function will be retried following the rules described in the package documentation, and will return the values of the first successful run or the final unsuccessful run.

Example 
package main

import (
	"context"
	"errors"
	"fmt"

	"andy.dev/redo"
)

func main() {
	fnToRetry := func(ctx context.Context) (string, error) {
		status := redo.GetStatus(ctx)
		try := status.TryNumber
		val := fmt.Sprintf("value from try %d", try)
		if try < 3 {
			return "", errors.New("not yet")
		}
		return val, nil
	}

	str, err := redo.FnOutCtx(context.Background(), fnToRetry, redo.MaxTries(3))
	if err != nil {
		fmt.Println(err)
	}
	fmt.Printf("Got: %s", str)
}

Output:

Got: value from try 3

func Halt 

func Halt(e error) *haltErr

Halt allows you to return a halting error from within the retry loop itself, as an alternative to using HaltFn. Simply: 

return retry.Halt(err)

To stop the retry run immediately.

func Halted 

func Halted(e error) bool

Halted returns true if the retry was manually halted by the user by returning. an error wrapped with Halt

Types

type Option 

type Option func(o *opts)

Option represents an optional retry setting.

func CtxCause 

func CtxCause(enabled bool) Option

CtxCause will enable or disable automatic context cancellation cause extraction. If enabled, redo will call context.Cause on all values of context.Canceled and context.DeadlineExceeded to get the underlying error, if it is set. Defaults to true, which enables this behavior

func Each 

func Each(eachFn func(Status)) Option

Each allows you to set a function to be called directly after each failed retry. It is passed a Status value that you can use for logging or reporting. Defaults to nil, which will take no action.

Example 
package main

import (
	"context"
	"fmt"

	"andy.dev/redo"
)

func someFunction() error {
	return fmt.Errorf("some error")
}

type testLogger struct{}

func (testLogger) Printf(msg string, a ...any) {
	fmt.Printf(msg+"\n", a...)
}

func (testLogger) Println(a ...any) {
	fmt.Println(a...)
}

var log testLogger

func main() {
	fnToRetry := func(ctx context.Context) error {
		if err := someFunction(); err != nil {
			return err
		}
		return nil
	}

	eachFn := func(s redo.Status) {
		log.Printf("got error while retrying: %v (%s)", s.Err, s)
	}

	err := redo.FnCtx(context.Background(), fnToRetry, redo.MaxTries(3), redo.Each(eachFn))
	if err != nil {
		log.Println(err)
	}
}

Output:

got error while retrying: some error (attempt 1/3)
got error while retrying: some error (attempt 2/3)
got error while retrying: some error (attempt 3/3)
some error

func FirstFast 

func FirstFast(firstRetryImmediate bool) Option

FirstFast defines whether or not the first retry should be made immediately. Defaults to false.

func HaltErrors 

func HaltErrors(errs ...error) Option

HaltErrors is a shortcut to writing a HaltFn of the form 

func(e error) bool {
    return errors.Is(e, Err1) || errors.Is(e, Err2) /* ... */
}

Note: context.Canceled and context.DeadlineExceeded, are already handled specially, so adding them using HaltErrors is a no-op.

func HaltFn 

func HaltFn(haltFn func(error) bool) Option

HaltFn allows you to set a function to use for identifying fatal errors. It will be called for each error returned from the target function. If it returns true, the retry loop will terminate immediately. Defaults to nil.

Note: this will not affect the processing of context.Canceled and context.DeadlineExceeded, which will always halt the retry loop.

func InitialDelay 

func InitialDelay(duration time.Duration) Option

InitialDelay sets the initial median delay of the first retry, and will serve to scale the rest of the run. If this is <= 0, it will default to DefaultInitialDelay (1 * time.Second)

func MaxDelay 

func MaxDelay(duration time.Duration) Option

MaxDelay will cap the exponential delay to a maximum value. If this is <= 0, it will default to DefaultMaxDelay (20 * time.Minutes) or InitialDelay, whichever is greater.

func MaxTries 

func MaxTries(tries int) Option

MaxTries is the number of tries to attempt. A negative value will retry until explicitly cancelled via context or a call to Halt. If unset, it will default to DefaultMaxTries (10)

func WithPolicy 

func WithPolicy(p Policy) Option

WithPolicy applies a the settings in a Policy to a run, allowing you to reuse a set of options for multiple functions.

type Policy 

type Policy struct {
	// Initial median delay.
	// Default: (1 * time.Second)
	InitialDelay time.Duration
	// Maximum delay allowed.
	// Default: (20*time.Minutes >= InitialDelay)
	MaxDelay time.Duration
	// Maximum number of tries to attempt.
	// Default: 10
	MaxTries int
	// Whether to retry the first time immdiaitely.
	// Default: false
	FirstFast bool
	// Halt allows you to set a function to check for fatal errors -- see [Halt]
	Halt func(error) bool
	// Each allows you to run a function directly after each failure -- see [Each]
	Each func(Status)
	// NoCtxCause disables automatic extraction of context cause -- see [CtxCause]
	NoCtxCause bool
}

Policy allows you to predefine all of the options for a retry run ahead of time and set them using WithPolicy

type RefreshError 

type RefreshError struct {
	// contains filtered or unexported fields
}

RefreshError will be returned if a RefreshFn returns an error. The underlying error that caused the retry will be combined with this error using errors.Join. If you would like to inspect just the original error, you can use errors.As to get the *RefreshError value and call the [RetryErr] Method.

func (*RefreshError) Error 

func (re *RefreshError) Error() string

Error implements the error interface.

func (*RefreshError) RetryErr 

func (re *RefreshError) RetryErr() error

RetryErr returns the error that caused the function to retry before the RefreshFn failed.

func (*RefreshError) Unwrap 

func (re *RefreshError) Unwrap() []error

Unwrap allows a *RefreshError to work with errors.Is and errors.As

type RefreshFn 

type RefreshFn[T any] func() (T, error)

RefreshFn is a function that can be passed to any of the -Refresh retriers to recreate or reset the input argument to the function between retries. If this function returns an error, it will be wrapped in a *RefreshError value, along with the underlying error that triggered the retry.

type RetryFn 

type RetryFn interface {
	func() error | func(context.Context) error
}

type RetryFnIO 

type RetryFnIO[IN, OUT any] interface {
	func(IN) (OUT, error) | func(context.Context, IN) (OUT, error)
}

type RetryFnIn 

type RetryFnIn[IN any] interface {
	func(IN) error | func(context.Context, IN) error
}

type RetryFnOut 

type RetryFnOut[OUT any] interface {
	func() (OUT, error) | func(context.Context) (OUT, error)
}

type Status 

type Status struct {
	TryNumber int
	MaxTries  int
	Err       error
	NextDelay time.Duration
}

Status represents the state of the current retry loop.GetStatus

func GetStatus 

func GetStatus(ctx context.Context) Status

GetStatus can be used to retrieve information about the current retry loop from within the function being retried, as opposed to setting a callback with Each. It will return Status{} if not called in a retry context, so make sure to use [Retrying] if your function might be run outside of a retry loop.

func (Status) Format 

func (s Status) Format(state fmt.State, verb rune)

Format implements fmt.Formatter it supports the %s and %q print verbs. Output is flag-dependent: 

%s -  "attempt #"
%+s - "attempt # - next in <duration>"

Where '#' is the attempt number as an integer such starting from '1' optionally followed by `/#` and the maximum number of tries if MaxTries is set.

func (Status) LogValue 

func (s Status) LogValue() slog.Value

LogValue implements slog.LogValuer, allowing the retry status to be logged as a slog.GroupValue

func (Status) Next 

func (s Status) Next() time.Time

Next returns a time.Time value representing the approximate time the next iteration will occur, assuming it has just failed.

func (Status) String 

func (s Status) String() string

String implements fmt.Stringer

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.