sqlrange

package module
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 14, 2024 License: BSD-3-Clause, MIT Imports: 7 Imported by: 0

README

sqlrange Go Reference

Library using the database/sql package and Go 1.22 range functions to execute queries against SQL databases.

Installation

This package is intended to be used as a library and installed with:

go get github.com/achille-roussel/sqlrange

⚠ The package depends on Go 1.22 (currently in rc1 release) and enabling the rangefunc experiment.

To download Go 1.22 rc1: https://pkg.golang.ir/golang.org/dl/go1.22rc1

go install golang.org/dl/go1.22rc1@latest
go1.22rc1 download

Then to enable the rangefunc experiment, set the GOEXPERIMENT environment variable in the shell that executes the go commands:

export GOEXPERIMENT=rangefunc

For a more detailed guide of how to configure Go 1.22 with range functions see Go 1.22 rc1 installation to enable the range functions experiment.

Usage

The sqlrange package contains two kinds of functions called Exec and Query which wrap the standard library's database/sql methods with the same names. The package adds stronger type safety and the ability to use range functions as iterators to pass values to the queries or retrieve results.

Note that sqlrange IS NOT AN ORM, it is a lightweight package which does not hide any of the details and simply provides library functions to structure applications that stream values in and out of databases.

Query

The Query functions are used to read streams of values from databases, in the same way that sql.(*DB).Query does, but using range functions to simplify the code constructs, and type parameters to automatically decode SQL results into Go struct values.

The type parameter must be a struct with fields containing "sql" struct tags to define the names of columns that the fields are mapped to:

type Point struct {
    X float64 `sql:"x"`
    Y float64 `sql:"y"`
}
for p, err := range sqlrange.Query[Point](db, `select x, y from points`) {
    if err != nil {
        ...
    }
    ...
}

Note that resource management here is automated by the range function returned by calling Query, the underlying *sql.Rows value is automatically closed when the program exits the body of the range loop consuming the rows.

Exec

The Exec functions are used to execute insert, update, or delete queries against databases, accepting a stream of parameters as arguments (in the form of a range function), and returning a stream of results.

Since the function will send multiple queries to the database, it is often preferable to apply it to a transaction (or a statement derived from a transaction via sql.(*Tx).Stmt) to ensure atomicity of the operation.

tx, err := db.Begin()
if err != nil {
    ...
}
defer tx.Rollback()

for r, err := range sqlrange.Exec(tx,
    `insert into table (col1, col2, col3) values (?, ?, ?)`,
    // This range function yields the values that will be inserted into
    // the database by executing the query above.
    func(yield func(RowType, error) bool) {
        ...
    },
    // Inject the arguments for the SQL query being executed.
    // The function is called for each value yielded by the range
    // function above.
    sqlrange.ExecArgs(func(args []any, row RowType) []any {
        return append(args, row.Col1, row.Col2, row.Col3)
    }),
) {
    // Each results of each execution are streamed and must be consumed
    // by the program to drive the operation.
    if err != nil {
        ...
    }
    ...
}

if err := tx.Commit(); err != nil {
    ...
}

Documentation

Overview

Package sqlrange integrates database/sql with Go 1.22 range functions.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Exec

func Exec[Row any](e Executable, query string, seq iter.Seq2[Row, error], opts ...ExecOption[Row]) iter.Seq2[sql.Result, error]

Exec is like ExecContext but it uses the background context.

Example
type Row struct {
	Age  int    `sql:"age"`
	Name string `sql:"name"`
}

db := newTestDB(new(testing.T), "people")
defer db.Close()

for res, err := range sqlrange.Exec(db, `INSERT|people|name=?,age=?`,
	func(yield func(Row, error) bool) {
		if !yield(Row{Age: 19, Name: "Luke"}, nil) {
			return
		}
		if !yield(Row{Age: 42, Name: "Hitchhiker"}, nil) {
			return
		}
	},
	sqlrange.ExecArgsFields[Row]("name", "age"),
) {
	if err != nil {
		log.Fatal()
	}
	rowsAffected, err := res.RowsAffected()
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(rowsAffected)
}
Output:

1
1

func ExecContext

func ExecContext[Row any](ctx context.Context, e Executable, query string, seq iter.Seq2[Row, error], opts ...ExecOption[Row]) iter.Seq2[sql.Result, error]

ExecContext executes a query for each row in the sequence.

To ensure that the query is executed atomicatlly, it is usually useful to call ExecContext on a transaction (sql.Tx), for example:

tx, err := db.BeginTx(ctx, nil)
if err != nil {
  ...
}
defer tx.Rollback()
for r, err := range sqlrange.ExecContext[RowType](ctx, tx, query, rows) {
  if err != nil {
    ...
  }
  ...
}
if err := tx.Commit(); err != nil {
  ...
}

Since the function makes one query execution for each row read from the sequence, latency of the query execution can quickly increase. In some cases, such as inserting values in a database, the program can amortize the cost of query latency by batching the rows being inserted, for example:

for r, err := range sqlrange.ExecContext(ctx, tx,
	`insert into table (col1, col2, col3) values `,
	// yield groups of rows to be inserted in bulk
	func(yield func([]RowType, error) bool) {
	  ...
	},
	// append values for the insert query
	sqlrange.ExecArgs(func(args []any, rows []RowType) []any {
	  for _, row := range rows {
	    args = append(args, row.Col1, row.Col2, row.Col3)
	  }
	  return args
	}),
	// generate placeholders for the insert query
	sqlrange.ExecQuery(func(query string, rows []RowType) string {
	  return query + strings.Repeat(`(?, ?, ?)`, len(rows))
	}),
) {
	...
}

Batching operations this way is necessary to achieve high throughput when inserting values into a database.

func Fields

Fields returns a sequence of the fields of a struct type that have a "sql" tag.

func Query

func Query[Row any](q Queryable, query string, args ...any) iter.Seq2[Row, error]

Query is like QueryContext but it uses the background context.

Example
type Row struct {
	Age  int    `sql:"age"`
	Name string `sql:"name"`
}

db := newTestDB(new(testing.T), "people")
defer db.Close()

for row, err := range sqlrange.Query[Row](db, `SELECT|people|age,name|`) {
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(row)
}
Output:

{1 Alice}
{2 Bob}
{3 Chris}

func QueryContext

func QueryContext[Row any](ctx context.Context, q Queryable, query string, args ...any) iter.Seq2[Row, error]

QueryContext returns the results of the query as a sequence of rows.

The returned function automatically closes the unerlying sql.Rows value when it completes its iteration. The function can only be iterated once, it will not retain the values that it has seen.

A typical use of QueryContext is:

for row, err := range sqlrange.QueryContext[RowType](ctx, db, query, args...) {
  if err != nil {
    ...
  }
  ...
}

The q parameter represents a queryable type, such as *sql.DB, *sql.Stmt, or *sql.Tx.

See Scan for more information about how the rows are mapped to the row type parameter Row.

func Scan

func Scan[Row any](rows *sql.Rows) iter.Seq2[Row, error]

Scan returns a sequence of rows from a sql.Rows value.

The returned function automatically closes the rows passed as argument when it completes its iteration. The function can only be iterated once, it will not retain the values that it has seen.

A typical use of Scan is:

rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
  ...
}
for row, err := range sqlrange.Scan[RowType](rows) {
  if err != nil {
    ...
  }
  ...
}

Scan uses reflection to map the columns of the rows to the fields of the struct passed as argument. The mapping is done by matching the name of the columns with the name of the fields. The name of the columns is taken from the "sql" tag of the fields. For example:

type Row struct {
  ID   int64  `sql:"id"`
  Name string `sql:"name"`
}

The fields of the struct that do not have a "sql" tag are ignored.

Ranging over the returned function will panic if the type parameter is not a struct.

Types

type ExecOption

type ExecOption[Row any] func(*execOptions[Row])

ExecOption is a functional option type to configure the Exec and ExecContext functions.

func ExecArgs

func ExecArgs[Row any](fn func([]any, Row) []any) ExecOption[Row]

ExecArgs is an option that specifies the function being called to generate the list of arguments passed when executing a query.

By default, the Row value is converted to a list of arguments by taking the fields with a "sql" struct tag in the order they appear in the struct, as defined by the reflect.VisibleFields function.

The function must append the arguments to the slice passed as argument and return the resulting slice.

func ExecArgsFields

func ExecArgsFields[Row any](columnNames ...string) ExecOption[Row]

ExecArgsFields constructs an option that specifies the fields to include in the query arguments from a list of column names.

This option is useful when the query only needs a subset of the fields from the row type, or when the query arguments are in a different order than the struct fields.

func ExecQuery

func ExecQuery[Row any](fn func(string, Row) string) ExecOption[Row]

ExecQuery is an option that specifies the function being called to generate the query to execute for a given Row value.

The function receives the original query value passed to Exec or ExecContext, and returns the query to execute.

This is useful when parts of the query depend on the Row value that the query is being executed on, for example when the query is an insert.

type Executable

type Executable interface {
	ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
}

Executable is the interface implemented by sql.DB, sql.Stmt, or sql.Tx.

type Queryable

type Queryable interface {
	QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
}

Queryable is an interface implemented by types that can send SQL queries, such as *sql.DB, *sql.Stmt, or *sql.Tx.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL