sqlds

package module
v3.0.0-...-890dad5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 11, 2024 License: Apache-2.0 Imports: 21 Imported by: 0

README

Build Status

sqlds

sqlds stands for SQL Datasource.

Most SQL-driven datasources, like Postgres, MySQL, and MSSQL share extremely similar codebases.

The sqlds package is intended to remove the repetition of these datasources and centralize the datasource logic. The only thing that the datasources themselves should have to define is connecting to the database, and what driver to use, and the plugin frontend.

Usage

if err := datasource.Manage("my-datasource", datasourceFactory, datasource.ManageOpts{}); err != nil {
  log.DefaultLogger.Error(err.Error())
  os.Exit(1)
}

func datasourceFactory(ctx context.Context, s backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
  ds := sqlds.NewDatasource(&myDatasource{})
  return ds.NewDatasource(ctx, s)
}

Standardization

Macros

The sqlds package defines a set of default macros:

  • $__timeFilter(time_column): Filters by timestamp using the query period. Resolves to: time >= '0001-01-01T00:00:00Z' AND time <= '0001-01-01T00:00:00Z'
  • $__timeFrom(time_column): Filters by timestamp using the start point of the query period. Resolves to time >= '0001-01-01T00:00:00Z'
  • $__timeTo(time_column): Filters by timestamp using the end point of the query period. Resolves to time <= '0001-01-01T00:00:00Z'
  • $__timeGroup(time_column, period): To group times based on a period. Resolves to (minute example): "datepart(year, time), datepart(month, time)'"
  • $__table: Returns the table configured in the query.
  • $__column: Returns the column configured in the query.

Documentation

Index

Constants

View Source
const MockDataFolder = "/mock-data"

MockDataFolder is the default folder that will contain data files

Variables

View Source
var (
	// ErrorNotImplemented is returned if the function is not implemented by the provided Driver (the Completable pointer is nil)
	ErrorNotImplemented = errors.New("not implemented")
	// ErrorWrongOptions when trying to parse Options with a invalid JSON
	ErrorWrongOptions = errors.New("error reading query options")
)
View Source
var (
	ErrorMissingMultipleConnectionsConfig = PluginError(errors.New("received connection arguments but the feature is not enabled"))
	ErrorMissingDBConnection              = PluginError(errors.New("unable to get default db connection"))
	HeaderKey                             = "grafana-http-headers"
	// Deprecated: ErrorMissingMultipleConnectionsConfig should be used instead
	MissingMultipleConnectionsConfig = ErrorMissingMultipleConnectionsConfig
	// Deprecated: ErrorMissingDBConnection should be used instead
	MissingDBConnection = ErrorMissingDBConnection
)
View Source
var (
	// ErrorBadDatasource is returned if the data source could not be asserted to the correct type (this should basically never happen?)
	ErrorBadDatasource = errors.New("type assertion to datasource failed")
	// ErrorJSON is returned when json.Unmarshal fails
	ErrorJSON = errors.New("error unmarshaling query JSON the Query Model")
	// ErrorQuery is returned when the query could not complete / execute
	ErrorQuery = errors.New("error querying the database")
	// ErrorTimeout is returned if the query has timed out
	ErrorTimeout = errors.New("query timeout exceeded")
	// ErrorNoResults is returned if there were no results returned
	ErrorNoResults = errors.New("no results returned from query")
)
View Source
var (
	// ErrorBadArgumentCount is returned from macros when the wrong number of arguments were provided
	ErrorBadArgumentCount = errors.New("unexpected number of arguments")
)

Functions

func CreateMockData

func CreateMockData(table string, folder string, csvData string) error

CreateData will create a "table" (csv file) in the data folder that can be queried with SQL

func CreateMockTable

func CreateMockTable(table string, folder string) error

Create will create a "table" (csv file) in the data folder that can be queried with SQL

func DownstreamError

func DownstreamError(err error, override ...bool) error

func ErrorSource

func ErrorSource(err error) backend.ErrorSource

func Interpolate

func Interpolate(driver Driver, query *Query) (string, error)

Interpolate returns an interpolated query string given a backend.DataQuery

func PluginError

func PluginError(err error, override ...bool) error

func QueryDB

func QueryDB(ctx context.Context, db Connection, converterGenerator *sqlutil.ConverterGenerator, converters []sqlutil.Converter, fillMode *data.FillMissing, query *Query, args ...interface{}) (data.Frames, error)

QueryDB sends the query to the connection and converts the rows to a dataframe.

Types

type Completable

type Completable interface {
	Schemas(ctx context.Context, options Options) ([]string, error)
	Tables(ctx context.Context, options Options) ([]string, error)
	Columns(ctx context.Context, options Options) ([]string, error)
}

Completable will be used to autocomplete Tables Schemas and Columns for SQL languages

type Connection

type Connection interface {
	Close() error
	Ping() error
	PingContext(ctx context.Context) error
	QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
}

Connection represents a SQL connection and is satisfied by the *sql.DB type For now, we only add the functions that we need / actively use. Some other candidates for future use could include the ExecContext and BeginTxContext functions

type Driver

type Driver interface {
	// Connect connects to the database. It does not need to call `db.Ping()`
	Connect(context.Context, backend.DataSourceInstanceSettings, json.RawMessage) (*sql.DB, error)
	// Settings are read whenever the plugin is initialized, or after the data source settings are updated
	Settings(context.Context, backend.DataSourceInstanceSettings) DriverSettings
	Macros() Macros
	Converters() []sqlutil.Converter
	ConverterGenerator() *sqlutil.ConverterGenerator
}

Driver is a simple interface that defines how to connect to a backend SQL datasource Plugin creators will need to implement this in order to create a managed datasource

type DriverSettings

type DriverSettings struct {
	Timeout        time.Duration
	FillMode       *data.FillMissing
	Retries        int
	Pause          int
	RetryOn        []string
	ForwardHeaders bool
	Errors         bool
}

type FormatQueryOption

type FormatQueryOption uint32

FormatQueryOption defines how the user has chosen to represent the data

const (
	// FormatOptionTimeSeries formats the query results as a timeseries using "WideToLong"
	FormatOptionTimeSeries FormatQueryOption = iota
	// FormatOptionTable formats the query results as a table using "LongToWide"
	FormatOptionTable
	// FormatOptionLogs sets the preferred visualization to logs
	FormatOptionLogs
	// FormatOptionsTrace sets the preferred visualization to trace
	FormatOptionTrace
)

type Macro

type Macro struct {
	Name string
	Args []string
}

type MacroFunc

type MacroFunc func(*Query, []string) (string, error)

MacroFunc defines a signature for applying a query macro Query macro implementations are defined by users / consumers of this package

type Macros

type Macros map[string]MacroFunc

Macros is a list of MacroFuncs. The "string" key is the name of the macro function. This name has to be regex friendly.

var DefaultMacros Macros = Macros{
	"timeFilter": macroTimeFilter,
	"timeFrom":   macroTimeFrom,
	"timeGroup":  macroTimeGroup,
	"timeTo":     macroTimeTo,
	"table":      macroTable,
	"column":     macroColumn,
}

type Options

type Options map[string]string

Options are used to query schemas, tables and columns. They will be encoded in the request body (e.g. {"database": "mydb"})

func ParseOptions

func ParseOptions(rawOptions json.RawMessage) (Options, error)

type Query

type Query struct {
	RawSQL         string            `json:"rawSql"`
	Format         FormatQueryOption `json:"format"`
	ConnectionArgs json.RawMessage   `json:"connectionArgs"`

	RefID         string            `json:"-"`
	Interval      time.Duration     `json:"-"`
	TimeRange     backend.TimeRange `json:"-"`
	MaxDataPoints int64             `json:"-"`
	FillMissing   *data.FillMissing `json:"fillMode,omitempty"`

	// Macros
	Schema string `json:"schema,omitempty"`
	Table  string `json:"table,omitempty"`
	Column string `json:"column,omitempty"`
}

Query is the model that represents the query that users submit from the panel / queryeditor. For the sake of backwards compatibility, when making changes to this type, ensure that changes are only additive.

func GetQuery

func GetQuery(query backend.DataQuery, headers http.Header, setHeaders bool) (*Query, error)

GetQuery returns a Query object given a backend.DataQuery using json.Unmarshal

func (*Query) WithSQL

func (q *Query) WithSQL(query string) *Query

WithSQL copies the Query, but with a different RawSQL value. This is mostly useful in the Interpolate function, where the RawSQL value is modified in a loop

type QueryArgSetter

type QueryArgSetter interface {
	SetQueryArgs(ctx context.Context, headers http.Header) []interface{}
}

QueryArgSetter is an additional interface that could be implemented by driver. This adds the ability to the driver to optionally set query args that are then sent down to the database.

type QueryMutator

type QueryMutator interface {
	MutateQuery(ctx context.Context, req backend.DataQuery) (context.Context, backend.DataQuery)
}

QueryMutator is an additional interface that could be implemented by driver. This adds ability to the driver it can mutate query before run.

type Response

type Response struct {
	// contains filtered or unexported fields
}

func NewResponse

func NewResponse(res *backend.QueryDataResponse) *Response

func (*Response) Response

func (r *Response) Response() *backend.QueryDataResponse

func (*Response) Set

func (r *Response) Set(refID string, res backend.DataResponse)

type ResponseMutator

type ResponseMutator interface {
	MutateResponse(ctx context.Context, res data.Frames) (data.Frames, error)
}

ResponseMutator is an additional interface that could be implemented by driver. This adds ability to the driver, so it can mutate a response from the driver before its returned to the client.

type SQLDatasource

type SQLDatasource struct {
	Completable

	backend.CallResourceHandler
	CustomRoutes map[string]func(http.ResponseWriter, *http.Request)
	// Enabling multiple connections may cause that concurrent connection limits
	// are hit. The datasource enabling this should make sure connections are cached
	// if necessary.
	EnableMultipleConnections bool
	// contains filtered or unexported fields
}

func NewDatasource

func NewDatasource(c Driver) *SQLDatasource

NewDatasource initializes the Datasource wrapper and instance manager

func (*SQLDatasource) CheckHealth

CheckHealth pings the connected SQL database

func (*SQLDatasource) Dispose

func (ds *SQLDatasource) Dispose()

Dispose cleans up datasource instance resources. Note: Called when testing and saving a datasource

func (*SQLDatasource) DriverSettings

func (ds *SQLDatasource) DriverSettings() DriverSettings

func (*SQLDatasource) GetDBFromQuery

func (ds *SQLDatasource) GetDBFromQuery(ctx context.Context, q *Query, datasourceUID string) (*sql.DB, error)

func (*SQLDatasource) NewDatasource

NewDatasource creates a new `SQLDatasource`. It uses the provided settings argument to call the ds.Driver to connect to the SQL server

func (*SQLDatasource) QueryData

QueryData creates the Responses list and executes each query

type SQLMock

type SQLMock struct {
	// contains filtered or unexported fields
}

SQLMock connects to a local folder with csv files

func (*SQLMock) Connect

Connect opens a sql.DB connection using datasource settings

func (*SQLMock) Converters

func (h *SQLMock) Converters() []sqlutil.Converter

Converters defines list of string convertors

func (*SQLMock) Macros

func (h *SQLMock) Macros() Macros

Macros returns list of macro functions convert the macros of raw query

Directories

Path Synopsis
csv

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL