README
¶
PostgreSQL Lock Client for Go
The PostgreSQL Lock Client for Go is a general purpose distributed locking library built for PostgreSQL. The PostgreSQL Lock Client for Go supports both fine-grained and coarse-grained locking as the lock keys can be any arbitrary string, up to a certain length. Please create issues in the GitHub repository with questions, pull request are very much welcome.
Recommended PostgreSQL version: 11 or newer
Use cases
A common use case for this lock client is: let's say you have a distributed system that needs to periodically do work on a given campaign (or a given customer, or any other object) and you want to make sure that two boxes don't work on the same campaign/customer at the same time. An easy way to fix this is to write a system that takes a lock on a customer, but fine-grained locking is a tough problem. This library attempts to simplify this locking problem on top of PostgreSQL.
Another use case is leader election. If you only want one host to be the leader, then this lock client is a great way to pick one. When the leader fails, it will fail over to another host within a customizable lease duration that you set.
Getting Started
To use the PostgreSQL Lock Client for Go, you must make it sure it is present in
$GOPATH or in your vendor directory.
$ go get -u cirello.io/pglock
This package has the go.mod file to be used with Go's module system. If you
need to work on this package, use go mod edit -replace=cirello.io/pglock@yourlocalcopy.
For your convenience, there is a function in the package called CreateTable
that you can use to set up your table, or you may use the schema.sql file.
The package level documentation comment has an example of how to use this
package. Here is some example code to get you started:
package main
import (
"log"
"cirello.io/pglock"
)
func main() {
db, err := sql.Open("postgres", *dsn)
if err != nil {
log.Fatal("cannot connect to test database server:", err)
}
c, err := pglock.New(db,
pglock.WithLeaseDuration(3*time.Second),
pglock.WithHeartbeatFrequency(1*time.Second),
)
if err != nil {
log.Fatal("cannot create lock client:", err)
}
if err := c.CreateTable(); err != nil {
log.Fatal("cannot create table:", err)
}
l, err := c.Acquire("lock-name")
if err != nil {
log.Fatal("unexpected error while acquiring 1st lock:", err)
}
defer l.Close()
// execute the logic
}
Selected Features
Send Automatic Heartbeats
When you create the lock client, you can specify WithHeartbeatFrequency(time.Duration)
like in the above example, and it will spawn a background goroutine that
continually updates the record version number on your locks to prevent them from
expiring (it does this by calling the SendHeartbeat() method in the lock
client.) This will ensure that as long as your application is running, your
locks will not expire until you call Release() or lockItem.Close()
Read the data in a lock without acquiring it
You can read the data in the lock without acquiring it. Here's how:
lock, err := lockClient.Get("kirk");
Logic to avoid problems with clock skew
The lock client never stores absolute times in PostgreSQL. The way locks are expired is that a call to tryAcquire reads in the current lock, checks the record version number of the lock and starts a timer. If the lock still has the same after the lease duration time has passed, the client will determine that the lock is stale and expire it.
What this means is that, even if two different machines disagree about what time it is, they will still avoid clobbering each other's locks.
Go Version Compatibility Promise
This package follows the same guidance as the Go's:
Each major Go release is supported until there are two newer major releases. For example, Go 1.5 was supported until the Go 1.7 release, and Go 1.6 was supported until the Go 1.8 release. We fix critical problems, including critical security problems, in supported releases as needed by issuing minor revisions (for example, Go 1.6.1, Go 1.6.2, and so on).
Documentation
¶
Overview ¶
Package pglock provides a simple utility for using PostgreSQL for managing distributed locks.
In order to use this package, the client must create a table in the database, although the client provides a convenience method for creating that table (CreateTable).
Basic usage:
db, err := sql.Open("postgres", *dsn)
if err != nil {
log.Fatal("cannot connect to test database server:", err)
}
name := randStr(32)
c, err := pglock.New(db)
if err != nil {
log.Fatal("cannot create lock client:", err)
}
l1, err := c.Acquire(name)
if err != nil {
log.Fatal("unexpected error while acquiring 1st lock:", err)
}
t.Log("acquired first lock")
var wg sync.WaitGroup
wg.Add(1)
var locked bool
go func() {
defer wg.Done()
l2, err := c.Acquire(name)
if err != nil {
log.Fatal("unexpected error while acquiring 2nd lock:", err)
}
t.Log("acquired second lock")
locked = true
l2.Close()
}()
time.Sleep(6 * time.Second)
l1.Close()
wg.Wait()
pglock.Client.Do can be used for long-running processes:
err = c.Do(context.Background(), name, func(ctx context.Context, l *pglock.Lock) error {
once := make(chan struct{}, 1)
once <- struct{}{}
for {
select {
case <-ctx.Done():
t.Log("context canceled")
return ctx.Err()
case <-once:
t.Log("executed once")
close(ranOnce)
}
}
})
if err != nil && err != context.Canceled {
log.Fatal("unexpected error while running under lock:", err)
}
This package is covered by this SLA: https://github.com/cirello-io/public/blob/master/SLA.md
Index ¶
- Constants
- Variables
- type Client
- func (c *Client) Acquire(name string, opts ...LockOption) (*Lock, error)
- func (c *Client) AcquireContext(ctx context.Context, name string, opts ...LockOption) (*Lock, error)
- func (c *Client) CreateTable() error
- func (c *Client) Do(ctx context.Context, name string, f func(context.Context, *Lock) error, ...) error
- func (c *Client) DropTable() error
- func (c *Client) Get(name string) (*Lock, error)
- func (c *Client) GetAllLocks() ([]*ReadOnlyLock, error)
- func (c *Client) GetAllLocksContext(ctx context.Context) ([]*ReadOnlyLock, error)
- func (c *Client) GetContext(ctx context.Context, name string) (*Lock, error)
- func (c *Client) GetData(name string) ([]byte, error)
- func (c *Client) GetDataContext(ctx context.Context, name string) ([]byte, error)
- func (c *Client) Release(l *Lock) error
- func (c *Client) ReleaseContext(ctx context.Context, l *Lock) error
- func (c *Client) SendHeartbeat(ctx context.Context, l *Lock) error
- func (c *Client) TryCreateTable() error
- type ClientOption
- type FailedPreconditionError
- type LevelLogger
- type Lock
- type LockOption
- type Logger
- type NotExistError
- type OtherError
- type ReadOnlyLock
- type UnavailableError
Constants ¶
const DefaultHeartbeatFrequency = 5 * time.Second
DefaultHeartbeatFrequency is the recommended frequency that client should refresh the lock so to avoid other clients from stealing it. Use WithHeartbeatFrequency to modify this value.
const DefaultLeaseDuration = 20 * time.Second
DefaultLeaseDuration is the recommended period of time that a lock can be considered valid before being stolen by another client. Use WithLeaseDuration to modify this value.
const DefaultTableName = "locks"
DefaultTableName defines the table which the client is going to use to store the content and the metadata of the locks. Use WithCustomTable to modify this value.
Variables ¶
var (
ErrDurationTooSmall = errors.New("Heartbeat period must be no more than half the length of the Lease Duration, " +
"or locks might expire due to the heartbeat thread taking too long to update them (recommendation is to make it much greater, for example " +
"4+ times greater)")
)
Validation errors.
var ErrLockAlreadyReleased = errors.New("lock is already released")
ErrLockAlreadyReleased indicates that a release call cannot be fulfilled because the client does not hold the lock.
var ErrLockNotFound = &NotExistError{errors.New("lock not found")}
ErrLockNotFound is returned for get calls on missing lock entries.
var ErrNotAcquired = errors.New("cannot acquire lock")
ErrNotAcquired indicates the given lock is already enforce to some other client.
var ErrNotPostgreSQLDriver = errors.New("this is not a PostgreSQL connection")
ErrNotPostgreSQLDriver is returned when an invalid database connection is passed to this locker client.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is the PostgreSQL's backed distributed lock. Make sure it is always configured to talk to leaders and not followers in the case of replicated setups.
func New ¶
func New(db *sql.DB, opts ...ClientOption) (*Client, error)
New returns a locker client from the given database connection. This function validates that *sql.DB holds a ratified postgreSQL driver (lib/pq).
func UnsafeNew ¶ added in v1.5.0
func UnsafeNew(db *sql.DB, opts ...ClientOption) (*Client, error)
UnsafeNew returns a locker client from the given database connection. This function does not check if *sql.DB holds a ratified postgreSQL driver.
func (*Client) Acquire ¶
func (c *Client) Acquire(name string, opts ...LockOption) (*Lock, error)
Acquire attempts to grab the lock with the given key name and wait until it succeeds.
func (*Client) AcquireContext ¶
func (c *Client) AcquireContext(ctx context.Context, name string, opts ...LockOption) (*Lock, error)
AcquireContext attempts to grab the lock with the given key name, wait until it succeeds or the context is done. It returns ErrNotAcquired if the context is canceled before the lock is acquired.
func (*Client) CreateTable ¶
CreateTable prepares a PostgreSQL table with the right DDL for it to be used by this lock client. If the table already exists, it will return an error.
func (*Client) Do ¶
func (c *Client) Do(ctx context.Context, name string, f func(context.Context, *Lock) error, opts ...LockOption) error
Do executes f while holding the lock for the named lock. When the lock loss is detected in the heartbeat, it is going to cancel the context passed on to f. If it ends normally (err == nil), it releases the lock.
func (*Client) DropTable ¶ added in v1.9.0
DropTable cleans up a PostgreSQL DB from what was created in the CreateTable function.
func (*Client) Get ¶ added in v1.1.0
Get returns the lock object from the given name in the table without holding it first.
func (*Client) GetAllLocks ¶ added in v1.12.0
func (c *Client) GetAllLocks() ([]*ReadOnlyLock, error)
GetAllLocks returns all known locks in a read-only fashion.
func (*Client) GetAllLocksContext ¶ added in v1.12.0
func (c *Client) GetAllLocksContext(ctx context.Context) ([]*ReadOnlyLock, error)
GetAllLocksContext returns all known locks in a read-only fashion.
func (*Client) GetContext ¶ added in v1.1.0
GetContext returns the lock object from the given name in the table without holding it first.
func (*Client) GetData ¶
GetData returns the data field from the given lock in the table without holding the lock first.
func (*Client) GetDataContext ¶
GetDataContext returns the data field from the given lock in the table without holding the lock first.
func (*Client) Release ¶
Release will update the mutex entry to be able to be taken by other clients.
func (*Client) ReleaseContext ¶
ReleaseContext will update the mutex entry to be able to be taken by other clients. If a heartbeat is running, it will stopped it.
func (*Client) SendHeartbeat ¶
SendHeartbeat refreshes the mutex entry so to avoid other clients from grabbing it.
func (*Client) TryCreateTable ¶ added in v1.12.0
TryCreateTable prepares a PostgreSQL table with the right DDL for it to be used by this lock client. If the table already exists, it will be a no-op.
type ClientOption ¶
type ClientOption func(*Client)
ClientOption reconfigures the lock client.
func WithCustomTable ¶
func WithCustomTable(tableName string) ClientOption
WithCustomTable reconfigures the lock client to use an alternate lock table name.
func WithHeartbeatFrequency ¶
func WithHeartbeatFrequency(d time.Duration) ClientOption
WithHeartbeatFrequency defines the frequency of the heartbeats. Heartbeats should have no more than half of the duration of the lease.
func WithLeaseDuration ¶
func WithLeaseDuration(d time.Duration) ClientOption
WithLeaseDuration defines how long should the lease be held.
func WithLevelLogger ¶ added in v1.14.0
func WithLevelLogger(l LevelLogger) ClientOption
WithLevelLogger injects a logger into the client, so its internals can be recorded.
func WithLogger ¶
func WithLogger(l Logger) ClientOption
WithLogger injects a logger into the client, so its internals can be recorded. Deprecated: Use WithLevelLogger instead.
func WithOwner ¶ added in v1.1.0
func WithOwner(owner string) ClientOption
WithOwner reconfigures the lock client to use a custom owner name.
type FailedPreconditionError ¶ added in v1.2.0
type FailedPreconditionError struct {
// contains filtered or unexported fields
}
FailedPreconditionError is an error wrapper that gives the FailedPrecondition kind to an error.
func (*FailedPreconditionError) Error ¶ added in v1.2.0
func (err *FailedPreconditionError) Error() string
func (*FailedPreconditionError) Unwrap ¶ added in v1.2.0
func (err *FailedPreconditionError) Unwrap() error
Unwrap returns the next error in the error chain.
type LevelLogger ¶ added in v1.14.0
LevelLogger is used for internal inspection of the lock client.
type Lock ¶
type Lock struct {
// contains filtered or unexported fields
}
Lock is the mutex entry in the database.
func (*Lock) IsReleased ¶
IsReleased indicates whether the lock is either released or lost after heartbeat.
func (*Lock) RecordVersionNumber ¶ added in v1.3.0
RecordVersionNumber is the expectation that this lock entry has about its consistency in the database. If the RecordVersionNumber from the database mismatches the one in the lock, it means that some clock drift has taken place and this lock is no longer valid.
type LockOption ¶
type LockOption func(*Lock)
LockOption reconfigures how the lock behaves on acquire and release.
func FailIfLocked ¶
func FailIfLocked() LockOption
FailIfLocked will not retry to acquire the lock, instead returning.
func KeepOnRelease ¶
func KeepOnRelease() LockOption
KeepOnRelease preserves the lock entry when Close() is called on the lock.
func ReplaceData ¶
func ReplaceData() LockOption
ReplaceData will force the new content to be stored in the lock entry.
func WithCustomHeartbeatContext ¶ added in v1.7.0
func WithCustomHeartbeatContext(ctx context.Context) LockOption
WithCustomHeartbeatContext will override the context used for the heartbeats. It means the cancelation now is responsibility of the caller of the lock.
type Logger ¶
type Logger interface {
Println(v ...interface{})
}
Logger is used for internal inspection of the lock client.
type NotExistError ¶ added in v1.2.0
type NotExistError struct {
// contains filtered or unexported fields
}
NotExistError is an error wrapper that gives the NotExist kind to an error.
func (*NotExistError) Error ¶ added in v1.2.0
func (err *NotExistError) Error() string
func (*NotExistError) Unwrap ¶ added in v1.2.0
func (err *NotExistError) Unwrap() error
Unwrap returns the next error in the error chain.
type OtherError ¶ added in v1.2.0
type OtherError struct {
// contains filtered or unexported fields
}
OtherError is an error wrapper that gives the Other kind to an error.
func (*OtherError) Error ¶ added in v1.2.0
func (err *OtherError) Error() string
func (*OtherError) Unwrap ¶ added in v1.2.0
func (err *OtherError) Unwrap() error
Unwrap returns the next error in the error chain.
type ReadOnlyLock ¶ added in v1.12.0
type ReadOnlyLock Lock
ReadOnlyLock holds a copy of the information of a lock in the database.
func (*ReadOnlyLock) Data ¶ added in v1.12.0
func (l *ReadOnlyLock) Data() []byte
Data returns the content of the lock, if any is available.
func (*ReadOnlyLock) Name ¶ added in v1.12.0
func (l *ReadOnlyLock) Name() string
Name returns the lock's name.
func (*ReadOnlyLock) Owner ¶ added in v1.12.0
func (l *ReadOnlyLock) Owner() string
Owner returns who currently owns the lock.
type UnavailableError ¶ added in v1.2.0
type UnavailableError struct {
// contains filtered or unexported fields
}
UnavailableError is an error wrapper that gives the Unavailable kind to an error.
func (*UnavailableError) Error ¶ added in v1.2.0
func (err *UnavailableError) Error() string
func (*UnavailableError) Unwrap ¶ added in v1.2.0
func (err *UnavailableError) Unwrap() error
Unwrap returns the next error in the error chain.
Source Files