Documentation ¶
Overview ¶
Package wal implements a WAL. WAL is short for Write-Ahead-Log. Its purpose is durability. Before a write is acknowledged, it is copied to the WAL. If the machine crashes after a write to the WAL but before it is acknowledged, we can recover and apply the write by reading it from the WAL. If instead the machine crashes before the write to WAL, we can safely retry the write without fear of the write being applied twice (exactly once semantics).
A write may be pending in user-space buffers or the page cache. It must be synced to disk for it to be made durable. There are multiple ways to do this, though they make a trade-off between the level of durability and performance:
Sync after every write. This is the slowest option but offers the strongest durability.
Sync after N writes, where N is the batch size. This is faster since it calls sync less frequently, but you can lose at most N writes. However, if < N writes come in, there must be a mechanism to force-sync the batch at some point even though it hasn't reached N yet.
Sync at a regular T interval, where T is a time interval. This is faster since it calls sync less frequently, but you can lose un-synced writes. Another downside is that you must find an appropriate interval, dynamically/adaptively or via manual tuning.
Don't force sync. This is the fastest but most dangerous: the OS will asynchronously write to disk (i.e. choose when to flush the dirty page back to disk).
WAL is agnostic and lets the user decide which strategy is appropriate: call Write() and Sync() as one pleases.
Often we want to fetch all written records from an index onwards. To optimize this search pattern, WAL writes records to "segment" files that live in a "WAL" directory. Each segment has a maximum size, so when one is filled up, WAL will automatically close that file and begin a new one. Segment files are formatted as "{seq}-{index}.wal", where seq is the `seq`-th segment file, and the first record in that segment is the `index`-th overall record. Thus, we don't have to read every segment in the directory to find the appropriate records.
Index ¶
Constants ¶
const ( // SegmentSizeBytes is the preallocated size of each segment file. // The actual size might actually be larger than this. In general, the // default value should be used, but this is defined as an exported variable // so that tests can set a different segment size. SegmentSizeBytes = 64 * 1000 * 1000 // SegExt is the segment file extension. SegExt = ".seg" // ScratchSuffix is the suffix of the scratch directory for the write-ahead-log. ScratchSuffix = ".tmp" )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type WAL ¶
type WAL struct {
// contains filtered or unexported fields
}
WAL is a write-ahead-log.
func (*WAL) Sync ¶
Sync persists accumulated writes from both the user-land buffer and kernel page cache to disk.