block

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 21, 2022 License: BSD-3-Clause Imports: 8 Imported by: 2

README

State Sync: Engine role and workflow

State sync promises to dramatically cut down the time it takes for a validator to join a chain by directly downloading and verifying the state of the chain, rather than rebuilding the state by processing all historical blocks.

Coinflect's approach to state sync leaves most of specifics to each VM, to allow maximal flexibility. However, the Coinflect engine plays a well defined role in selecting and validating state summaries, which are the state sync seeds used by a VM to kickstart its syncing process.

In this document, we outline the engine's role in state sync and the requirements for VMs implementing state sync.

StateSyncableVM Interface

The StateSyncableVM interface enumerates all the features a VM must implement to support state sync.

The Coinflect engine begins bootstrapping a VM through state-sync if StateSyncEnabled() returns true. Otherwise, the engine falls back to processing all historical blocks.

The Coinflect engine performs two state-syncing phases:

  • Frontier retrieval: retrieve the most recent state summaries from a random subset of validators
  • Frontier validation: the retrieved state summaries are voted on by all connected validators

Security is guaranteed by accepting the state summary if and only if a sufficient fraction of stake has validated them. This prevents malicious actors from poisoning a VM with a corrupt or unavailable state summary.

Frontier retrieval

The Coinflect engine waits to begin frontier retrieval until enough stake is connected.

The engine first calls GetOngoingSyncStateSummary() to retrieve a local state summary from the VM. If available, this state summary is added to the frontier.

A state summary may be locally available if the VM was previously shut down while state syncing. By revalidating this local summary, Coinflect engine helps the VM understand whether its local state summary is still widely supported by the network. If the local state summary is widely supported, the engine will allow the VM to resume state syncing from it. Otherwise the VM will be requested to drop it in favour of a fresher, more available state summary.

State summary frontier is collected as follows:

  1. The Coinflect engine samples a random subset of validators and sends each of them a GetStateSummaryFrontier message to retrieve the latest state summaries.
  2. Each target validator pulls its latest state summary from the VM via the GetLastStateSummary method, and responds with a StateSummaryFrontier message.
  3. StateSummaryFrontier responses are parsed via the ParseStateSummary method and added to the state summary frontier to be validated.

The Coinflect engine does not pose major constraints on the state summary structure; as its parsing is left to the VM to implement. However, the Coinflect engine does require each state summary to be uniquely described by two identifiers, Height and ID. The reason for this double identification will be clearer as we describe the state summary validation phase.

Height is a uint64 and represents the block height that the state summaries refers to. Height offers the most succinct way to address an accepted state summary.

ID is an ids.ID type and is the verifiable way to address a state summary, regardless of its status. In most implementations, a state summary ID is the hash of the state summary's bytes.

Frontier validation

Once the frontiers have been retrieved, a network wide voting round is initiated to validate them. Coinflect sends a GetAcceptedStateSummary message to each connected validator, containing a list of the state summary frontier's Heights. Heights provide a unique yet succinct way to identify state summaries, and they help reduce the size of the GetAcceptedStateSummary message.

When a validator receives a GetAcceptedStateSummary message, it will respond with IDs corresponding to the Heights sent in the message. This is done by calling GetStateSummary(summaryHeight uint64) on the VM for each Height. Unknown or unsupported state summary Heights are skipped. Responding with the summary IDs allows the client to verify votes easily and ensures the integrity of the state sync starting point.

AcceptedStateSummary messages returned to the state syncing node are validated by comparing responded state summaries IDs with the IDs calculated from state summary frontier previously retrieved. Valid responses are stored along with cumulative validator stake.

Once all validators respond or timeout, Coinflect will select all the state summaries with a sufficient stake verifying them.

If no state summary is backed by sufficient stake, the process of collecting a state frontier and validating it is restarted again, up to a configurable number of times.

Of all the valid state summaries, one is selected and passed to the VM by calling Summary.Accept(). The preferred state summary is selected as follows: if the locally available one is still valid and supported by the network it will be accepted to allow the VM to resume the previously interrupted state sync. Otherwise, the most recent state summary (with the highest Height value) is picked. Note that Coinflect engine will block upon Summary.Accept()'s response, hence the VM should perform actual state syncing asynchronously.

The Coinflect engine declares state syncing complete in the following cases:

  1. Summary.Accept() returns (false, nil) signalling that the VM considered the summary valid but skips the whole syncing process. This may happen if the VM estimates that bootstrapping would be faster than state syncing with the provided summary.
  2. The VM sends StateSyncDone via the Notify channel.

Note that any error returned from Summary.Accept() is considered fatal and causes the engine to shutdown.

After state sync is complete, Coinflect will continue bootstrapping the remaining blocks until the node has reached the block frontier.

Documentation

Overview

Copyright (C) 2019-2021, Ava Labs, Inc. All rights reserved. Copyright (C) 2022, Coinflect. All rights reserved. See the file LICENSE for licensing terms.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrHeightIndexedVMNotImplemented = errors.New("vm does not implement HeightIndexedChainVM interface")
	ErrIndexIncomplete               = errors.New("query failed because height index is incomplete")
)
View Source 
var ErrRemoteVMNotImplemented = errors.New("vm does not implement RemoteVM interface")
View Source 
var ErrStateSyncableVMNotImplemented = errors.New("vm does not implement StateSyncableVM interface")

Functions

func BatchedParseBlock 

func BatchedParseBlock(vm Parser, blks [][]byte) ([]snowman.Block, error)

func GetAncestors 

func GetAncestors(
	vm Getter,
	blkID ids.ID,
	maxBlocksNum int,
	maxBlocksSize int,
	maxBlocksRetrivalTime time.Duration,
) ([][]byte, error)

Types

type BatchedChainVM 

type BatchedChainVM interface {
	GetAncestors(
		blkID ids.ID,
		maxBlocksNum int,
		maxBlocksSize int,
		maxBlocksRetrivalTime time.Duration,
	) ([][]byte, error)

	BatchedParseBlock(blks [][]byte) ([]snowman.Block, error)
}

BatchedChainVM extends the minimal functionalities exposed by ChainVM for VMs communicating over network (gRPC in our case). This allows more efficient operations since calls over network can be duly batched

type ChainVM 

type ChainVM interface {
	common.VM

	Getter
	Parser

	// Attempt to create a new block from data contained in the VM.
	//
	// If the VM doesn't want to issue a new block, an error should be
	// returned.
	BuildBlock() (snowman.Block, error)

	// Notify the VM of the currently preferred block.
	//
	// This should always be a block that has no children known to consensus.
	SetPreference(ids.ID) error

	// LastAccepted returns the ID of the last accepted block.
	//
	// If no blocks have been accepted by consensus yet, it is assumed there is
	// a definitionally accepted block, the Genesis block, that will be
	// returned.
	LastAccepted() (ids.ID, error)
}

ChainVM defines the required functionality of a Snowman VM.

A Snowman VM is responsible for defining the representation of state, the representation of operations on that state, the application of operations on that state, and the creation of the operations. Consensus will decide on if the operation is executed and the order operations are executed in.

For example, suppose we have a VM that tracks an increasing number that is agreed upon by the network. The state is a single number. The operation is setting the number to a new, larger value. Applying the operation will save to the database the new value. The VM can attempt to issue a new number, of larger value, at any time. Consensus will ensure the network agrees on the number at every block height.

type Getter 

type Getter interface {
	// Attempt to load a block.
	//
	// If the block does not exist, database.ErrNotFound should be returned.
	//
	// It is expected that blocks that have been successfully verified should be
	// returned correctly. It is also expected that blocks that have been
	// accepted by the consensus engine should be able to be fetched. It is not
	// required for blocks that have been rejected by the consensus engine to be
	// able to be fetched.
	GetBlock(ids.ID) (snowman.Block, error)
}

Getter defines the functionality for fetching a block by its ID.

type HeightIndexedChainVM 

type HeightIndexedChainVM interface {
	// VerifyHeightIndex should return:
	// - nil if the height index is available.
	// - ErrHeightIndexedVMNotImplemented if the height index is not supported.
	// - ErrIndexIncomplete if the height index is not currently available.
	// - Any other non-standard error that may have occurred when verifying the
	//   index.
	VerifyHeightIndex() error

	// GetBlockIDAtHeight returns the ID of the block that was accepted with
	// [height].
	GetBlockIDAtHeight(height uint64) (ids.ID, error)
}

HeightIndexedChainVM extends ChainVM to allow querying block IDs by height.

type Parser 

type Parser interface {
	// Attempt to create a block from a stream of bytes.
	//
	// The block should be represented by the full byte array, without extra
	// bytes.
	//
	// It is expected for all historical blocks to be parseable.
	ParseBlock([]byte) (snowman.Block, error)
}

Parser defines the functionality for fetching a block by its bytes.

type StateSummary 

type StateSummary interface {
	// ID uniquely identifies this state summary, regardless of the chain state.
	ID() ids.ID

	// Height uniquely identifies this an accepted state summary.
	Height() uint64

	// Bytes returns a byte slice than can be used to reconstruct this summary.
	Bytes() []byte

	// Accept triggers the VM to start state syncing this summary.
	//
	// The returned boolean will be [true] if the VM has started state sync or
	// [false] if the VM has skipped state sync.
	Accept() (bool, error)
}

StateSummary represents all the information needed to download, verify, and rebuild its state.

type StateSyncableVM 

type StateSyncableVM interface {
	// StateSyncEnabled indicates whether the state sync is enabled for this VM.
	// If StateSyncableVM is not implemented, as it may happen with a wrapper
	// VM, StateSyncEnabled should return false, nil
	StateSyncEnabled() (bool, error)

	// GetOngoingSyncStateSummary returns an in-progress state summary if it
	// exists.
	//
	// The engine can then ask the network if the ongoing summary is still
	// supported, thus helping the VM decide whether to continue an in-progress
	// sync or start over.
	//
	// Returns database.ErrNotFound if there is no in-progress sync.
	GetOngoingSyncStateSummary() (StateSummary, error)

	// GetLastStateSummary returns the latest state summary.
	//
	// Returns database.ErrNotFound if no summary is available.
	GetLastStateSummary() (StateSummary, error)

	// ParseStateSummary parses a state summary out of [summaryBytes].
	ParseStateSummary(summaryBytes []byte) (StateSummary, error)

	// GetStateSummary retrieves the state summary that was generated at height
	// [summaryHeight].
	//
	// Returns database.ErrNotFound if no summary is available at
	// [summaryHeight].
	GetStateSummary(summaryHeight uint64) (StateSummary, error)
}

StateSyncableVM contains the functionality to allow VMs to sync to a given state, rather then boostrapping from genesis.

type TestBatchedVM 

type TestBatchedVM struct {
	T *testing.T

	CantGetAncestors    bool
	CantBatchParseBlock bool

	GetAncestorsF func(
		blkID ids.ID,
		maxBlocksNum int,
		maxBlocksSize int,
		maxBlocksRetrivalTime time.Duration,
	) ([][]byte, error)

	BatchedParseBlockF func(blks [][]byte) ([]snowman.Block, error)
}

TestBatchedVM is a BatchedVM that is useful for testing.

func (*TestBatchedVM) BatchedParseBlock 

func (vm *TestBatchedVM) BatchedParseBlock(blks [][]byte) ([]snowman.Block, error)

func (*TestBatchedVM) Default 

func (vm *TestBatchedVM) Default(cant bool)

func (*TestBatchedVM) GetAncestors 

func (vm *TestBatchedVM) GetAncestors(
	blkID ids.ID,
	maxBlocksNum int,
	maxBlocksSize int,
	maxBlocksRetrivalTime time.Duration,
) ([][]byte, error)

type TestHeightIndexedVM 

type TestHeightIndexedVM struct {
	T *testing.T

	CantVerifyHeightIndex  bool
	CantGetBlockIDAtHeight bool

	VerifyHeightIndexF  func() error
	GetBlockIDAtHeightF func(height uint64) (ids.ID, error)
}

TestBatchedVM is a BatchedVM that is useful for testing.

func (*TestHeightIndexedVM) GetBlockIDAtHeight 

func (vm *TestHeightIndexedVM) GetBlockIDAtHeight(height uint64) (ids.ID, error)

func (*TestHeightIndexedVM) VerifyHeightIndex 

func (vm *TestHeightIndexedVM) VerifyHeightIndex() error

type TestStateSummary 

type TestStateSummary struct {
	IDV     ids.ID
	HeightV uint64
	BytesV  []byte

	T          *testing.T
	CantAccept bool
	AcceptF    func() (bool, error)
}

func (*TestStateSummary) Accept 

func (s *TestStateSummary) Accept() (bool, error)

func (*TestStateSummary) Bytes 

func (s *TestStateSummary) Bytes() []byte

func (*TestStateSummary) Height 

func (s *TestStateSummary) Height() uint64

func (*TestStateSummary) ID 

func (s *TestStateSummary) ID() ids.ID

type TestStateSyncableVM 

type TestStateSyncableVM struct {
	T *testing.T

	CantStateSyncEnabled,
	CantStateSyncGetOngoingSummary,
	CantGetLastStateSummary,
	CantParseStateSummary,
	CantGetStateSummary bool

	StateSyncEnabledF           func() (bool, error)
	GetOngoingSyncStateSummaryF func() (StateSummary, error)
	GetLastStateSummaryF        func() (StateSummary, error)
	ParseStateSummaryF          func(summaryBytes []byte) (StateSummary, error)
	GetStateSummaryF            func(uint64) (StateSummary, error)
}

func (*TestStateSyncableVM) GetLastStateSummary 

func (vm *TestStateSyncableVM) GetLastStateSummary() (StateSummary, error)

func (*TestStateSyncableVM) GetOngoingSyncStateSummary 

func (vm *TestStateSyncableVM) GetOngoingSyncStateSummary() (StateSummary, error)

func (*TestStateSyncableVM) GetStateSummary 

func (vm *TestStateSyncableVM) GetStateSummary(key uint64) (StateSummary, error)

func (*TestStateSyncableVM) ParseStateSummary 

func (vm *TestStateSyncableVM) ParseStateSummary(summaryBytes []byte) (StateSummary, error)

func (*TestStateSyncableVM) StateSyncEnabled 

func (vm *TestStateSyncableVM) StateSyncEnabled() (bool, error)

type TestVM 

type TestVM struct {
	common.TestVM

	CantBuildBlock,
	CantParseBlock,
	CantGetBlock,
	CantSetPreference,
	CantLastAccepted bool

	BuildBlockF    func() (snowman.Block, error)
	ParseBlockF    func([]byte) (snowman.Block, error)
	GetBlockF      func(ids.ID) (snowman.Block, error)
	SetPreferenceF func(ids.ID) error
	LastAcceptedF  func() (ids.ID, error)
}

TestVM is a ChainVM that is useful for testing.

func (*TestVM) BuildBlock 

func (vm *TestVM) BuildBlock() (snowman.Block, error)

func (*TestVM) Default 

func (vm *TestVM) Default(cant bool)

func (*TestVM) GetBlock 

func (vm *TestVM) GetBlock(id ids.ID) (snowman.Block, error)

func (*TestVM) LastAccepted 

func (vm *TestVM) LastAccepted() (ids.ID, error)

func (*TestVM) ParseBlock 

func (vm *TestVM) ParseBlock(b []byte) (snowman.Block, error)

func (*TestVM) SetPreference 

func (vm *TestVM) SetPreference(id ids.ID) error

Source Files

View all Source files

Directories

Path Synopsis
Package mocks is a generated GoMock package.
 Package mocks is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.