shares

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 3, 2024 License: Apache-2.0 Imports: 11 Imported by: 9

README

Shares

See the celestia-app specs for shares.

Documentation

Overview

Package shares contains the Share data structure.

Index

Constants

View Source 
const (
	// ShareSize is the size of a share in bytes.
	ShareSize = 512

	// ShareInfoBytes is the number of bytes reserved for information. The info
	// byte contains the share version and a sequence start idicator.
	ShareInfoBytes = 1

	// SequenceLenBytes is the number of bytes reserved for the sequence length
	// that is present in the first share of a sequence.
	SequenceLenBytes = 4

	// ShareVersionZero is the first share version format.
	ShareVersionZero = uint8(0)

	// DefaultShareVersion is the defacto share version. Use this if you are
	// unsure of which version to use.
	DefaultShareVersion = ShareVersionZero

	// CompactShareReservedBytes is the number of bytes reserved for the location of
	// the first unit (transaction, ISR) in a compact share.
	// Deprecated: use ShareReservedBytes instead.
	CompactShareReservedBytes = ShareReservedBytes

	// ShareReservedBytes is the number of bytes reserved for the location of
	// the first unit (transaction, ISR) in a compact share.
	ShareReservedBytes = 4

	// FirstCompactShareContentSize is the number of bytes usable for data in
	// the first compact share of a sequence.
	FirstCompactShareContentSize = ShareSize - namespace.NamespaceSize - ShareInfoBytes - SequenceLenBytes - ShareReservedBytes

	// ContinuationCompactShareContentSize is the number of bytes usable for
	// data in a continuation compact share of a sequence.
	ContinuationCompactShareContentSize = ShareSize - namespace.NamespaceSize - ShareInfoBytes - ShareReservedBytes

	// FirstSparseShareContentSize is the number of bytes usable for data in the
	// first sparse share of a sequence.
	FirstSparseShareContentSize = ShareSize - namespace.NamespaceSize - ShareInfoBytes - SequenceLenBytes

	// ContinuationSparseShareContentSize is the number of bytes usable for data
	// in a continuation sparse share of a sequence.
	ContinuationSparseShareContentSize = ShareSize - namespace.NamespaceSize - ShareInfoBytes

	// MinSquareSize is the smallest original square width.
	MinSquareSize = 1

	// MinShareCount is the minimum number of shares allowed in the original
	// data square.
	MinShareCount = MinSquareSize * MinSquareSize

	// MaxShareVersion is the maximum value a share version can be.
	MaxShareVersion = 127
)

Variables

View Source 
var (
	ErrIncorrectNumberOfIndexes = errors.New(
		"number of indexes is not identical to the number of blobs",
	)
	ErrUnexpectedFirstBlobShareIndex = errors.New(
		"the first blob started at an unexpected index",
	)
)
View Source 
var SupportedShareVersions = []uint8{ShareVersionZero}

SupportedShareVersions is a list of supported share versions.

Functions

func AvailableBytesFromCompactShares 

func AvailableBytesFromCompactShares(n int) int

AvailableBytesFromCompactShares returns the maximum amount of bytes that could fit in `n` compact shares. Note that all compact shares are length prefixed. To account for this use `RawTxSize`.

func AvailableBytesFromSparseShares 

func AvailableBytesFromSparseShares(n int) int

AvailableBytesFromSparseShares returns the maximum amount of bytes that could fit in `n` sparse shares

func CheckSubArray 

func CheckSubArray(txList [][]byte, subTxList [][]byte) bool

CheckSubArray returns whether subTxList is a subarray of txList

func CompactSharesNeeded 

func CompactSharesNeeded(sequenceLen int) (sharesNeeded int)

CompactSharesNeeded returns the number of compact shares needed to store a sequence of length sequenceLen. The parameter sequenceLen is the number of bytes of transactions or intermediate state roots in a sequence.

func DelimLen 

func DelimLen(size uint64) int

DelimLen calculates the length of the delimiter for a given unit size

func ExtractShareIndexes 

func ExtractShareIndexes(txs [][]byte) []uint32

ExtractShareIndexes iterates over the transactions and extracts the share indexes from wrapped transactions. It returns nil if the transactions are from an old block that did not have share indexes in the wrapped txs.

func GenerateRandomTxs 

func GenerateRandomTxs(count, size int) [][]byte

func GenerateRandomlySizedTxs 

func GenerateRandomlySizedTxs(count, maxSize int) [][]byte

func GetRandomSubSlice 

func GetRandomSubSlice(size int) (start int, length int)

GetRandomSubSlice returns two integers representing a randomly sized range in the interval [0, size]

func IsPowerOfTwo 

func IsPowerOfTwo[I constraints.Integer](input I) bool

IsPowerOfTwo returns true if input is a power of two.

func MarshalDelimitedTx 

func MarshalDelimitedTx(tx []byte) ([]byte, error)

MarshalDelimitedTx prefixes a transaction with the length of the transaction encoded as a varint.

func NewReservedBytes 

func NewReservedBytes(byteIndex uint32) ([]byte, error)

NewReservedBytes returns a byte slice of length ShareReservedBytes that contains the byteIndex of the first unit that starts in a compact share.

func ParseBlobs 

func ParseBlobs(shares []Share) ([]*blob.Blob, error)

ParseBlobs collects all blobs from the shares provided

func ParseDelimiter 

func ParseDelimiter(input []byte) (inputWithoutLenDelimiter []byte, unitLen uint64, err error)

ParseDelimiter attempts to parse a varint length delimiter from the input provided. It returns the input without the len delimiter bytes, the length parsed from the varint optionally an error. Unit length delimiters are used in compact shares where units (i.e. a transaction) are prefixed with a length delimiter that is encoded as a varint. Input should not contain the namespace ID or info byte of a share.

func ParseReservedBytes 

func ParseReservedBytes(reservedBytes []byte) (uint32, error)

ParseReservedBytes parses a byte slice of length ShareReservedBytes into a byteIndex.

func ParseTxs 

func ParseTxs(shares []Share) ([][]byte, error)

ParseTxs collects all of the transactions from the shares provided

func RawTxSize 

func RawTxSize(desiredSize int) int

RawTxSize returns the raw tx size that can be used to construct a tx of desiredSize bytes. This function is useful in tests to account for the length delimiter that is prefixed to a tx when it is converted into a compact share

func RoundDownPowerOfTwo 

func RoundDownPowerOfTwo[I constraints.Integer](input I) (I, error)

RoundDownPowerOfTwo returns the next power of two less than or equal to input.

func RoundUpPowerOfTwo 

func RoundUpPowerOfTwo[I constraints.Integer](input I) I

RoundUpPowerOfTwo returns the next power of two greater than or equal to input.

func RoundUpPowerOfTwoStrict 

func RoundUpPowerOfTwoStrict[I constraints.Integer](input I) I

RoundUpPowerOfTwoStrict returns the next power of two that is strictly greater than input.

func SparseSharesNeeded 

func SparseSharesNeeded(sequenceLen uint32) (sharesNeeded int)

SparseSharesNeeded returns the number of shares needed to store a sequence of length sequenceLen.

func SplitTxs 

func SplitTxs(txs [][]byte) (txShares []Share, pfbShares []Share, shareRanges map[[sha256.Size]byte]Range, err error)

func ToBytes 

func ToBytes(shares []Share) (bytes [][]byte)

Types

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

func NewBuilder 

func NewBuilder(ns namespace.Namespace, shareVersion uint8, isFirstShare bool) (*Builder, error)

NewBuilder returns a new share builder.

func NewEmptyBuilder 

func NewEmptyBuilder() *Builder

func (*Builder) AddData 

func (b *Builder) AddData(rawData []byte) (rawDataLeftOver []byte)

func (*Builder) AvailableBytes 

func (b *Builder) AvailableBytes() int

func (*Builder) Build 

func (b *Builder) Build() (*Share, error)

func (*Builder) FlipSequenceStart 

func (b *Builder) FlipSequenceStart()

FlipSequenceStart flips the sequence start indicator of the share provided

func (*Builder) ImportRawShare 

func (b *Builder) ImportRawShare(rawBytes []byte) *Builder

func (*Builder) IsEmptyShare 

func (b *Builder) IsEmptyShare() bool

IsEmptyShare returns true if no data has been written to the share

func (*Builder) MaybeWriteReservedBytes 

func (b *Builder) MaybeWriteReservedBytes() error

MaybeWriteReservedBytes will be a no-op if the reserved bytes have already been populated. If the reserved bytes are empty, it will write the location of the next unit of data to the reserved bytes.

func (*Builder) WriteSequenceLen 

func (b *Builder) WriteSequenceLen(sequenceLen uint32) error

WriteSequenceLen writes the sequence length to the first share.

func (*Builder) ZeroPadIfNecessary 

func (b *Builder) ZeroPadIfNecessary() (bytesOfPadding int)

type CompactShareCounter 

type CompactShareCounter struct {
	// contains filtered or unexported fields
}

func NewCompactShareCounter 

func NewCompactShareCounter() *CompactShareCounter

NewCompactShareCounter creates a new instance of a counter which calculates the amount of compact shares a set of data will be split into.

func (*CompactShareCounter) Add 

func (c *CompactShareCounter) Add(dataLen int) int

Add adds the length of the data to the counter and returns the amount of shares the counter has been increased by.

func (*CompactShareCounter) Remainder 

func (c *CompactShareCounter) Remainder() int

func (*CompactShareCounter) Revert 

func (c *CompactShareCounter) Revert()

Revert reverts the last Add operation. This can be called multiple times but only works the first time after an add operation.

func (*CompactShareCounter) Size 

func (c *CompactShareCounter) Size() int

Size returns the amount of shares the compact share counter has counted.

type CompactShareSplitter 

type CompactShareSplitter struct {
	// contains filtered or unexported fields
}

CompactShareSplitter will write raw data compactly across a progressively increasing set of shares. It is used to lazily split block data such as transactions or intermediate state roots into shares.

func NewCompactShareSplitter 

func NewCompactShareSplitter(ns namespace.Namespace, shareVersion uint8) *CompactShareSplitter

NewCompactShareSplitter returns a CompactShareSplitter using the provided namespace and shareVersion.

func (*CompactShareSplitter) Count 

func (css *CompactShareSplitter) Count() int

Count returns the number of shares that would be made if `Export` was invoked on this compact share splitter.

func (*CompactShareSplitter) Export 

func (css *CompactShareSplitter) Export() ([]Share, error)

Export returns the underlying compact shares

func (*CompactShareSplitter) ShareRanges 

func (css *CompactShareSplitter) ShareRanges(shareRangeOffset int) map[[sha256.Size]byte]Range

ShareRanges returns a map of share ranges to the corresponding tx keys. All share ranges in the map of shareRanges will be offset (i.e. incremented) by the shareRangeOffset provided. shareRangeOffset should be 0 for the first compact share sequence in the data square (transactions) but should be some non-zero number for subsequent compact share sequences (e.g. pfb txs).

func (*CompactShareSplitter) WriteTx 

func (css *CompactShareSplitter) WriteTx(tx []byte) error

WriteTx adds the delimited data for the provided tx to the underlying compact share splitter.

type InfoByte 

type InfoByte byte

InfoByte is a byte with the following structure: the first 7 bits are reserved for version information in big endian form (initially `0000000`). The last bit is a "sequence start indicator", that is `1` if this is the first share of a sequence and `0` if this is a continuation share.

func NewInfoByte 

func NewInfoByte(version uint8, isSequenceStart bool) (InfoByte, error)

func ParseInfoByte 

func ParseInfoByte(i byte) (InfoByte, error)

func (InfoByte) IsSequenceStart 

func (i InfoByte) IsSequenceStart() bool

IsSequenceStart returns whether this share is the start of a sequence.

func (InfoByte) Version 

func (i InfoByte) Version() uint8

Version returns the version encoded in this InfoByte. Version is expected to be between 0 and MaxShareVersion (inclusive).

type Range 

type Range struct {
	// Start is the index of the first share occupied by this range.
	Start int
	// End is the next index after the last share occupied by this range.
	End int
}

Range is an end exclusive set of share indexes.

func EmptyRange 

func EmptyRange() Range

func GetShareRangeForNamespace 

func GetShareRangeForNamespace(shares []Share, ns namespace.Namespace) (Range, error)

GetShareRangeForNamespace returns all shares that belong to a given namespace. It will return an empty range if the namespace could not be found. This assumes that the slice of shares are lexicographically sorted by namespace. Ranges here are always end exclusive.

func NewRange 

func NewRange(start, end int) Range

func (*Range) Add 

func (r *Range) Add(value int)

func (Range) IsEmpty 

func (r Range) IsEmpty() bool

type Share 

type Share struct {
	// contains filtered or unexported fields
}

Share contains the raw share data (including namespace ID).

func FromBytes 

func FromBytes(bytes [][]byte) (shares []Share, err error)

func NamespacePaddingShare 

func NamespacePaddingShare(ns namespace.Namespace, shareVersion uint8) (Share, error)

NamespacePaddingShare returns a share that acts as padding. Namespace padding shares follow a blob so that the next blob may start at an index that conforms to blob share commitment rules. The ns and shareVersion parameters provided should be the namespace and shareVersion of the blob that precedes this padding in the data square.

func NamespacePaddingShares 

func NamespacePaddingShares(ns namespace.Namespace, shareVersion uint8, n int) ([]Share, error)

NamespacePaddingShares returns n namespace padding shares.

func NewShare 

func NewShare(data []byte) (*Share, error)

func ReservedPaddingShare 

func ReservedPaddingShare() Share

ReservedPaddingShare returns a share that acts as padding. Reserved padding shares follow all significant shares in the reserved namespace so that the first blob can start at an index that conforms to non-interactive default rules.

func ReservedPaddingShares 

func ReservedPaddingShares(n int) []Share

ReservedPaddingShares returns n reserved padding shares.

func SplitBlobs 

func SplitBlobs(blobs ...*blob.Blob) ([]Share, error)

SplitBlobs splits the provided blobs into shares.

func TailPaddingShare 

func TailPaddingShare() Share

TailPaddingShare is a share that is used to pad a data square to the desired square size. Tail padding shares follow the last blob share in the data square.

func TailPaddingShares 

func TailPaddingShares(n int) []Share

TailPaddingShares returns n tail padding shares.

func (*Share) DoesSupportVersions 

func (s *Share) DoesSupportVersions(supportedShareVersions []uint8) error

func (*Share) InfoByte 

func (s *Share) InfoByte() (InfoByte, error)

func (Share) IsCompactShare 

func (s Share) IsCompactShare() (bool, error)

IsCompactShare returns true if this is a compact share.

func (*Share) IsPadding 

func (s *Share) IsPadding() (bool, error)

IsPadding returns whether this *share is padding or not.

func (*Share) IsSequenceStart 

func (s *Share) IsSequenceStart() (bool, error)

IsSequenceStart returns true if this is the first share in a sequence.

func (*Share) Len 

func (s *Share) Len() int

func (*Share) Namespace 

func (s *Share) Namespace() (namespace.Namespace, error)

func (*Share) RawData 

func (s *Share) RawData() (rawData []byte, err error)

RawData returns the raw share data. The raw share data does not contain the namespace ID, info byte, sequence length, or reserved bytes.

func (*Share) RawDataUsingReserved 

func (s *Share) RawDataUsingReserved() (rawData []byte, err error)

RawDataUsingReserved returns the raw share data while taking reserved bytes into account.

func (*Share) SequenceLen 

func (s *Share) SequenceLen() (sequenceLen uint32, err error)

SequenceLen returns the sequence length of this *share and optionally an error. It returns 0, nil if this is a continuation share (i.e. doesn't contain a sequence length).

func (*Share) ToBytes 

func (s *Share) ToBytes() []byte

func (*Share) Validate 

func (s *Share) Validate() error

func (*Share) Version 

func (s *Share) Version() (uint8, error)

type ShareSequence 

type ShareSequence struct {
	Namespace namespace.Namespace
	Shares    []Share
}

ShareSequence represents a contiguous sequence of shares that are part of the same namespace and blob. For compact shares, one share sequence exists per reserved namespace. For sparse shares, one share sequence exists per blob.

func ParseShares 

func ParseShares(shares []Share, ignorePadding bool) ([]ShareSequence, error)

ParseShares parses the shares provided and returns a list of ShareSequences. If ignorePadding is true then the returned ShareSequences will not contain any padding sequences.

func (ShareSequence) RawData 

func (s ShareSequence) RawData() (data []byte, err error)

RawData returns the raw share data of this share sequence. The raw data does not contain the namespace ID, info byte, sequence length, or reserved bytes.

func (ShareSequence) SequenceLen 

func (s ShareSequence) SequenceLen() (uint32, error)

type SparseShareSplitter 

type SparseShareSplitter struct {
	// contains filtered or unexported fields
}

SparseShareSplitter lazily splits blobs into shares that will eventually be included in a data square. It also has methods to help progressively count how many shares the blobs written take up.

func NewSparseShareSplitter 

func NewSparseShareSplitter() *SparseShareSplitter

func (*SparseShareSplitter) Count 

func (sss *SparseShareSplitter) Count() int

Count returns the current number of shares that will be made if exporting.

func (*SparseShareSplitter) Export 

func (sss *SparseShareSplitter) Export() []Share

Export finalizes and returns the underlying shares.

func (*SparseShareSplitter) Write 

func (sss *SparseShareSplitter) Write(blob *blob.Blob) error

Write writes the provided blob to this sparse share splitter. It returns an error or nil if no error is encountered.

func (*SparseShareSplitter) WriteNamespacePaddingShares 

func (sss *SparseShareSplitter) WriteNamespacePaddingShares(count int) error

WriteNamespacePaddingShares adds padding shares with the namespace of the last written share. This is useful to follow the non-interactive default rules. This function assumes that at least one share has already been written.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.