encoding

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 14, 2024 License: BSD-3-Clause Imports: 8 Imported by: 4

README

logplex/encoding

What's this?

A set of libraries we use to parse messages, and to also publish these same syslog RFC5424 messages.

How to use?

We have 2 scanners available. If you're trying to build a logplex compatible ingress, you can use the regular scanner.

Scanner
func handler(w http.ResponseWriter, r *http.Request) {
	s := NewScanner(r.Body)

	for s.Scan() {
		log.Printf("%+v", scanner.Message())
	}

	if s.Err() != nil {
		log.Printf("err: %v", s.Err())
	}
}
DrainScanner

If the intent is to write an application which acts as a heroku drain, then using the DrainScanner is preferrable -- primarily because it doesn't require structured data.

func handler(w http.ResponseWriter, r *http.Request) {
	s := NewDrainScanner(r.Body)

	for s.Scan() {
		log.Printf("%+v", scanner.Message())
	}

	if s.Err() != nil {
		log.Printf("err: %v", s.Err())
	}
}

Documentation

Index

Constants

View Source
const (
	// MaxFrameLength is the maximum message size to parse
	MaxFrameLength = 10240

	// OptimalFrameLength is the initial buffer size for scanning
	OptimalFrameLength = 1024
)
View Source
const FlexibleSyslogTimeFormat = "2006-01-02T15:04:05.999999Z07:00"

FlexibleSyslogTimeFormat accepts both 'Z' and TZ notation for event time.

View Source
const HumanTimeFormat = "2006-01-02T15:04:05.000000-07:00"

HumanTimeFormat defines the human friendly time format used in CLI/UI.

View Source
const L15Error = "L15: Error displaying log lines. Please try again."

L15Error is the message returned with an L15 error

View Source
const SyslogTimeFormat = "2006-01-02T15:04:05.999999-07:00"

SyslogTimeFormat defines the exact time format used in our logs.

Variables

View Source
var (
	// ErrBadFrame is returned when the scanner cannot parse syslog message boundaries
	ErrBadFrame = errors.New("bad frame")

	// ErrInvalidStructuredData is returned when structure data has any value other than '-' (blank)
	ErrInvalidStructuredData = errors.New("invalid structured data")

	// ErrInvalidPriVal is returned when pri-val is not properly formatted
	ErrInvalidPriVal = errors.New("invalid pri-val")
)
View Source
var ErrInvalidMessage = errors.New("invalid message")

ErrInvalidMessage returned when trying to encode an invalid syslog message

Functions

func Encode

func Encode(msg Message) ([]byte, error)

Encode serializes a syslog message into their wire format ( octet-framed syslog ) Disabling RFC 5424 compliance is the default and needed due to https://github.com/heroku/logplex/issues/204

func SyslogSplitFunc added in v0.0.31

func SyslogSplitFunc(data []byte, atEOF bool) (advance int, token []byte, err error)

SyslogSplitFunc splits the data based on the defined length prefix. format: 64 <190>1 2019-07-20T17:50:10.879238Z shuttle token shuttle - - 99\n65 <190>1 2019-07-20T17:50:10.879238Z shuttle token shuttle - - 100\n ^ frame size ^ boundary

func TruncatingSyslogSplitFunc added in v0.0.31

func TruncatingSyslogSplitFunc(maxLength int) bufio.SplitFunc

TruncatingSyslogSplitFunc enforces a maximum line length after parsing.

Types

type Encoder

type Encoder interface {
	Encode(msg Message) error
	KeepAlive() error
}

Encoder abstracts away how messages are written out

func NewPlain

func NewPlain(w io.Writer) Encoder

NewPlain creates a plain encoder. It dumps the log message directly without massaging it

func NewSSE

func NewSSE(w io.Writer) Encoder

NewSSE instantiates a new SSE encoder

type Message

type Message struct {
	Timestamp    time.Time
	Hostname     string
	Application  string
	Process      string
	ID           string
	Message      string
	Version      uint16
	Priority     uint8
	RFCCompliant bool
}

Message is a syslog message

func Decode

func Decode(raw []byte, hasStructuredData bool) (Message, error)

Decode converts a rfc5424 message to our model

func (Message) Size

func (m Message) Size() (int, error)

Size returns the message size in bytes, including the octet framing header

func (Message) WriteTo

func (m Message) WriteTo(w io.Writer) (int64, error)

WriteTo writes the message to a stream

type Scanner

type Scanner interface {
	Scan() bool
	Err() error
	Message() Message
}

Scanner is the general purpose primitive for parsing message bodies coming from log-shuttle, logfwd, logplex and all sorts of logging components.

func NewDrainScanner

func NewDrainScanner(r io.Reader, opts ...ScannerOption) Scanner

NewDrainScanner returns a scanner for use with drain endpoints. The primary difference is that it's loose and doesn't check for structured data.

func NewScanner

func NewScanner(r io.Reader, opts ...ScannerOption) Scanner

NewScanner is a syslog octet frame stream parser

type ScannerOption added in v0.0.31

type ScannerOption func(*syslogScanner)

func RFCCompliant added in v0.0.31

func RFCCompliant(compliant bool) ScannerOption

func WithBuffer added in v0.0.31

func WithBuffer(optimalFrameLength, maxFrameLength int) ScannerOption

func WithSplit added in v0.0.31

func WithSplit(splitFunc bufio.SplitFunc) ScannerOption

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL