logharbour

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 2, 2024 License: Apache-2.0 Imports: 25 Imported by: 28

Documentation

Overview

Package logharbour is a comprehensive logging system. It supports different log levels, log types, and can encode log entries in JSON. It also provides a fallback mechanism in case the primary log writer fails.

Example

Example of using With prefixed methods to set various fields of the logger. 

// Open a file for logging.
file, err := os.OpenFile("log.txt", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
	log.Fatal(err)
}
defer file.Close()

// Create a fallback writer that uses the file as the primary writer and stdout as the fallback.
fallbackWriter := NewFallbackWriter(file, os.Stdout)

// Create a logger context with the default priority.
lctx := NewLoggerContext(Info)

// Initialize the logger.
logger := NewLogger(lctx, "MyApp", fallbackWriter)

// Create a new logger with various fields set.
logger = logger.WithModule("Module1").
	WithWho("John Doe").
	WithStatus(Success).
	WithRemoteIP("192.168.1.1")

// Use the new logger to log an activity.
logger.LogActivity("User logged in", map[string]any{"username": "john"})

// Log a data change entry.
changeInfo := NewChangeInfo("User", "Update")
changeInfo = changeInfo.AddChange("email", "[email protected]", "[email protected]")
if err != nil {
	// Handle the error
	fmt.Println("Error adding change:", err)
	return
}
logger.LogDataChange("User updated profile", *changeInfo)

// Change logger priority at runtime.
lctx.ChangeMinLogPriority(Debug2)

// Log a debug entry.
logger.LogDebug("Debugging user session", map[string]any{"sessionID": "12345"})

Output:

Index

Examples

Constants

View Source 
const (
	DIALTIMEOUT = 500 * time.Second
	ACTIVITY    = "A"
	DEBUG       = "D"
)
View Source 
const (
	LogPriorityDebug2  = "Debug2"
	LogPriorityDebug1  = "Debug1"
	LogPriorityDebug0  = "Debug0"
	LogPriorityInfo    = "Info"
	LogPriorityWarn    = "Warn"
	LogPriorityErr     = "Err"
	LogPriorityCrit    = "Crit"
	LogPrioritySec     = "Sec"
	LogPriorityUnknown = "Unknown"
)
View Source 
const (
	LogTypeChange   = "C"
	LogTypeActivity = "A"
	LogTypeDebug    = "D"
	LogTypeUnknown  = "U"
)
View Source 
const DefaultPriority = Info

Variables

View Source 
var (
	Index                     = "logharbour"
	LOGHARBOUR_GETLOGS_MAXREC = 5

	Priority = []string{"Debug2", "Debug1", "Debug0", "Info", "Warn", "Err", "Crit", "Sec"}
)

Functions

func GetApps added in v0.16.0 

func GetApps(querytoken string, client *elasticsearch.TypedClient) (apps []string, err error)

GetApps is used to retrieve the list of apps

func GetDebugInfo 

func GetDebugInfo(skip int) (fileName string, lineNumber int, functionName string, stackTrace string)

GetDebugInfo returns debug information including file name, line number, function name and stack trace. The 'skip' parameter determines how many stack frames to ascend So, skip = 0 means GetDebugInfo itself skip = 1 means the caller of GetDebugInfo

func GetLocalIPAddress added in v0.16.0 

func GetLocalIPAddress() (string, error)

GetLocalIPAddress returns the local IPv4 address of the system.

func GetSet added in v0.16.0 

func GetSet(queryToken string, client *elasticsearch.TypedClient, setAttr string, setParam GetSetParam) (map[string]int64, error)

GetSet gets a set of values for an attribute from the log entries specified. This is a faceted search for one attribute.

func GetUnusualIP added in v0.16.0 

func GetUnusualIP(queryToken string, client *elasticsearch.TypedClient, unusualPercent float64, logParam GetUnusualIPParam) ([]string, error)

GetUnusualIP will go through the logs of the last ndays days which match the search criteria, and pull out all the remote IP addresses which account for a low enough percentage of the total to be treated as unusual or suspicious.

Types

type ActivityInfo 

type ActivityInfo any

ActivityInfo holds information about system activities like web service calls or function executions.

type ChangeDetail added in v0.11.0 

type ChangeDetail struct {
	Field  string `json:"field"`
	OldVal any    `json:"old_value"`
	NewVal any    `json:"new_value"`
}

ChangeDetail holds information about a single field change in a data change. It is used to log changes to a single field in a data entity.

func NewChangeDetail added in v0.11.0 

func NewChangeDetail(field string, oldValue, newValue any) ChangeDetail

NewChangeDetail creates a new ChangeDetail instance from given field, oldValue, and newValue. The oldValue and newValue are any type, and internally converted to their string representations using convertToString() in utils.go. This design allows for flexibility in logging changes without enforcing a strict type constraint on the values being logged. It ensures that regardless of the original value type, the change details are stored as strings, which is required for storing it in logharbour storage.

type ChangeInfo 

type ChangeInfo struct {
	Entity  string         `json:"entity"`
	Op      string         `json:"op"`
	Changes []ChangeDetail `json:"changes"`
}

ChangeInfo holds information about data changes such as creations, updates, or deletions.

Example usage of ChangeInfo and ChangeDetail to log changes to an entity. 

func LogEntityChange() {
	 // Create a new ChangeInfo for the "User" entity undergoing an "Update" operation and add changes.
	 changeInfo := NewChangeInfo("User", "Update").
		 AddChange("email", "[email protected]", "[email protected]").
		 AddChange("username", "oldUsername", "newUsername")
	 logger.LogDataChange("User details updated", *changeInfo)
}

func NewChangeInfo added in v0.11.0 

func NewChangeInfo(entity, operation string) *ChangeInfo

NewChangeInfo creates a new ChangeInfo instance.

func (*ChangeInfo) AddChange added in v0.11.0 

func (ci *ChangeInfo) AddChange(field string, oldValue, newValue any) *ChangeInfo

AddChange adds a new change to the ChangeInfo struct. It accepts a field name and old/new values of any type. Internally, it uses NewChangeDetail to create a ChangeDetail struct, which converts the old/new values to strings. This method simplifies the process of adding changes to a log entry, allowing developers to pass values of any type without worrying about their string conversion. The use of convertToString() ensures that all values are consistently logged as strings, which is required for storing them in logharbour storage.

type Consumer added in v0.12.0 

type Consumer interface {
	Start(batchSize int) (<-chan error, error)
	Stop() error
}

Consumer defines the interface for a Kafka consumer.

func NewConsumer added in v0.12.0 

func NewConsumer(brokers []string, topic string, handler MessageHandler) (Consumer, error)

type DebugInfo 

type DebugInfo struct {
	Pid          int    `json:"pid"`
	Runtime      string `json:"runtime"`
	FileName     string `json:"file"`
	LineNumber   int    `json:"line"`
	FunctionName string `json:"func"`
	StackTrace   string `json:"stackTrace"`
	Data         any    `json:"data"`
}

DebugInfo holds debugging information that can help in software diagnostics.

type ElasticsearchClient added in v0.12.0 

type ElasticsearchClient struct {
	// contains filtered or unexported fields
}

func NewElasticsearchClient added in v0.12.0 

func NewElasticsearchClient(cfg elasticsearch.Config) (*ElasticsearchClient, error)

NewElasticsearchClient creates a new Elasticsearch client with the given configuration

func (*ElasticsearchClient) CreateIndex added in v0.17.0 

func (ec *ElasticsearchClient) CreateIndex(indexName, mapping string) error

func (*ElasticsearchClient) IndexExists added in v0.17.0 

func (ec *ElasticsearchClient) IndexExists(indexName string) (bool, error)

func (*ElasticsearchClient) Write added in v0.12.0 

func (ec *ElasticsearchClient) Write(index string, documentID string, body string) error

Write sends a document to Elasticsearch. It implements ElasticsearchWriter.

type ElasticsearchWriter added in v0.12.0 

type ElasticsearchWriter interface {
	Write(index string, documentID string, body string) error
}

ElasticsearchWriter defines methods for Elasticsearch writer

type FallbackWriter 

type FallbackWriter struct {
	// contains filtered or unexported fields
}

FallbackWriter provides an io.Writer that automatically falls back to a secondary writer if the primary writer fails. It is also used if logentry is not valid so that we can still log erroneous entries without writing them to the primary writer.

func NewFallbackWriter 

func NewFallbackWriter(primary, fallback io.Writer) *FallbackWriter

NewFallbackWriter creates a new FallbackWriter with a specified primary and fallback writer.

func (*FallbackWriter) Write 

func (fw *FallbackWriter) Write(p []byte) (n int, err error)

Write attempts to write the byte slice to the primary writer, falling back to the secondary writer on error. It returns the number of bytes written and any error encountered that caused the write to stop early.

type GetLogsParam added in v0.16.0 

type GetLogsParam struct {
	App              *string
	Type             *LogType
	Module           *string
	Who              *string
	Class            *string
	Instance         *string
	Operation        *string
	FromTS           *time.Time
	ToTS             *time.Time
	NDays            *int
	RemoteIP         *string
	Priority         *LogPriority
	SearchAfterTS    *string
	SearchAfterDocID *string
	Field            *string
}

type GetSetParam added in v0.16.0 

type GetSetParam struct {
	App      *string      `json:"app" validate:"omitempty,alpha,lt=30"`
	Type     *LogType     `json:"type" validate:"omitempty,oneof=1 2 3 4"`
	Who      *string      `json:"who" validate:"omitempty,alpha,lt=20"`
	Class    *string      `json:"class" validate:"omitempty,alpha,lt=30"`
	Instance *string      `json:"instance" validate:"omitempty,alpha,lt=30"`
	Op       *string      `json:"op" validate:"omitempty,alpha,lt=25"`
	Fromts   *time.Time   `json:"fromts" validate:"omitempty"`
	Tots     *time.Time   `json:"tots" validate:"omitempty"`
	Ndays    *int         `json:"ndays" validate:"omitempty,number,lt=100"`
	RemoteIP *string      `json:"remoteIP" validate:"omitempty"`
	Pri      *LogPriority `json:"pri" validate:"omitempty,oneof=1 2 3 4 5 6 7 8"`
	// contains filtered or unexported fields
}

type GetUnusualIPParam added in v0.16.0 

type GetUnusualIPParam struct {
	App       *string
	Who       *string
	Class     *string
	Operation *string
	NDays     *int
}

type KafkaConfig added in v0.11.0 

type KafkaConfig struct {
	Brokers []string // List of broker addresses
	Topic   string   // Kafka topic to write messages to

	// Producer configurations
	Retries          *int                 // Maximum number of times to retry sending a message
	RequiredAcks     *sarama.RequiredAcks // Number of acknowledgments required before considering a message as sent
	Timeout          *time.Duration       // Maximum duration to wait for the broker to acknowledge the receipt of a message
	ReturnErrors     *bool                // Whether to return errors that occurred while producing the message
	ReturnSuccesses  *bool                // Whether to return successes of produced messages
	CompressionLevel *int                 // Compression level to use for messages

	// Network configurations
	DialTimeout     *time.Duration // Timeout for establishing network connections
	ReadTimeout     *time.Duration // Timeout for network reads
	WriteTimeout    *time.Duration // Timeout for network writes
	MaxOpenRequests *int           // Maximum number of unacknowledged requests to send before blocking

	// Client configurations
	ClientID *string // User-provided string sent with every request for logging, debugging, and auditing purposes
}

KafkaConfig is a struct that holds the configuration for a Kafka producer. All fields are pointers, which allows us to distinguish between a field that is not set and a field set with its zero value.

type KafkaWriter added in v0.11.0 

type KafkaWriter interface {
	Write(p []byte) (n int, err error)
	Close() error
}

KafkaWriter defines methods for Kafka writer

func NewKafkaWriter added in v0.11.0 

func NewKafkaWriter(kafkaConfig KafkaConfig, opts ...KafkaWriterOption) (KafkaWriter, error)

type KafkaWriterOption added in v0.11.0 

type KafkaWriterOption func(*kafkaWriter)

func WithPoolSize added in v0.11.0 

func WithPoolSize(size int) KafkaWriterOption

type LogData added in v0.16.0 

type LogData struct {
	ChangeData   *ChangeInfo `json:"change_data,omitempty"`   // Data change information
	ActivityData string      `json:"activity_data,omitempty"` // Activity information
	DebugData    *DebugInfo  `json:"debug_data,omitempty"`    // Debug information
}

type LogEntry 

type LogEntry struct {
	Id         string      `json:"id"`                  // id of the log
	App        string      `json:"app"`                 // Name of the application.
	System     string      `json:"system"`              // System where the application is running.
	Module     string      `json:"module"`              // The module or subsystem within the application
	Type       LogType     `json:"type"`                // Type of the log entry.
	Pri        LogPriority `json:"pri"`                 // Severity level of the log entry.
	When       time.Time   `json:"when"`                // Time at which the log entry was created.
	Who        string      `json:"who,omitempty"`       // User or service performing the operation.
	Op         string      `json:"op,omitempty"`        // Operation being performed
	Class      string      `json:"class,omitempty"`     // Unique ID, name of the object instance on which the operation was being attempted
	InstanceId string      `json:"instance,omitempty"`  // Unique ID, name, or other "primary key" information of the object instance on which the operation was being attempted
	Status     Status      `json:"status,omitempty"`    // 0 or 1, indicating success (1) or failure (0), or some other binary representation
	Error      string      `json:"error,omitempty"`     // Error message or error chain related to the log entry, if any.
	RemoteIP   string      `json:"remote_ip,omitempty"` // IP address of the caller from where the operation is being performed.
	Msg        string      `json:"msg,omitempty"`       // A descriptive message for the log entry.
	Data       *LogData    `json:"data,omitempty"`      // The payload of the log entry, can be any type.
}

LogEntry encapsulates all the relevant information for a log message.

func GetChanges added in v0.16.0 

func GetChanges(querytoken string, client *elasticsearch.TypedClient, logParam GetLogsParam) ([]LogEntry, int, error)

GetChanges retrieves an slice of logEntry from Elasticsearch based on the fields provided in logParam.

func GetLogs added in v0.16.0 

func GetLogs(querytoken string, client *elasticsearch.TypedClient, logParam GetLogsParam) ([]LogEntry, int, error)

GetLogs retrieves an slice of logEntry from Elasticsearch based on the fields provided in logParam.

type LogPriority 

type LogPriority int

logPriority defines the severity level of a log message. 

const (
	Debug2 LogPriority = iota + 1 // Debug2 represents extremely verbose debugging information.
	Debug1                        // Debug1 represents detailed debugging information.
	Debug0                        // Debug0 represents high-level debugging information.
	Info                          // Info represents informational messages.
	Warn                          // Warn represents warning messages.
	Err                           // Err represents error messages where operations failed to complete.
	Crit                          // Crit represents critical failure messages.
	Sec                           // Sec represents security alert messages.
)

func (LogPriority) MarshalJSON 

func (lp LogPriority) MarshalJSON() ([]byte, error)

MarshalJSON is required by the encoding/json package. It converts the logPriority to its string representation and returns it as a JSON-encoded value.

func (LogPriority) String added in v0.5.0 

func (lp LogPriority) String() string

String returns the string representation of the logPriority.

func (*LogPriority) UnmarshalJSON added in v0.5.0 

func (lp *LogPriority) UnmarshalJSON(data []byte) error

type LogType 

type LogType int

LogType defines the category of a log message. 

const (
	// Change represents a log entry for data changes.
	Change LogType = iota + 1
	// Activity represents a log entry for activities such as web service calls.
	Activity
	// Debug represents a log entry for debug information.
	Debug
	// Unknown represents an unknown log type.
	Unknown
)

func (LogType) MarshalJSON 

func (lt LogType) MarshalJSON() ([]byte, error)

MarshalJSON is required by the encoding/json package. It converts the LogType to its string representation and returns it as a JSON-encoded value.

func (LogType) String added in v0.5.0 

func (lt LogType) String() string

String returns the string representation of the LogType.

func (*LogType) UnmarshalJSON added in v0.5.0 

func (lt *LogType) UnmarshalJSON(data []byte) error

type Logger 

type Logger struct {
	// contains filtered or unexported fields
}

Logger provides a structured interface for logging. It's designed for each goroutine to have its own instance. Logger is safe for concurrent use. However, it's not recommended to share a Logger instance across multiple goroutines.

If the writer is a FallbackWriter and validation of a log entry fails, the Logger will automatically write the invalid entry to the FallbackWriter's fallback writer. If writing to fallback writer also fails then it writes to STDERR.

The 'With' prefixed methods in the Logger are used to create a new Logger instance with a specific field set to a new value. These methods create a copy of the current Logger, then set the desired field to the new value, and finally return the new Logger. This approach provides a flexible way to create a new Logger with specific settings, without having to provide all settings at once or change the settings of an existing Logger.

func NewLogger 

func NewLogger(context *LoggerContext, appName string, writer io.Writer) *Logger

NewLogger creates a new Logger with the specified application name and writer. We recommend using NewLoggerWithFallback instead of this method.

func NewLoggerWithFallback 

func NewLoggerWithFallback(context *LoggerContext, appName string, fallbackWriter *FallbackWriter) *Logger

NewLoggerWithFallback creates a new Logger with a fallback writer. The fallback writer is used if the primary writer fails or if validation of a log entry fails.

func (*Logger) Crit added in v0.7.0 

func (l *Logger) Crit() *Logger

Crit returns a new Logger with the 'priority' field set to Crit.

func (*Logger) Debug0 added in v0.7.0 

func (l *Logger) Debug0() *Logger

Debug0 returns a new Logger with the 'priority' field set to Debug0.

func (*Logger) Debug1 added in v0.7.0 

func (l *Logger) Debug1() *Logger

Debug1 returns a new Logger with the 'priority' field set to Debug1.

func (*Logger) Debug2 added in v0.7.0 

func (l *Logger) Debug2() *Logger

Debug2 returns a new Logger with the 'priority' field set to Debug2.

func (*Logger) Err added in v0.7.0 

func (l *Logger) Err() *Logger

Err returns a new Logger with the 'priority' field set to Err.

func (*Logger) Error added in v0.9.0 

func (l *Logger) Error(err error) *Logger

Err sets the "error" field for the logger.

func (*Logger) Info added in v0.7.0 

func (l *Logger) Info() *Logger

Info returns a new Logger with the 'priority' field set to Info.

func (*Logger) Log added in v0.5.0 

func (l *Logger) Log(message string)

Log logs a generic message as an activity event.

func (*Logger) LogActivity 

func (l *Logger) LogActivity(message string, data ActivityInfo)

LogActivity logs an activity event.

func (*Logger) LogDataChange 

func (l *Logger) LogDataChange(message string, data ChangeInfo)

LogDataChange logs a data change event.

func (*Logger) LogDebug 

func (l *Logger) LogDebug(message string, data any)

LogDebug logs a debug event.

func (*Logger) Sec added in v0.7.0 

func (l *Logger) Sec() *Logger

Sec returns a new Logger with the 'priority' field set to Sec.

func (*Logger) Warn added in v0.7.0 

func (l *Logger) Warn() *Logger

Warn returns a new Logger with the 'priority' field set to Warn.

func (*Logger) WithClass added in v0.12.0 

func (l *Logger) WithClass(whatClass string) *Logger

WithClass returns a new Logger with the 'whatClass' field set to the specified value.

func (*Logger) WithInstanceId added in v0.12.0 

func (l *Logger) WithInstanceId(whatInstanceId string) *Logger

WithInstanceId returns a new Logger with the 'whatInstanceId' field set to the specified value.

func (*Logger) WithModule 

func (l *Logger) WithModule(module string) *Logger

WithModule returns a new Logger with the 'module' field set to the specified value.

func (*Logger) WithOp 

func (l *Logger) WithOp(op string) *Logger

WithOp returns a new Logger with the 'op' field set to the specified value.

func (*Logger) WithPriority 

func (l *Logger) WithPriority(priority LogPriority) *Logger

WithPriority returns a new Logger with the 'priority' field set to the specified value.

There are shortcut functions like Info(), Warn(), etc. provided as a convenient way to set the priority level for a single call. Each of these function creates a new Logger instance with the specified priority and returns it. The original Logger instance remains unchanged. For example, instead of writing 

logger.WithPriority(logharbour.Info).LogChange(...),

you can simply write 

logger.Info().LogChange(...)

func (*Logger) WithRemoteIP 

func (l *Logger) WithRemoteIP(remoteIP string) *Logger

WithRemoteIP returns a new Logger with the 'remoteIP' field set to the specified value.

func (*Logger) WithStatus 

func (l *Logger) WithStatus(status Status) *Logger

WithStatus returns a new Logger with the 'status' field set to the specified value.

func (*Logger) WithWho 

func (l *Logger) WithWho(who string) *Logger

WithWho returns a new Logger with the 'who' field set to the specified value.

type LoggerContext added in v0.10.0 

type LoggerContext struct {
	// contains filtered or unexported fields
}

LoggerContext provides a shared context (state) for instances of Logger. It contains a minLogPriority field that determines the minimum log priority level that should be logged by any Logger using this context. The mu Mutex ensures that all operations on the minLogPriority field are mutually exclusive, regardless of which goroutine they are performed in.

func NewLoggerContext added in v0.10.0 

func NewLoggerContext(minLogPriority LogPriority) *LoggerContext

NewLoggerContext creates a new LoggerContext with the specified minimum log priority. The returned LoggerContext can be used to create Logger instances with a shared context for all the Logger instances.

func (*LoggerContext) ChangeMinLogPriority added in v0.12.0 

func (lc *LoggerContext) ChangeMinLogPriority(minLogPriority LogPriority)

ChangePriority changes the priority level of the Logger.

func (*LoggerContext) IsDebugMode added in v0.12.0 

func (lc *LoggerContext) IsDebugMode() bool

IsDebugMode checks if debug mode is enabled atomically.

func (*LoggerContext) SetDebugMode added in v0.12.0 

func (lc *LoggerContext) SetDebugMode(enable bool)

SetDebugMode sets the debug mode for all loggers sharing this context. Passing true enables debug logging, while false disables it.

type MessageHandler added in v0.12.0 

type MessageHandler func(messages []*sarama.ConsumerMessage) error

MessageHandler is a function type that processes messages from Kafka.

type Status 

type Status int
const (
	Success Status = iota
	Failure
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.