kafka-topic-tailer
Description
This utility is intended to allow you to directly tail logs from kafka, which have been published there via filebeat, instead of waiting until they've been processed and become available at the final destination (typically Elasticsearch).
Flags
-brokers
: (string) Comma separated list of brokers in IP:PORT format
-json
: Parse log entry and only display @timestamp
, beat.hostname
and message
fields (default: false
)
-kver
: (string) Version of Kafka (default: 2.1.0
)
-oldest
: Start the kafka consumer from oldest ofset (default: true
)
-topic
: (string) Name of the kafka topic to consume from
-r
: (string) Regex to filter for specific messages
-tz
: (string) Convert the @timestamp value to a timezone of your choice. (default: Etc/UTC
)
-d
: Enable debug mode logging
-v
: Print version info and exit
Command usage example
./kafka-topic-tailer -brokers "kafka01:2181,kafka02:2181,kafka03:2181" -topic "my-log-topic" -tz "America/Chicago"
Download pre-compiled builds
The pre-compiled releases can be found here
Building from source
Building for your specific OS:
Building for the three main OS varieties (Linux, Darwin & Windows):
Building with debug symbols present:
See Makefile for all available options