chronowave

ChronoWave: A schema agnostic data store for append only machine generated data

Chronowave batches input data stream into segments (or blocks) and transforms the semi structured data into columnar formats, followed by creating self compressed index to produce a succinct index data structure.

Simple, Small and Performant

Simple to develop, use and operate. Machine generated data can be in different shape and size. Chronowave simplifies development cycle with its support of schema agnostic semi structured data. Operation is also fairly easy. Chronowave is engineered on succinct self-compressed index data structure, the basis of shared nothing architecture that can scale out on load.

Chronowave uses only single copy of machine generated data to support major use cases, like Analytics, AI and Real Time Monitoring. The size of self-compressed index is only a fraction of the original, Chronowave requires only the index to filter/extract information or restore entire data set with SSQL, Semi Structured Query Language.

Chronowave transforms semi structured data into columnar formats, leverages modern CPU SIMD instructions to execute queries and full text search.

command line example

1. build command line executable

cd cmd/waverider
go build

sample.json is a distributed trace data generated by HotRod app captured in Jaeger. Only partial data listed.

{
   "tags" : [
      {
         "type" : "string",
         "key" : "http.url",
         "value" : "http://0.0.0.0:8083/route?dropoff=577%2C322&pickup=516%2C208"
      }
   ],
  "startTime" : 1601613777130370,
  "spanID" : "2ef6e3c30af421ea",
   "traceID" : "464382d9a88849ff"
}

1. construct index: /startTime as time partition required, /traceID, /spanID will be used latter as K/V usage (optional)

./waverider index ./testdata/sample.json -d data -t '/startTime' -k '/traceID' -k '/spanID'

1. query data: timeframe is a SSQL keyword that tells Chronowave searches data between the required time range. Chronowave supports partial words and wild card full text search.

./waverider query -d data 'find $log where [$log /logs][/startTime timeframe(1601613777130350, 1801613777130470)] [/tags [/key contain("http.url")] [/value contain("dropoff*pickup")]]'

1. key/value lookup: key is a SSQL keyword that tells Chronowave lookups by the key. The JSON path of the key must be provided at the time of building index.

./waverider query -d data 'find $a where [$a /process][/traceID key("464382d9a88849ff")]'

SSQL: Semi Structured Query Language

coming soon