searchgoose

🦢 Simple, distributed, lightweight
RESTful search engine implementation written in go 🔍

Introduction

Searchgoose is simple distributed RESTful search engine supporting searching and indexing data. For Study purposes, mostly understanding the implementation details of how elasticearch is built, focusing on clustering distributed system and supporting full-text search using bleve. Searchgoose aims to provide fast and flexible data handling by solving the problems of the existing relational database, which is bothering tasks to search and divide string data. To solve this problem, this project uses a data structure called an inverted index that supports fast full-text searches. It also implements raft-like algorithm and quorum-based voting system for discovery and election service in clustering mode.

Architecture

REST API

Cluster API

• /_nodes
• /_nodes/stats
• /_cluster/health
• /_cluster/state
• /_cluster/stats

Index / Document API

• /{index}
• /{index}/_doc
• /{index}/_search
• /{index}/_refresh

Build

DockerFile

$ docker build . --rm --tag searchgoose:latest

Makefile

$ make build

Run

Single mode

$ go run main.go -transport.port=8180 -http.port=8080

Clustering mode

$ go run main.go -node.name=sg-node-01 -transport.port=8180 -http.port=8080
$ go run main.go -node.name=sg-node-02 -seed_hosts=127.0.0.1:8180 -transport.port=8179 -http.port=8081
$ go run main.go -node.name=sg-node-03 -seed_hosts=127.0.0.1:8180 -transport.port=8181 -http.port=8082

API

To try any of the below queries you can use the above example quries

Create Index

PUT /test15
content-type: application/json

{
  "settings": {
    "number_of_shards": 3
  },
  "mappings": {
    "properties": {
      "field1": {
        "type": "text"
      }
    }
  }
}

Document Index

PUT /test15/_doc/4
content-type: application/json

{
  "field1": "test",
  "field2": "test2"
}

Search

POST /test15/_search
content-type: application/json

{
  "size": 100,
  "query": {
    "match": {
      "field1": "field test"
    } 
  }
}

Example

The above image describes Searchgoose test application using Guttenberg-Search.

Visualization

The above image describes simplified interface for managing and monitoring Searchgoose clusters by elasticsearch-HQ.