esworker

go-esworker

go-esworker is an async worker that documents can bulk insert, update, delete to the elasticsearch using Golang. It is support to an infrastructure on AWS, GCP, Elastic Cloud, and so on.

Installation

go get -u github.com/gjbae1212/go-esworker

Usage

import (
	"context"
	"log"

	"github.com/gjbae1212/go-esworker"
)

func main() {

	// Create dispatcher
	dispatcher, err := esworker.NewDispatcher(
    		esworker.WithESVersionOption(esworker.V6),
    		esworker.WithAddressesOption([]string{"http://localhost:9200"}),
    		esworker.WithUsernameOption("user"),
    		esworker.WithPasswordOption("password"),
    		esworker.WithErrorHandler(func(err error) {
    			log.Println(err)
    		}),
    	)
    	if err != nil {
    		log.Panic(err)
    	}

	// Start dispatcher
	if err := dispatcher.Start(); err != nil {
		log.Panic(err)
	}

	// Process operations in bulk.
	ctx := context.Background()
	// create doc
	dispatcher.AddAction(ctx, &esworker.StandardAction{
		op:    esworker.ES_CREATE,
		index: "allan",
		id:    "1",
		doc:   map[string]interface{}{"field1": 10},
	})

	// update doc
	dispatcher.AddAction(ctx, &esworker.StandardAction{
		op:    esworker.ES_UPDATE,
		index: "allan",
		id:    "1",
		doc:   map[string]interface{}{"field1": 20},
	})

	// delete doc
	dispatcher.AddAction(ctx, &esworker.StandardAction{
		op:    esworker.ES_DELETE,
		index: "allan",
		id:    "1",
	})
}

Dispatcher Parameters

It should pass parameters for dependency injection when you are creating a go-esworker dispatcher.
A list to support the parameters below.

Action Interface

To deal with operation as insert and update and delete to, you would use to the StandardAction struct or a struct which is implementing esworker.Action interface.

// generate and start dispatcher 
dispatcher, _ := esworker.NewDispatcher()
dispatcher.Start()

// Ex) Standard Action Example
act := &esworker.StandardAction{
	Op: ES_CREATE
	Index: "sample",
	DocType: "_doc",
	Id: "test-id",
	Doc: map[string]interface{}{"field": 1},
}
dispatcher.AddAction(context.Background(), act)


// Ex) Custom Action Example
sampleAction struct {}

func (act *sampleAction) GetOperation() esworker.ESOperation {
	// return esworker.ES_CREATE
	// return esworker.ES_INDEX
	// return esworker.ES_UPDATE
	// return esworker.ES_DELETE    
}

func (act *sampleAction) GetIndex() string {
	// return "your index name"
}

func (act *sampleAction) GetDocType() string {
	//return ""
	//return "doc type"	
}

func (act *sampleAction) GetID() string {
	//return ""
	//return "doc id"		
}

func (act *sampleAction) GetDoc() map[string]interface{} {
	//return map[string]interface{}{}
}
dispatcher.AddAction(context.Background(), &sampleAction{})

If you will make to a custom struct which is implementing esworker.Action interface, it must implement 5 methods.

Elastic Cloud

If you use to infrastructure on Elastic Cloud, you could access to ElasticSearch without endpoint and basic authentication. (How to use API-KEY)

dispatcher, err := esworker.NewDispatcher(
		esworker.WithESVersionOption(esworker.V7),
		esworker.WithCloudIdOption("your-cloud-id"),
		esworker.WithApiKeyOption("api-key"),
)
dispatcher.Start()

LICENSE

This project is following The MIT.