data

package
v0.0.0-...-69ec8c5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 28, 2017 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package data is the package for dealing with the data layer of townsourced. This means the following: 

 	RethinkDB
	Memcache
	Elasticsearch

NOTE: For Rethinkdb, be very careful when using maps in structs, as updates won't remove fields in maps 

You'll need to due a full replace instead.  Generally it's safer, and more performant to use a slice instead

Index

Constants

View Source 
const (
	CommentSortNew = "new"
	CommentSortOld = "old"
)

CommentSort sets the sort order on comments

View Source 
const (
	LocationUnitMeter        = "m"
	LocationUnitKilometer    = "km"
	LocationUnitMile         = "mi"
	LocationUnitNauticalMile = "nm"
	LocationUnitFoot         = "ft"
)

LocationUnit defines units of measure for location based queries

View Source 
const (
	// PostStatusDraft not public visible yet
	PostStatusDraft = "draft"
	// PostStatusPublished publicly visible
	PostStatusPublished = "published"
	//PostStatusClosed not searchable but link is still valid
	PostStatusClosed = "closed"
)
View Source 
const (
	PostSearchSortNone           = ""
	PostSearchPriceSortHighToLow = "pricehigh"
	PostSearchPriceSortLowToHigh = "pricelow"
	PostSearchSortNew            = "new"
	PostSearchSortOld            = "old"
)

PostSort sets the sort order on post search results

View Source 
const AnnouncementTown = "announcements"

AnnouncementTown is the key of the townsourced announcements town

View Source 
const CommentMaxDepth = 7

CommentMaxDepth is the max depth at which to retrieve nested comment children

View Source 
const DatabaseName = "townsourced"

DatabaseName is the name of the townsourced database

View Source 
const EmptyKey = Key("")

EmptyKey is an empty key value

View Source 
const EmptyUUID = UUID("")

EmptyUUID is an empty UUID value

View Source 
const MaxKeyLength = 127

MaxKeyLength is the longest possible Key

Variables

View Source 
var ErrNotFound = errors.New("Data not found")

ErrNotFound is the error returned when no records are found for the given criteria

View Source 
var ErrVersionStale = &fail.Fail{
	Message: "You are trying to update this record based on old information.  " +
		"Please refresh your screen and try again.",
	HTTPStatus: http.StatusConflict,
}

ErrVersionStale is when an update is attempted with a version older than the current version in the database

Functions

func AdminLastPosts 

func AdminLastPosts(result interface{}) error

AdminLastPosts returns the last 3 posts published

func AdminLastTowns 

func AdminLastTowns(result interface{}) error

AdminLastTowns returns the last 3 towns registered

func AdminLastUsers 

func AdminLastUsers(result interface{}) error

AdminLastUsers returns the last 3 users who signed up

func AdminPostCountTrend 

func AdminPostCountTrend(result interface{}, since time.Time) error

AdminPostCountTrend gets the trend in post counts by day

func AdminTownCountTrend 

func AdminTownCountTrend(result interface{}, since time.Time) error

AdminTownCountTrend gets the trend in town counts by day

func AdminUserCountTrend 

func AdminUserCountTrend(result interface{}, since time.Time) error

AdminUserCountTrend gets the trend in user counts by day

func AttemptsGet 

func AttemptsGet(result interface{}, id, requestType string) error

AttemptsGet gets all previous attempts for the given ipaddress+type

func AttemptsSet 

func AttemptsSet(attempts interface{}, id, requestType string, timeRange time.Duration) error

AttemptsSet sets the current attemps for the given ipaddress / type

func CacheClient 

func CacheClient() *memcache.Client

CacheClient returns the underlying memcached client should usually only be used in tools and tests

func CommentGet 

func CommentGet(result interface{}, key UUID) error

CommentGet retrieves a single comment

func CommentGetTree 

func CommentGetTree(result interface{}, key UUID, limit int, sort string) error

CommentGetTree retrieves a single comment and it's children

func CommentUpdate 

func CommentUpdate(comment interface{}, key UUID) error

CommentUpdate updates an existing comment

func CommentsGet 

func CommentsGet(result interface{}, postKey, parent UUID, from, limit int, sort string) (err error)

CommentsGet retrieves a set of comments

func CommentsGetByUser 

func CommentsGetByUser(result interface{}, username Key, public bool, since time.Time, limit int) (err error)

CommentsGetByUser retrieves a set of comments posted by a given user

func DatabaseSession 

func DatabaseSession() *rt.Session

DatabaseSession returns the underlying rethinkdb database session should usually only be used in tools and tests

func FromUUID 

func FromUUID(g UUID) string

FromUUID creates a url friendly shortend string from the passed in UUID

func IP2LocationGet 

func IP2LocationGet(result interface{}, ipAddress string) error

IP2LocationGet retrieves a LatLng location from the passed in IPAddress

func IP2LocationImport 

func IP2LocationImport(entries interface{}) error

IP2LocationImport imports an array for IP2Location entries

func IP2LocationTruncate 

func IP2LocationTruncate() error

IP2LocationTruncate truncates the IP2Location table by dropping it and recreating it. Much faster than deleting all the records individually

func IPNumber 

func IPNumber(address string) uint64

IPNumber returns a sortable int version of a string IP Address

func ImageDelete 

func ImageDelete(key UUID) error

ImageDelete deletes an image

func ImageDeleteOrphans 

func ImageDeleteOrphans(updatedSince time.Time) error

ImageDeleteOrphans deletes all images that aren't currently in use and haven't been updated since the passed in time

func ImageGet 

func ImageGet(result interface{}, key UUID, thumb, placeholder bool) error

ImageGet retrieves an image by it's key

func ImageUpdate 

func ImageUpdate(image interface{}, key UUID) error

ImageUpdate updates an existing image

func Init 

func Init(cfg *Config) error

Init initialized the data layer based on the passed in configuration

func Log 

func Log(entry interface{}) error

Log writes a new log entry

func NotificationGet 

func NotificationGet(result interface{}, notificationKey UUID) error

NotificationGet retrieves a specific notification for a user

func NotificationGetAll 

func NotificationGetAll(result interface{}, username Key, since time.Time, limit int) error

NotificationGetAll retrieves all notifications for a user

func NotificationGetUnread 

func NotificationGetUnread(result interface{}, username Key, since time.Time, limit int) error

NotificationGetUnread retrieves all unread notifications for a user

func NotificationInsert 

func NotificationInsert(notification interface{}) error

NotificationInsert inserts a new user notification into the database

func NotificationUnreadCount 

func NotificationUnreadCount(username Key) (int, error)

NotificationUnreadCount retrieves the number of unread notifications for a user

func NotificationUpdate 

func NotificationUpdate(notification interface{}, key UUID) error

NotificationUpdate updates a single notification

func NotificationUpdateUnread 

func NotificationUpdateUnread(notification interface{}, username Key) error

NotificationUpdateUnread updates all unread notifications

func NotificationsGetSent 

func NotificationsGetSent(result interface{}, username Key, since time.Time, limit int) (err error)

NotificationsGetSent gets all sent notifications for a user

func PostAllCount 

func PostAllCount() (int, error)

PostAllCount returns the count of the total number of posts, usually used by maintenance and not the frontend

func PostGet 

func PostGet(result interface{}, key UUID) error

PostGet retrieves a post by a specific post key

func PostGetAll 

func PostGetAll(result interface{}, from, limit int) error

PostGetAll retrieves all posts This likely shouldn't be used for the actual website, and should only be used for maintenance / tasks

func PostGetByTowns 

func PostGetByTowns(result interface{}, towns []Key, category string, since time.Time, limit int,
	showModerated bool) (err error)

PostGetByTowns retrieves posts by the passed in list of townkeys

func PostGetByUser 

func PostGetByUser(result interface{}, username Key, status string, public bool, since time.Time, limit int) (err error)

PostGetByUser retrieves posts by a specific user

func PostGetUserSaved 

func PostGetUserSaved(result interface{}, username Key, status string, from, limit int) (err error)

PostGetUserSaved retrieves posts saved by a specific user in the order in which they were saved

func PostIndex 

func PostIndex(post interface{}, key UUID) error

PostIndex indexes the post for full text searching

func PostRemoveIndex 

func PostRemoveIndex(key UUID) error

PostRemoveIndex removes the given post from the full text search index

func PostUpdate 

func PostUpdate(post interface{}, key UUID) error

PostUpdate updates an existing post

func SearchClient 

func SearchClient() *elastic.Client

SearchClient returns the underlying elasticSearch client should usually only be used in tools and tests

func SessionGet 

func SessionGet(result interface{}, sessionKey string) error

SessionGet retrieves a session

func SessionInsert 

func SessionInsert(s interface{}, sessionKey string, expires time.Time) error

SessionInsert inserts a session

func SessionUpdate 

func SessionUpdate(s interface{}, sessionKey string, expires time.Time) error

SessionUpdate updates a session

func TaskClaim 

func TaskClaim(owner Key, limit uint) error

TaskClaim marks the next set of unclaimed, non-closed tasks as owned by the given user this is to immediately prevent any other task runners from sharing these tasks A task should only belong to one runner at a time

func TaskDeleteClosed 

func TaskDeleteClosed() error

TaskDeleteClosed deletes all closed tasks

func TaskGetMine 

func TaskGetMine(result interface{}, owner Key) (err error)

TaskGetMine retrieves all unprocessed tasks that have been marked as the owenres, who's nextRun time has passed in order by priority, then oldest tasks

func TaskGetOpenType 

func TaskGetOpenType(result interface{}, taskType string, limit uint) (err error)

TaskGetOpenType gets all open tasks of a given type

func TaskInsert 

func TaskInsert(task interface{}) error

TaskInsert inserts a new task to be run

func TaskUpdate 

func TaskUpdate(task interface{}, key UUID) error

TaskUpdate updates a single task

func TempTokenExpire 

func TempTokenExpire(token string) error

TempTokenExpire will expire the temp token with the passed in ID

func TempTokenGet 

func TempTokenGet(tokenData interface{}, token string) error

TempTokenGet gets a tempory token

func TempTokenSet 

func TempTokenSet(tokenData interface{}, token string, expires time.Duration) error

TempTokenSet sets a session in the memcache if expires is less than 30 minutes, otherwise it goes in rethink

func TownAllCount 

func TownAllCount() (int, error)

TownAllCount returns the count of the total number of towns, usually used by maintenance and not the frontend

func TownGet 

func TownGet(result interface{}, key Key) error

TownGet retrieves a single town

func TownGetAll 

func TownGetAll(result interface{}, from, limit int) error

TownGetAll retrieves all towns This likely shouldn't be used for the actual website, and should only be used for maintenance / tasks

func TownGetByLocation 

func TownGetByLocation(result interface{}, locationQry LocationSearcher, from, limit int) error

TownGetByLocation retrieves all towns within the distance passed from the location passed in

func TownGetPopulation 

func TownGetPopulation(key Key) (int, error)

TownGetPopulation retrieves the current population of a town

func TownIndex 

func TownIndex(town interface{}, key Key) error

TownIndex indexes the town for full text searching

func TownInsert 

func TownInsert(data interface{}, key Key) error

TownInsert inserts a new town

func TownRemoveIndex 

func TownRemoveIndex(key Key) error

TownRemoveIndex removes the given town from the full text search indexes

func TownUpdate 

func TownUpdate(data interface{}, key Key) error

TownUpdate updates a town

func Towns 

func Towns(result interface{}, keys ...Key) (err error)

Towns returns a set of towns in the database

func UserAllCount 

func UserAllCount() (int, error)

UserAllCount returns the count of the total number of users

func UserGet 

func UserGet(result interface{}, username Key) error

UserGet gets a user

func UserGetEmail 

func UserGetEmail(result interface{}, email string) error

UserGetEmail gets a user with an email address

func UserGetFacebook 

func UserGetFacebook(result interface{}, facebookID string) error

UserGetFacebook gets a user with their FacebookID

func UserGetGoogle 

func UserGetGoogle(result interface{}, googleID string) error

UserGetGoogle gets a user with their GoogleID

func UserGetMatching 

func UserGetMatching(result interface{}, match string, limit int) error

UserGetMatching retrieves all users who's username starts with the passed in string

func UserGetTwitter 

func UserGetTwitter(result interface{}, twitterID string) error

UserGetTwitter gets a user with their TwitterID

func UserInsert 

func UserInsert(user interface{}) error

UserInsert inserts a new user into the database

func UserUpdate 

func UserUpdate(user interface{}, username Key) error

UserUpdate updates an existing user

Types

type AreaSearch 

type AreaSearch struct {
	// contains filtered or unexported fields
}

AreaSearch is a location search for everything withing a rectangle area

func NewAreaSearch 

func NewAreaSearch(northBounds, southBounds, eastBounds, westBounds float64) (*AreaSearch, error)

NewAreaSearch returns a new area search query

type CacheConfig 

type CacheConfig struct {
	Addresses []string `json:"addresses,omitempty"`
}

CacheConfig is cache server connection configuration

type Config 

type Config struct {
	DB      DBConfig     `json:"db"`
	Cache   CacheConfig  `json:"cache"`
	Search  SearchConfig `json:"search"`
	DevMode bool         `json:"-"`
}

Config is data layer configuration thinks like database ip addresses, cache servers, etc

func DefaultConfig 

func DefaultConfig() *Config

DefaultConfig returns the default configuration for the data layer

type DBConfig 

type DBConfig struct {
	Address   string   `json:"address,omitempty"`
	Addresses []string `json:"addresses,omitempty"`
	Database  string   `json:"database,omitempty"`
	AuthKey   string   `json:"authkey,omitempty"`
	Timeout   string   `json:"timeout,omitempty"`

	TLSConfig *tls.Config `json:"tlsconfig,omitempty"`

	MaxIdle int `json:"max_idle,omitempty"`
	MaxOpen int `json:"max_open,omitempty"`

	// DiscoverHosts is used to enable host discovery, when true the driver
	// will attempt to discover any new nodes added to the cluster and then
	// start sending queries to these new nodes.
	DiscoverHosts bool `json:"discoverHosts,omitempty"`
	// NodeRefreshInterval is used to determine how often the driver should
	// refresh the status of a node.
	NodeRefreshInterval time.Duration `json:"nodeRefreshInterval,omitempty"`
	// contains filtered or unexported fields
}

DBConfig is database config

type DistanceSearch 

type DistanceSearch struct {
	// contains filtered or unexported fields
}

DistanceSearch is a location search based on distance from a single point

func NewDistanceSearch 

func NewDistanceSearch(latLng LatLng, maxDistance float64, unit string) *DistanceSearch

NewDistanceSearch creates a new distance search type

type Key 

type Key string

Key is a unique identifier for a given peice of data 

Key handling rules are as follows
	Never use data.Key(value), use data.NewKey(value)
	Web layer code deals with strings, app layer code deals with keys
		the transfer from Web to App will make sure keys are properly cased

func NewKey 

func NewKey(s string) Key

NewKey returns a new Key from the passed in string use this instead of data.Key(key) to make new keys to ensure that it's properly lowercase

func NewKeySlice 

func NewKeySlice(s []string) []Key

NewKeySlice is a help for building a slice of keys from strings

func (Key) KeyWhen 

func (k Key) KeyWhen() KeyWhen

KeyWhen returns a new KeyWhen struct

func (*Key) UnmarshalJSON 

func (k *Key) UnmarshalJSON(input []byte) error

UnmarshalJSON implments a custom JSON unmarshaller for reading a Key from JSON input, to ensure all keys coming from user input via JSON are properly lower case

type KeyWhen 

type KeyWhen struct {
	Key  Key       `json:"key"`
	When time.Time `json:"when"`
}

KeyWhen is a helper for the serveral instances where you want to keep track of a list of joining keys as well as when those keys where added joining a town saved posts, etc

type KeyWhenSlice 

type KeyWhenSlice []KeyWhen

KeyWhenSlice is a slice of KeyWhens

func (KeyWhenSlice) Keys 

func (k KeyWhenSlice) Keys() []Key

Keys gets a list of keys form a KeyWhen slice

type LatLng 

type LatLng types.Point

LatLng is a Latitude and Longitude grouping

func NewLatLng 

func NewLatLng(latitude, longitude float64) (LatLng, error)

NewLatLng returns a new LatLng type for use with locations

type LocationSearcher 

type LocationSearcher interface {
	// contains filtered or unexported methods
}

LocationSearcher is an interface for searching based on location

type SearchConfig 

type SearchConfig struct {
	Addresses  []string          `json:"addresses,omitempty"`
	MaxRetries int               `json:"maxRetries,omitEmpty"`
	Index      SearchIndexConfig `json:"index,omitempty"`
}

SearchConfig is search server connection configuration

type SearchIndexConfig 

type SearchIndexConfig struct {
	Name     string `json:"name,omitempty"`
	Shards   int    `json:"shards,omitempty"`
	Replicas int    `json:"replicas,omitempty"`
}

SearchIndexConfig is config for how the main elasticSearch index should be configure

type SearchResult 

type SearchResult struct {
	// contains filtered or unexported fields
}

SearchResult is the result values of a search 

used for Deserializing data from the results of a search

TODO: Highlight results? TODO: Search Suggestions

func PostSearch 

func PostSearch(searchText string, tags []string, towns []Key, category string, from, limit int, postSort string,
	minPrice, maxPrice float64, showModerated bool) (*SearchResult, error)

PostSearch retrieves posts in relevant order by the search text or tags, for the given towns use PostSort* enumeration for sorting

func TownGetBySearch 

func TownGetBySearch(search string, from, limit int) (*SearchResult, error)

TownGetBySearch retrieves a list of towns based on a full text search of the town's name and description

func (*SearchResult) Count 

func (r *SearchResult) Count() int

Count returns the number of search results

func (*SearchResult) Next 

func (r *SearchResult) Next(result interface{}) error

Next fetches the next value from the search result returns io.EOF when there are no more items example: 

posts = make([]Post, result.Count())

for i := range posts {
	err = result.Next(&posts[i])
	if err != nil {
		return nil, err
	}
}

type UUID 

type UUID string

A UUID is like a key (a unique identifier), but it is machine generated rather than user generated and it is shortened for user facing urls and json data

func CommentInsert 

func CommentInsert(comment interface{}) (UUID, error)

CommentInsert inserts a new user notification into the database

func ImageInsert 

func ImageInsert(image interface{}) (UUID, error)

ImageInsert inserts a new image into the database

func NewUUID 

func NewUUID() UUID

NewUUID returns a new generated UUID this should rarely be used, and the database should mostly be responsible for generating new UUIDs

func PostInsert 

func PostInsert(post interface{}) (UUID, error)

PostInsert inserts a new post into the database

func ToUUID 

func ToUUID(b64 string) UUID

ToUUID takes a Base64 encoded string and converts it to a full UUID if it can't be decoded then an empty UUID is returned

func (*UUID) MarshalJSON 

func (g *UUID) MarshalJSON() ([]byte, error)

MarshalJSON implements a custom JSON marshaler for a UUID to shorten the UUID via base64 for more readable URLs

func (UUID) UUIDWhen 

func (g UUID) UUIDWhen() UUIDWhen

UUIDWhen returns a new UUIDWhen struct

func (*UUID) UnmarshalJSON 

func (g *UUID) UnmarshalJSON(input []byte) error

UnmarshalJSON implments a custom JSON unmarshaller for reading the base64 encoded value that the UUID was written to

type UUIDWhen 

type UUIDWhen struct {
	Key  UUID      `json:"key"`
	When time.Time `json:"when"`
}

UUIDWhen is a helper for the serveral instances where you want to keep track of a list of joining UUIDS as well as when those guids where added saved posts, etc

type UUIDWhenSlice 

type UUIDWhenSlice []UUIDWhen

UUIDWhenSlice is a slice of UUIDWhens

func (UUIDWhenSlice) UUIDs 

func (g UUIDWhenSlice) UUIDs() []UUID

UUIDs gets a list of guids form a UUIDWhen slice

type Version 

type Version struct {
	VerTag  string    `json:"vertag,omitempty" gorethink:",omitempty"`
	Created time.Time `json:"created,omitempty" gorethink:",omitempty"`
	Updated time.Time `json:"updated,omitempty" gorethink:",omitempty"`
}

Version is meant to be used as an anonymous struct to gain built in version protection on updates, and track created an updated dates

func (*Version) Rev 

func (v *Version) Rev()

Rev generates a new unique version identifier

func (*Version) Ver 

func (v *Version) Ver() string

Ver returns the unique version of this record

func (*Version) VerField 

func (v *Version) VerField() string

VerField returns the field in which the version is stored

Source Files

View all Source files

Directories

Path Synopsis
Package private is for private items like passwords, API keys, and anything else that shouldn't be released to a public repository *when* townsourced is opensourced.
 Package private is for private items like passwords, API keys, and anything else that shouldn't be released to a public repository *when* townsourced is opensourced.
tools
ip2location
reindex/post
reindex/town

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.