shared

package module
v0.0.0-...-bfbcaf2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 25, 2024 License: MIT Imports: 12 Imported by: 16

README

stablecog/sc-go/shared

Various shared "models" used by the other modules in this workspace.

Documentation

Index

Constants

View Source
const (
	QUEUE_PRIORITY_1 uint8 = iota + 1
	QUEUE_PRIORITY_2
	QUEUE_PRIORITY_3
	QUEUE_PRIORITY_4
	QUEUE_PRIORITY_5
	QUEUE_PRIORITY_6
	QUEUE_PRIORITY_7
	QUEUE_PRIORITY_8
	QUEUE_PRIORITY_9
	QUEUE_PRIORITY_10
)

Queue priorities

View Source
const API_TOKEN_PREFIX = "sc-"

Prefix on all API tokens

View Source
const APP_VERSION = "5.66"

App version

View Source
const AUTO_UPSCALE_RETRY_DURATION = 30 * time.Second

! Auto-upscale If criteria not met for auto-upscale, retry after this period

View Source
const COG_REDIS_QUEUE = "input_queue"

The name of the redis stream used to enqueue worker requests

View Source
const COG_REDIS_VOICEOVER_QUEUE = "input_queue_for_voiceover"

The name of the redis stream used to enqueue worker voiceover requests

View Source
const DEFAULT_API_TOKEN_NAME = "Secret key"

Default name for API tokens

View Source
const DEFAULT_GENERATE_GUIDANCE_SCALE float32 = 7.0
View Source
const DEFAULT_GENERATE_INFERENCE_STEPS int32 = 30
View Source
const DEFAULT_GENERATE_NUM_OUTPUTS int32 = 2
View Source
const DEFAULT_GENERATE_OUTPUT_EXTENSION = JPEG

! Image Generation Defaults

View Source
const DEFAULT_GENERATE_OUTPUT_QUALITY = 85
View Source
const DEFAULT_GENERATE_PROMPT_STRENGTH float32 = 0.5
View Source
const DEFAULT_PROCESS_TYPE = GENERATE

Default image extension for generate

View Source
const DEFAULT_UPSCALE_OUTPUT_EXTENSION = JPEG
View Source
const DEFAULT_UPSCALE_OUTPUT_QUALITY = 85
View Source
const DEFAULT_UPSCALE_SCALE = int32(4)

! TODO - Does nothing

View Source
const DEFAULT_VOICEOVER_DENOISE_AUDIO = true
View Source
const DEFAULT_VOICEOVER_REMOVE_SILENCE = true
View Source
const DEFAULT_VOICEOVER_TEMPERATURE float32 = 0.7
View Source
const DELETE_BANNED_USER_DATA_AFTER = 24 * time.Hour

! Deleting user data

View Source
const DISCORD_VERIFY_TOKEN_EXPIRY = 5 * time.Minute

Discord Bot/Connection

View Source
const FREE_CREDIT_AMOUNT_DAILY = 10

Free credit replenishments How much to give per day

View Source
const FREE_CREDIT_REPLENISHMENT_INTERVAL = 12 * time.Hour

How often to replenish (related to updated_at on credits) They get up to FREE_CREDIT_AMOUNT_DAILY in this time period

View Source
const MAX_API_TOKENS_PER_USER = 10

! API Tokens Maximum number of tokens a user can have at any given time

View Source
const MAX_GENERATE_HEIGHT = 1536
View Source
const MAX_GENERATE_IMAGE_SIZE = 1024 * 1024 * 5

Maximum size of a custom image sent to generate for img2img

View Source
const MAX_GENERATE_INTERFERENCE_STEPS_FREE = 30
View Source
const MAX_GENERATE_NUM_OUTPUTS = 4
View Source
const MAX_GENERATE_WIDTH = 1536

Generation related

View Source
const MAX_GUIDANCE_SCALE = 30.0
View Source
const MAX_PROMPT_LENGTH = 500

Prompt related

View Source
const MAX_PROMPT_STRENGTH = 1.0
View Source
const MAX_PRO_PIXEL_STEPS = 1024 * 1024 * 30
View Source
const MAX_QUEUED_ITEMS_FREE = 1
View Source
const MAX_QUEUED_ITEMS_PRO = 3
View Source
const MAX_QUEUED_ITEMS_STARTER = 2
View Source
const MAX_QUEUED_ITEMS_ULTIMATE = 4

Max queued items allowed

View Source
const MAX_QUEUED_ITEMS_VOICEOVER = 1
View Source
const MAX_TOKEN_NAME_SIZE = 50

Max chars in an API token name

View Source
const MAX_UPLOAD_SIZE_MB = 10

Max upload size allowed for img2img/upscale

View Source
const MAX_UPSCALE_IMAGE_SIZE = 1024 * 1024 * 10

Maximum size of a custom image sent to upscale 10MB

View Source
const MAX_UPSCALE_MEGAPIXELS = 1024 * 1024

For custom images

View Source
const MIN_GENERATE_HEIGHT = 256
View Source
const MIN_GENERATE_NUM_OUTPUTS = 1
View Source
const MIN_GENERATE_WIDTH = 256
View Source
const MIN_GUIDANCE_SCALE = 1.0
View Source
const MIN_INFERENCE_STEPS = 10
View Source
const MIN_PROMPT_STRENGTH = 0.0
View Source
const NSFW_ERROR = "NSFW"

NSFW Error

View Source
const QUEUE_OVERFLOW_MAX = 50

! API Queue Overflow Max items in overflow queue

View Source
const QUEUE_OVERFLOW_PENALTY_MS = 150

Penalty for queue overflow Computed as time.Sleep(QUEUE_OVERFLOW_PENALTY * QUEUE_OVERFLOW_SIZE)

View Source
const REDIS_APITOKEN_COG_CHANNEL = "cog:apitoken_message"

This redis channel is when webhook sends a request from API token

View Source
const REDIS_DISCORD_COG_CHANNEL = "cog:discord_message"

This redis channel is for discord bot, when a user has connected their account

View Source
const REDIS_INTERNAL_COG_CHANNEL = "cog:internal_message"

This redis channel is when webhook sends an internal request we care about

View Source
const REDIS_KEY = "queue_throttler"
View Source
const REDIS_SSE_BROADCAST_CHANNEL = "sse:broadcast_channel"

This redis channel our servers publish to when we want to broadcast SSE events to clients

View Source
const REQUEST_COG_TIMEOUT = 180 * time.Second

After this period, a request will timeout and a user will be refunded But the generation/upscale may still go through, if it takes longer than this

View Source
const REQUEST_COG_TIMEOUT_VOICEOVER = 180 * time.Second
View Source
const TIMEOUT_ERROR = "TIMEOUT"

Timeout

View Source
const TIPPABLE_CREDIT_MULTIPLIER = 0.2

Tippable credits Give credits * TIPPABLE_CREDIT_MULTIPLIER as "tippable" type when replenishing subscription/adhoc purchases

View Source
const VOICEOVER_CREDIT_COST_PER_CHARACTER = 0.0175

! Voiceover Calculated as math.Ceil(VOICEOVER_CREDIT_COST_PER_CHARACTER * len(text))

View Source
const VOICEOVER_MAX_TEXT_LENGTH = 500

Variables

View Source
var ALLOWED_PROCESS_TYPES = []ProcessType{GENERATE, UPSCALE, GENERATE_AND_UPSCALE}

Allowed image extensions for upload

View Source
var ALLOWS_IMAGE_EXTENSIONS_UPLOAD = []ImageExtension{WEBP, JPEG, PNG}

Allowed image extensions for upload

Functions

func IsBlacklisted

func IsBlacklisted(name string) bool

Types

type Cache

type Cache struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

A singleton that caches the features available to free users Avoids having to query the database every time a user requests the features

func GetCache

func GetCache() *Cache

func (*Cache) AdminIDs

func (f *Cache) AdminIDs() []uuid.UUID

func (*Cache) BannedWords

func (f *Cache) BannedWords() []*ent.BannedWords

func (*Cache) DisposableEmailDomains

func (f *Cache) DisposableEmailDomains() []string

func (*Cache) GenerationModels

func (f *Cache) GenerationModels() []*ent.GenerationModel

func (*Cache) GetCompatibleSchedulerIDsForModel

func (f *Cache) GetCompatibleSchedulerIDsForModel(ctx context.Context, modelId uuid.UUID) []uuid.UUID

func (*Cache) GetDefaultGenerationModel

func (f *Cache) GetDefaultGenerationModel() *ent.GenerationModel

func (*Cache) GetDefaultScheduler

func (f *Cache) GetDefaultScheduler() *ent.Scheduler

func (*Cache) GetDefaultSchedulerIDForModel

func (f *Cache) GetDefaultSchedulerIDForModel(modelId uuid.UUID) uuid.UUID

func (*Cache) GetDefaultUpscaleModel

func (f *Cache) GetDefaultUpscaleModel() *ent.UpscaleModel

func (*Cache) GetDefaultVoiceoverModel

func (f *Cache) GetDefaultVoiceoverModel() *ent.VoiceoverModel

func (*Cache) GetDefaultVoiceoverSpeaker

func (f *Cache) GetDefaultVoiceoverSpeaker() *ent.VoiceoverSpeaker

func (*Cache) GetGenerationModelByID

func (f *Cache) GetGenerationModelByID(id uuid.UUID) *ent.GenerationModel

func (*Cache) GetGenerationModelNameFromID

func (f *Cache) GetGenerationModelNameFromID(id uuid.UUID) string

func (*Cache) GetSchedulerNameFromID

func (f *Cache) GetSchedulerNameFromID(id uuid.UUID) string

func (*Cache) GetUpscaleModelNameFromID

func (f *Cache) GetUpscaleModelNameFromID(id uuid.UUID) string

func (*Cache) GetVoiceoverModelNameFromID

func (f *Cache) GetVoiceoverModelNameFromID(id uuid.UUID) string

func (*Cache) GetVoiceoverSpeakerNameFromID

func (f *Cache) GetVoiceoverSpeakerNameFromID(speakerId uuid.UUID) string

func (*Cache) GetVoiceoverSpeakersForModel

func (f *Cache) GetVoiceoverSpeakersForModel(modelId uuid.UUID) []*ent.VoiceoverSpeaker

func (*Cache) IPBlacklist

func (f *Cache) IPBlacklist() []string

func (*Cache) IsAdmin

func (f *Cache) IsAdmin(id uuid.UUID) bool

func (*Cache) IsDisposableEmail

func (f *Cache) IsDisposableEmail(email string) bool

func (*Cache) IsIPBanned

func (f *Cache) IsIPBanned(ip string) bool

func (*Cache) IsThumbmarkIDBanned

func (f *Cache) IsThumbmarkIDBanned(ip string) bool

func (*Cache) IsUsernameBlacklisted

func (f *Cache) IsUsernameBlacklisted(username string) bool

func (*Cache) IsValidGenerationModelID

func (f *Cache) IsValidGenerationModelID(id uuid.UUID) bool

func (*Cache) IsValidShedulerID

func (f *Cache) IsValidShedulerID(id uuid.UUID) bool

func (*Cache) IsValidUpscaleModelID

func (f *Cache) IsValidUpscaleModelID(id uuid.UUID) bool

func (*Cache) IsValidVoiceoverModelID

func (f *Cache) IsValidVoiceoverModelID(id uuid.UUID) bool

func (*Cache) IsValidVoiceoverSpeakerID

func (f *Cache) IsValidVoiceoverSpeakerID(speakerId uuid.UUID, modelId uuid.UUID) bool

func (*Cache) Schedulers

func (f *Cache) Schedulers() []*ent.Scheduler

func (*Cache) SetAdminUUIDs

func (f *Cache) SetAdminUUIDs(ids []uuid.UUID)

func (*Cache) ThumbmarkIDBlacklist

func (f *Cache) ThumbmarkIDBlacklist() []string

func (*Cache) UpdateBannedWords

func (f *Cache) UpdateBannedWords(bannedWords []*ent.BannedWords)

func (*Cache) UpdateDisposableEmailDomains

func (f *Cache) UpdateDisposableEmailDomains(domains []string)

func (*Cache) UpdateGenerationModels

func (f *Cache) UpdateGenerationModels(models []*ent.GenerationModel)

func (*Cache) UpdateIPBlacklist

func (f *Cache) UpdateIPBlacklist(ips []string)

func (*Cache) UpdateSchedulers

func (f *Cache) UpdateSchedulers(schedulers []*ent.Scheduler)

func (*Cache) UpdateThumbmarkIDBlacklist

func (f *Cache) UpdateThumbmarkIDBlacklist(tm []string)

func (*Cache) UpdateUpscaleModels

func (f *Cache) UpdateUpscaleModels(models []*ent.UpscaleModel)

func (*Cache) UpdateUsernameBlacklist

func (f *Cache) UpdateUsernameBlacklist(blacklist []string)

func (*Cache) UpdateVoiceoverModels

func (f *Cache) UpdateVoiceoverModels(models []*ent.VoiceoverModel)

func (*Cache) UpdateVoiceoverSpeakers

func (f *Cache) UpdateVoiceoverSpeakers(speakers []*ent.VoiceoverSpeaker)

func (*Cache) UpscaleModels

func (f *Cache) UpscaleModels() []*ent.UpscaleModel

func (*Cache) VoiceoverModels

func (f *Cache) VoiceoverModels() []*ent.VoiceoverModel

func (*Cache) VoiceoverSpeakers

func (f *Cache) VoiceoverSpeakers() []*ent.VoiceoverSpeaker

type ImageExtension

type ImageExtension string

Allowed image extensions used by various APIs

const (
	PNG  ImageExtension = "png"
	JPG  ImageExtension = "jpg"
	JPEG ImageExtension = "jpeg"
	WEBP ImageExtension = "webp"
)

type LivePageMessage

type LivePageMessage struct {
	ProcessType      ProcessType         `json:"process_type"`
	ID               string              `json:"id"`
	CountryCode      string              `json:"country_code"`
	Status           LivePageStatus      `json:"status"`
	FailureReason    string              `json:"failure_reason,omitempty"`
	Width            *int32              `json:"width,omitempty"`
	Height           *int32              `json:"height,omitempty"`
	TargetNumOutputs int32               `json:"target_num_outputs"`
	ActualNumOutputs int                 `json:"actual_num_outputs"`
	NSFWCount        *int32              `json:"nsfw_count,omitempty"`
	CreatedAt        time.Time           `json:"created_at"`
	StartedAt        *time.Time          `json:"started_at,omitempty"`
	CompletedAt      *time.Time          `json:"completed_at,omitempty"`
	ProductID        *string             `json:"product_id,omitempty"`
	SystemGenerated  bool                `json:"system_generated"`
	Source           enttypes.SourceType `json:"source,omitempty"`
	Temperature      *float32            `json:"temperature,omitempty"`
	RemoveSilence    *bool               `json:"remove_silence,omitempty"`
	DenoiseAudio     *bool               `json:"denoise_audio,omitempty"`
	SpeakerID        *uuid.UUID          `json:"speaker_id,omitempty"`
}

type LivePageStatus

type LivePageStatus string
const (
	LivePageQueued     LivePageStatus = "queued"
	LivePageProcessing LivePageStatus = "processing"
	LivePageSucceeded  LivePageStatus = "succeeded"
	LivePageFailed     LivePageStatus = "failed"
)

type ProcessType

type ProcessType string

Allowed process type

const (
	GENERATE             ProcessType = "generate"
	UPSCALE              ProcessType = "upscale"
	GENERATE_AND_UPSCALE ProcessType = "generate_and_upscale"
	VOICEOVER            ProcessType = "voiceover"
)

type SyncMap

type SyncMap[T any] struct {
	// contains filtered or unexported fields
}

SyncMap builds an thread-safe map

func NewSyncMap

func NewSyncMap[T any]() *SyncMap[T]

func (*SyncMap[T]) Delete

func (r *SyncMap[T]) Delete(key string)

Removes specified hash - synchronized

func (*SyncMap[T]) Exists

func (r *SyncMap[T]) Exists(key string) bool

See if element exists

func (*SyncMap[T]) Get

func (r *SyncMap[T]) Get(key string) T

Gets a value from the map - synchronized

func (*SyncMap[T]) GetAll

func (r *SyncMap[T]) GetAll() map[string]T

Get all keys and values

func (*SyncMap[T]) Put

func (r *SyncMap[T]) Put(key string, value T)

Put value into map - synchronized

type UserQueueThrottlerMap

type UserQueueThrottlerMap struct {
	// contains filtered or unexported fields
}

UserQueueThrottlerMap builds an thread-safe map For number of items enqueued by this user

func NewQueueThrottler

func NewQueueThrottler(ctx context.Context, redis *redis.Client, ttl time.Duration) *UserQueueThrottlerMap

func (*UserQueueThrottlerMap) DecrementBy

func (r *UserQueueThrottlerMap) DecrementBy(amount int, userID string) error

Decrement the number of items in queue for this user, minimum 0

func (*UserQueueThrottlerMap) IncrementBy

func (r *UserQueueThrottlerMap) IncrementBy(amount int, userID string) error

Increment the number of items in queue for this user

func (*UserQueueThrottlerMap) NumQueued

func (r *UserQueueThrottlerMap) NumQueued(userID string) (int, error)

Get the number of items in queue for this user

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL