Documentation
¶
Overview ¶
Package armaze represents an N-armed maze ("bandit") with each Arm having a distinctive CS stimulus at the start (could be one of multiple possibilities) and (some probability of) a US outcome at the end of the maze (could be either positive or negative, with (variable) magnitude and probability.
The maze can have open or closed arms -- open arms allow switching to a neighboring arm anytime, while closed arms only allow switching at the start.
Index ¶
- func MinMaxRand(mm minmax.F32, rand erand.SysRand) float32
- type Actions
- func (i Actions) Desc() string
- func (i Actions) Int64() int64
- func (i Actions) MarshalText() ([]byte, error)
- func (i *Actions) SetInt64(in int64)
- func (i *Actions) SetString(s string) error
- func (i Actions) String() string
- func (i *Actions) UnmarshalText(text []byte) error
- func (i Actions) Values() []enums.Enum
- type Arm
- type Config
- type Env
- func (ev *Env) Action(action string, nop etensor.Tensor)
- func (ev *Env) ArmIsMaxUtil(arm int) bool
- func (ev *Env) ArmIsMaxValue(arm int) bool
- func (ev *Env) ArmIsNegative(arm int) bool
- func (ev *Env) ConfigApproach()
- func (ev *Env) ConfigEnv(di int)
- func (ev *Env) ConsumeEffort() float32
- func (ev *Env) ConsumeUS(arm *Arm)
- func (ev *Env) Counter(scale env.TimeScales) (cur, prv int, changed bool)
- func (ev *Env) CurArm() *Arm
- func (ev *Env) CurCS() int
- func (ev *Env) DecodeAct(vt *etensor.Float32) Actions
- func (ev *Env) DecodeLocalist(vt *etensor.Float32) int
- func (ev *Env) Defaults()
- func (ev *Env) Desc() string
- func (ev *Env) ExValueUtil(pv *axon.Rubicon, ctx *axon.Context)
- func (ev *Env) ForwardEffort(arm *Arm) float32
- func (ev *Env) InactiveValue() float32
- func (ev *Env) Init(run int)
- func (ev *Env) InstinctAct() Actions
- func (ev *Env) Name() string
- func (ev *Env) NewStart()
- func (ev *Env) RenderAction(act Actions)
- func (ev *Env) RenderLocalist(name string, val int)
- func (ev *Env) RenderLocalist4D(name string, val int)
- func (ev *Env) RenderState()
- func (ev *Env) StartApproach()
- func (ev *Env) State(el string) etensor.Tensor
- func (ev *Env) Step() bool
- func (ev *Env) StepApproach()
- func (ev *Env) TakeAct(act Actions)
- func (ev *Env) TurnEffort() float32
- func (ev *Env) UpdateMaxLength()
- func (ev *Env) Validate() error
- type GUI
- func (vw *GUI) AddFloor(par *eve.Group, name string) *eve.Group
- func (vw *GUI) ConfigArms(par *eve.Group) *eve.Group
- func (vw *GUI) ConfigEmery(par *eve.Group, length float32) *eve.Group
- func (vw *GUI) ConfigStims(par *eve.Group, name string, width, height float32) *eve.Group
- func (vw *GUI) ConfigUSPlots()
- func (vw *GUI) ConfigView3D(se *xyz.Scene)
- func (vw *GUI) ConfigWorld()
- func (vw *GUI) ConfigWorldGUI(ev *Env) *core.Body
- func (vw *GUI) ConfigWorldView(tg *etview.TensorGrid)
- func (vw *GUI) Consume()
- func (vw *GUI) Forward()
- func (vw *GUI) GrabEyeImg()
- func (vw *GUI) Left()
- func (vw *GUI) Right()
- func (vw *GUI) SetEmeryPose()
- func (vw *GUI) UpdateStims()
- func (vw *GUI) UpdateWorld(ctx *axon.Context, ev *Env, net *axon.Network, state TraceStates)
- func (vw *GUI) UpdateWorldGUI()
- func (vw *GUI) ViewDepth(depth []float32)
- type Geom
- type Paradigms
- func (i Paradigms) Desc() string
- func (i Paradigms) Int64() int64
- func (i Paradigms) MarshalText() ([]byte, error)
- func (i *Paradigms) SetInt64(in int64)
- func (i *Paradigms) SetString(s string) error
- func (i Paradigms) String() string
- func (i *Paradigms) UnmarshalText(text []byte) error
- func (i Paradigms) Values() []enums.Enum
- type Params
- type StateTrace
- type TraceRec
- type TraceStates
- func (i TraceStates) Desc() string
- func (i TraceStates) Int64() int64
- func (i TraceStates) MarshalText() ([]byte, error)
- func (i *TraceStates) SetInt64(in int64)
- func (i *TraceStates) SetString(s string) error
- func (i TraceStates) String() string
- func (i *TraceStates) UnmarshalText(text []byte) error
- func (i TraceStates) Values() []enums.Enum
- type USParams
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Actions ¶
type Actions int32 //enums:enum
Actions is a list of mutually exclusive states for tracing the behavior and internal state of Emery
const ActionsN Actions = 5
ActionsN is the highest valid value for type Actions, plus one.
func ActionsValues ¶
func ActionsValues() []Actions
ActionsValues returns all possible values for the type Actions.
func (Actions) MarshalText ¶
MarshalText implements the encoding.TextMarshaler interface.
func (*Actions) SetString ¶
SetString sets the Actions value from its string representation, and returns an error if the string is invalid.
func (*Actions) UnmarshalText ¶
UnmarshalText implements the encoding.TextUnmarshaler interface.
type Arm ¶
type Arm struct { // length of arm: distance from CS start to US end for this arm Length int // range of different effort levels per step (uniformly randomly sampled per step) for going down this arm Effort minmax.F32 // index of US present at the end of this arm -- -1 if none US int // index of CS visible at the start of this arm, -1 if none CS int // current expected value = US.Prob * US.Mag * Drives-- computed at start of new approach ExValue float32 `edit:"-"` // current expected PVpos value = normalized ExValue -- computed at start of new approach ExPVpos float32 `edit:"-"` // current expected PVneg value = normalized time and effort costs ExPVneg float32 `edit:"-"` // current expected utility = effort discounted version of ExPVpos -- computed at start of new approach ExUtil float32 `edit:"-"` }
Arm represents the properties of a given arm of the N-maze. Arms have characteristic distance and effort factors for getting down the arm, and typically have a distinctive CS visible at the start and a US at the end, which has US-specific parameters for actually delivering reward or punishment.
type Config ¶
type Config struct { // experimental paradigm that governs the configuration and updating of environment state over time and the appropriate evaluation criteria. Paradigm Paradigms // for debugging, print out key steps including a trace of the action generation logic Debug bool // number of different drive-like body states (hunger, thirst, etc), that are satisfied by a corresponding positive US outcome -- this does not include the first curiosity drive NDrives int // number of negative US outcomes -- these are added after NDrives positive USs to total US list NNegUSs int // total number of USs = NDrives + NNegUSs NUSs int `edit:"-"` // number of different arms NArms int // maximum arm length (distance) MaxArmLength int // number of different CSs -- typically at least a unique CS per US -- relationship is determined in the US params NCSs int // parameters associated with each US. The first NDrives are positive USs, and beyond that are negative USs USs []*USParams // state of each arm: dist, effort, US, CS Arms []*Arm // misc params Params Params `view:"add-fields"` }
Config has environment configuration
type Env ¶
type Env struct { // name of environment -- Train or Test Nm string // our data parallel index Di int `edit:"-"` // configuration parameters Config Config // current drive strength for each of Config.NDrives in normalized 0-1 units of each drive: 0 = first sim drive, not curiosity Drives []float32 // arm-wise location: either facing (Pos=0) or in (Pos > 0) Arm int `edit:"-"` // current position in the Arm: 0 = at start looking in, otherwise at given distance into the arm Pos int `edit:"-"` // current integer time step since last NewStart Tick int `edit:"-"` // current target drive, in paradigms where that is used (e.g., Approach) TrgDrive int `edit:"-"` // Current US being consumed -- is -1 unless being consumed USConsumed int `edit:"-"` // reward or punishment value generated by the current US being consumed -- just the Magnitude of the US -- does NOT include any modulation by Drive USValue float32 `edit:"-"` // just finished consuming a US -- ready to start doing something new JustConsumed bool `edit:"-"` // arm(s) with maximum Drive * Mag * Prob US outcomes ArmsMaxValue []int `edit:"-"` // maximum value for ArmsMaxValue arms MaxValue float32 `edit:"-"` // arm(s) with maximum Value outcome discounted by Effort ArmsMaxUtil []int `edit:"-"` // maximum utility for ArmsMaxUtil arms MaxUtil float32 `edit:"-"` // arm(s) with negative US outcomes ArmsNeg []int `edit:"-"` // last action taken LastAct Actions `edit:"-"` // effort on current trial Effort float32 `edit:"-"` // last CS seen LastCS int `edit:"-"` // last US -- previous trial LastUS int `edit:"-"` // just gated on this trial -- set by sim-- used for instinct JustGated bool `edit:"-"` // has gated at some point during sequence -- set by sim -- used for instinct HasGated bool `edit:"-"` // named states -- e.g., USs, CSs, etc States map[string]*etensor.Float32 // maximum length of any arm MaxLength int `edit:"-"` // random number generator for the env -- all random calls must use this Rand erand.SysRand `view:"-"` // random seed RndSeed int64 `edit:"-"` }
Env implements an N-armed maze ("bandit") with each Arm having a distinctive CS stimulus visible at the start (could be one of multiple possibilities) and (some probability of) a US outcome at the end of the maze (could be either positive or negative, with (variable) magnitude and probability.
func (*Env) Action ¶
Action records the LastAct and renders it, but does not update the state accordingly.
func (*Env) ArmIsMaxUtil ¶
ArmIsMaxUtil returns true if the given arm is (one of) the arms with the best current expected outcome utility
func (*Env) ArmIsMaxValue ¶
ArmIsMaxValue returns true if the given arm is (one of) the arms with the best current expected outcome value
func (*Env) ArmIsNegative ¶
ArmIsNegative returns true if the given arm is (one of) the arms with negative outcomes
func (*Env) ConfigApproach ¶
func (ev *Env) ConfigApproach()
ConfigApproach does initial config for Approach paradigm
func (*Env) ConfigEnv ¶
ConfigEnv configures the environment. additional parameterization via specific configs is applied after this step, which initializes everything according to basic Ns takes the data parallel index di
func (*Env) ConsumeEffort ¶
ConsumeEffort returns a new random Effort value from Config.Params.ConsumeEffort param range.
func (*Env) ForwardEffort ¶
ForwardEffort returns a new random Effort value from Arm Effort range
func (*Env) InactiveValue ¶
InactiveVal returns a new random inactive value from Config.Params.Inactive param range.
func (*Env) Init ¶
Init does updating preparing to run -- params could have changed since initial config so updates everything except broad overall config stuff.
func (*Env) InstinctAct ¶
InstinctAct returns an "instinctive" action that implements a basic policy
func (*Env) RenderLocalist ¶
RenderLocalist renders one localist state
func (*Env) RenderLocalist4D ¶
RenderLocalist4D renders one localist state in 4D
func (*Env) StartApproach ¶
func (ev *Env) StartApproach()
StartApproach does new start state setting for Approach Selects a new TrgDrive at random, sets that to 1, others to inactive levels
func (*Env) StepApproach ¶
func (ev *Env) StepApproach()
func (*Env) TurnEffort ¶
TurnEffort returns a new random Effort value from Config.Params.TurnEffort param range.
func (*Env) UpdateMaxLength ¶
func (ev *Env) UpdateMaxLength()
type GUI ¶
type GUI struct { // update display -- turn off to make it faster Disp bool // the env being visualized Env *Env // name of current env -- number is NData index EnvName string // 3D visualization of the Scene SceneView *xyzview.SceneView // 2D visualization of the Scene Scene2D *core.SVG // list of material colors MatColors []string // internal state colors StateColors map[string]string // thickness (X) and height (Y) of walls WallSize math32.Vector2 // current internal / behavioral state State TraceStates // trace record of recent activity Trace StateTrace // view of the gui obj StructView *views.StructView `view:"-"` // ArmMaze TabView WorldTabs *core.Tabs `view:"-"` // ArmMaze is running IsRunning bool `view:"-"` // current depth map DepthValues []float32 // offscreen render camera settings Camera evev.Camera // color map to use for rendering depth map DepthMap views.ColorMapName // first-person right-eye full field view EyeRFullImage *core.Image `view:"-"` // first-person right-eye fovea view EyeRFovImage *core.Image `view:"-"` // depth map bitmap view DepthImage *core.Image `view:"-"` // plot of positive valence drives, active OFC US state, and reward USposPlot *eplot.Plot2D // data for USPlot USposData *etable.Table // plot of negative valence active OFC US state, and outcomes USnegPlot *eplot.Plot2D // data for USPlot USnegData *etable.Table // geometry of world Geom Geom // world World *eve.Group // 3D view of world View3D *evev.View `view:"-"` // emer group Emery *eve.Group `view:"-"` // arms group Arms *eve.Group `view:"-"` // stims group Stims *eve.Group `view:"-"` // Right eye of emery EyeR eve.Body `view:"-"` // contacts from last step, for body Contacts eve.Contacts `view:"-"` }
GUI renders multiple views of the flat world env
func (*GUI) ConfigArms ¶
ConfigArms adds all the arms
func (*GUI) ConfigEmery ¶
ConfigEmery constructs a new Emery virtual hamster
func (*GUI) ConfigStims ¶
ConfigStims constructs stimuli: CSs, USs
func (*GUI) ConfigUSPlots ¶
func (vw *GUI) ConfigUSPlots()
func (*GUI) ConfigView3D ¶
ConfigView3D makes the 3D view
func (*GUI) ConfigWorld ¶
func (vw *GUI) ConfigWorld()
ConfigWorld constructs a new virtual physics world for flat world
func (*GUI) ConfigWorldGUI ¶
ConfigWorldGUI configures all the world view GUI elements pass an initial env to use for configuring
func (*GUI) ConfigWorldView ¶
func (vw *GUI) ConfigWorldView(tg *etview.TensorGrid)
func (*GUI) GrabEyeImg ¶
func (vw *GUI) GrabEyeImg()
GrabEyeImg takes a snapshot from the perspective of Emer's right eye
func (*GUI) SetEmeryPose ¶
func (vw *GUI) SetEmeryPose()
func (*GUI) UpdateStims ¶
func (vw *GUI) UpdateStims()
func (*GUI) UpdateWorld ¶
func (*GUI) UpdateWorldGUI ¶
func (vw *GUI) UpdateWorldGUI()
type Geom ¶
type Geom struct { // width of arm -- emery rodent is 1 unit wide ArmWidth float32 `default:"2"` // total space between arms, ends up being divided on either side ArmSpace float32 `default:"1"` // multiplier per unit arm length -- keep square with width LengthScale float32 `default:"2"` // thickness of walls, floor Thick float32 `default:"0.1"` // height of walls Height float32 `default:"0.2"` // width + space ArmWidthTot float32 `edit:"-"` // computed total depth, starts at 0 goes deep Depth float32 `edit:"-"` // computed total width Width float32 `edit:"-"` // half width for centering on 0 X HalfWidth float32 `edit:"-"` }
Geom is overall geometry of the space
type Paradigms ¶
type Paradigms int32 //enums:enum
Paradigms is a list of experimental paradigms that govern the configuration and updating of environment state over time and the appropriate evaluation criteria.
const ( // Approach is a basic case where one Drive (chosen at random each trial) is fully active and others are at InactiveDrives levels -- goal is to approach the CS associated with the Drive-satisfying US, and avoid negative any negative USs. USs are always placed in same Arms (NArms must be >= NUSs -- any additional Arms are filled at random with additional US copies) Approach Paradigms = iota )
const ParadigmsN Paradigms = 1
ParadigmsN is the highest valid value for type Paradigms, plus one.
func ParadigmsValues ¶
func ParadigmsValues() []Paradigms
ParadigmsValues returns all possible values for the type Paradigms.
func (Paradigms) MarshalText ¶
MarshalText implements the encoding.TextMarshaler interface.
func (*Paradigms) SetString ¶
SetString sets the Paradigms value from its string representation, and returns an error if the string is invalid.
func (*Paradigms) UnmarshalText ¶
UnmarshalText implements the encoding.TextUnmarshaler interface.
type Params ¶
type Params struct { // effort for turning TurnEffort minmax.F32 `nest:"+" default:"{'Min':0.5, 'Max':0.5}"` // effort for consuming US ConsumeEffort minmax.F32 `nest:"+" default:"{'Min':0.5, 'Max':0.5}"` // permute the order of CSs prior to applying them to arms -- having this off makes it easier to visually determine match between Drive and arm approach, and shouldn't make any difference to behavior (model doesn't know about this ordering). PermuteCSs bool `default:"false"` // after running down an Arm, a new random starting location is selected (otherwise same arm as last run) RandomStart bool `default:"true"` // if true, allow movement between arms just by going Left or Right -- otherwise once past the start, no switching is allowed OpenArms bool `default:"true"` // strength of inactive inputs (e.g., Drives in Approach paradigm) Inactive minmax.F32 `nest:"+" default:"{'Min':0, 'Max':0}" view:"inline"` // number of Y-axis repetitions of localist stimuli -- for redundancy in spiking nets NYReps int `default:"4"` }
Params are misc environment parameters
type TraceRec ¶
type TraceRec struct { // absolute time Time float32 // trial counter Trial int // current arm Arm int // position in arm Pos int // behavioral / internal state summary State TraceStates // NDrives current drive state level Drives []float32 }
TraceRec holds record of info for tracing behavior, state
type TraceStates ¶
type TraceStates int32 //enums:enum
TraceStates is a list of mutually exclusive states for tracing the behavior and internal state of Emery
const ( // Searching is not yet goal engaged, looking for a goal TrSearching TraceStates = iota // Deciding is having some partial gating but not in time for action TrDeciding // JustEngaged means just decided to engage in a goal TrJustEngaged // Approaching is goal engaged, approaching the goal TrApproaching // Consuming is consuming the US, first step (prior to getting reward, step1) TrConsuming // Rewarded is just received reward from a US TrRewarded // GiveUp is when goal is abandoned TrGiveUp // Bumping is bumping into a wall TrBumping )
const TraceStatesN TraceStates = 8
TraceStatesN is the highest valid value for type TraceStates, plus one.
func TraceStatesValues ¶
func TraceStatesValues() []TraceStates
TraceStatesValues returns all possible values for the type TraceStates.
func (TraceStates) Desc ¶
func (i TraceStates) Desc() string
Desc returns the description of the TraceStates value.
func (TraceStates) Int64 ¶
func (i TraceStates) Int64() int64
Int64 returns the TraceStates value as an int64.
func (TraceStates) MarshalText ¶
func (i TraceStates) MarshalText() ([]byte, error)
MarshalText implements the encoding.TextMarshaler interface.
func (*TraceStates) SetInt64 ¶
func (i *TraceStates) SetInt64(in int64)
SetInt64 sets the TraceStates value from an int64.
func (*TraceStates) SetString ¶
func (i *TraceStates) SetString(s string) error
SetString sets the TraceStates value from its string representation, and returns an error if the string is invalid.
func (TraceStates) String ¶
func (i TraceStates) String() string
String returns the string representation of this TraceStates value.
func (*TraceStates) UnmarshalText ¶
func (i *TraceStates) UnmarshalText(text []byte) error
UnmarshalText implements the encoding.TextUnmarshaler interface.
func (TraceStates) Values ¶
func (i TraceStates) Values() []enums.Enum
Values returns all possible values for the type TraceStates.