A spatial action-policy model

Variational decision making

Targets provide evidence about which way to move. At every position, their bearings update a directional posterior, a single von Mises distribution approximates it, and that distribution becomes the agent’s next movement policy.

Position → target bearings → angular posterior → von Mises policy → next position

Drag targets · arrow keys also move them

Current modelForward-KL comparisonCurrent target bearings

Time 0.00 s

Computing spatial trajectories…

The policy driving this agent

Select a run and scrub time to inspect its local update.

Run

Angular posterior pAction policy qPrior π

Policy direction —Concentration κ —Expected speed —

Example Divergence Targets

Initial target separation 60°

Resets the target fan at equal starting distances.

Angular evidence κₑ 14

Evidence / utility weight β 1.00

Previous-policy retention ρ 0.00 · uniform prior

πₜ ∝ qₜ₋₁^ρ. At zero, every local update starts from a uniform prior.

Position update

Move by the policy’s expected velocity. A broad policy can slow or stall the agent between targets.

Number of runs

Initial variation

Lateral starting spread 0.025

A seeded, small positional variation breaks exact spatial symmetry. It is separate from sampled action noise.

Preparing the spatial model…

How this spatial model works

Geometry supplies angular evidence at every step

For an agent at xₜ and target gᵢ, the target bearing is θᵢ(xₜ) = atan2(gᵢ,y − xₜ,y, gᵢ,x − xₜ,x). Equally weighted alternative targets give Lₜ(θ) = (1/M) ∑ᵢ VM(θ; θᵢ(xₜ), κₑ). Targets do not contribute a fixed angle: the evidence changes whenever the agent moves. It is a mixture of alternative directions, not a product requiring every target to be simultaneously correct.

The local posterior is pₜ(θ) ∝ πₜ(θ)Lₜ(θ)^β. The prior is a tempered copy of the previous action policy, πₜ ∝ qₜ₋₁^ρ, with an initially uniform policy. At ρ = 0 the update is memoryless; positive retention introduces directional persistence. This is an exploratory local evidence-update rule, not a claim of calibrated independent sensory observations or optimal long-horizon planning.

The approximation is the action policy

The agent fits qₜ(θ) = VM(θ; μₜ, κₜ). Reverse KL minimizes KL(qₜ ∥ pₜ); forward KL minimizes KL(pₜ ∥ qₜ). Jensen–Shannon and squared Hellinger are also available. The unimodal constraint can turn a multimodal angular posterior into a choice of direction. With forward KL, moment matching can instead retain a compromise as the targets spread apart in the agent’s visual field.

For Sample heading, draw aₜ ∼ qₜ and update xₜ₊₁ = xₜ + v Δt (cos aₜ, sin aₜ). For Expected step, update xₜ₊₁ = xₜ + v Δt (E_qₜ cos θ, E_qₜ sin θ). The latter includes the mean resultant length: it does not move a uniform policy in an arbitrary direction. Here v = 0.65 and Δt = 0.04. Playback speed and scrubbing only display the precomputed trajectories; they do not change this scientific update.

The default two-target, uniform-prior example can therefore approach both alternatives before reverse KL selects a branch, while forward-KL expected motion approaches the intervening segment and slows. These are outcomes of this particular local policy and geometry, not universal guarantees about divergence choices. Initial lateral offsets are seeded; tied policy minima use a seeded tie-break when previous direction cannot distinguish them.

Utility + prior

This mode fits the action policy directly by minimizing −β E_qₜ[Uₜ(θ)] + λ D(qₜ, πₜ). Log angular evidence uses Uₜ = log Lₜ. Mean progress uses the average cos(θ − θᵢ(xₜ)); best progress uses their maximum. Both are local directional progress utilities, not complete future-path utilities. The standard KL(q ∥ π) penalty gives the Gibbs/generalized-Bayes update rₜ ∝ πₜ exp(βUₜ/λ). With log evidence and λ = 1, it agrees with reverse-KL Bayesian projection.

Other prior penalties define different regularized decision rules and are not claimed to have the sequential coherence of the standard generalized-Bayes loss update. Their scales differ, so equal λ does not mean equal penalty strength. Squared Hellinger is ½ ∫(√q − √π)²; Jensen–Shannon uses equal mixture weights and natural logarithms. The inspector shows the unrestricted angular-grid optimum in utility mode, alongside the constrained action policy.

Numerical scope and reproducibility

Angular integration uses 96 periodic nodes. Forward-KL posterior projection uses von Mises moment matching. Other objectives use multi-start natural-gradient optimization, seeded by the previous policy, the covering fit, a uniform policy and each target direction. The concentration is capped at κ = 40; bound hits are reported. This is an approximate search, not a guarantee of the global optimum at every step. Action sampling uses a von Mises rejection sampler rather than snapping actions to angular bins.

Each run stops on entry to a target’s radius-0.075 disk, after 220 updates, or on leaving the displayed spatial domain. A run that has not reached a target remains unresolved. The final arrival frame holds the action policy that produced the arrival; the inspector identifies the position at which that policy was evaluated. The forward-KL overlay uses the same starting positions and seed. Download data includes the targets, settings and every policy/position update. The original decision-making model is separate and unchanged.

Background: Blei, Kucukelbir & McAuliffe, Variational Inference; Bissiri, Holmes & Walker, General Bayesian updating.