Inverse Reinforcement Learning with Switching Rewards and History Dependency for Characterizing Animal Behaviors

Wed, 01 Jan 2025 00:00:00 +0000

Inferring the intentions behind behavior

SWIRL (Switching Inverse Reinforcement Learning) infers changing goals from behavioral trajectories. It models long sequences as transitions between hidden decision-making modes, each with its own reward function and policy.

Method

Goal switching: transitions between hidden modes depend on the previous mode and the animal’s current state.
Behavioral memory: rewards and policies can depend on recent states, capturing how past experience shapes actions.
Joint inference: expectation–maximization alternates between inferring behavioral modes and learning their transitions, rewards, and policies.

Experimental findings

Gridworld: combining both levels of history dependence improves reward recovery and behavioral segmentation.
Mouse navigation: recovers interpretable water-seeking, home-seeking, and exploration modes in a 127-node labyrinth.
Spontaneous behavior: outperforms autoregressive HMM baselines in held-out likelihood. State-dependent switching helps, while adding action-level history does not improve this dataset.

Research connection

This work connects learning from demonstrations with interpretable reward inference, offering a way to study changing intentions under the complicated behavior.

Reward Inference | Jiyi Wang

Inverse Reinforcement Learning with Switching Rewards and History Dependency for Characterizing Animal Behaviors

Inferring the intentions behind behavior

Method

Experimental findings

Research connection