Leveraging Dale’s Principle as an Inductive Bias in Recurrent Neural Networks

Sun, 27 Sep 2026 00:00:00 +0000

A biological principle for more stable learning

Biological neurons send either excitatory or inhibitory outputs. EISep RNN incorporates this constraint, known as Dale’s principle, into recurrent networks while allowing each neuron’s excitatory or inhibitory identity to be learned.

Method

The architecture combines a learnable sign assignment using Gumbel–Softmax. These choices let the network adapt its excitatory–inhibitory organization during training while improving optimization stability.

Experimental findings

Multi-task learning: competitive performance on a 20-task cognitive benchmark, with more stable gradients and recurrent dynamics.
Continual learning: reduced task interference and stronger resistance to catastrophic forgetting across sequential cognitive tasks.
Reinforcement learning: improved recurrent advantage actor–critic performance on Lunar Lander and HalfCheetah, covering discrete and continuous control.
Interpretable structure: sparse, modular connectivity emerges from the excitatory–inhibitory constraint.

Research connection

Reliable policy learning depends on stable optimization and representations that can be reused across tasks. This work explores architectural inductive biases for those properties.

Continual Learning | Jiyi Wang

Leveraging Dale’s Principle as an Inductive Bias in Recurrent Neural Networks

A biological principle for more stable learning

Method

Experimental findings

Research connection