Dendritic Delays in Self-Organizing Recurrent Networks

Introduction

Recurrent neural networks process temporal information by allowing past activity to influence future network states [1]. However, recurrent feedback can also produce excessive synchronization or rigid activity patterns, limiting the network’s ability to represent different input histories [2]. Here, we ask whether distributing recurrent feedback across delayed dendritic pathways can support longer memory and richer population dynamics.

Biological dendrites integrate signals arriving at different locations and times [4], while delayed interactions can influence synchronization and collective network dynamics [5]. We therefore introduced delayed recurrent integration into a self-organizing spiking neural network and compared it with the standard architecture.

Methods

Network

The self-organizing recurrent neural network (SORN) develops temporal representations through local plasticity [3]. STDP adapts recurrent connections, intrinsic plasticity regulates neuronal excitability, and synaptic normalization constrains recurrent weights.

We compared a Base SORN, in which recurrent activity is integrated in the next timestep, with a Delayed SORN that distributes recurrent signals across dendritic delay pathways. Each pathway uses delay-specific STDP, while all other plasticity mechanisms remain unchanged.

Figure 1: In the Base SORN, recurrent activity is integrated in the next timestep. In the Delayed SORN, recurrent signals are distributed across pathways with different delays and adapted through delay-specific STDP. — In the Base SORN, recurrent activity is integrated in the next timestep. In the Delayed SORN, recurrent signals are distributed across pathways with different delays and adapted through delay-specific STDP.

Both networks contained 100 excitatory neurons, including 20 neurons receiving external input. Apart from the delayed recurrent pathways, the models were trained and evaluated under matched conditions.

Stimuli

Ten symbols were encoded as unique 7-of-20 sparse binary patterns and presented every 10 timesteps. The intervening timesteps contained equally sparse random filler patterns. Successive symbols were independent, meaning that the Markov0 input contained no temporal dependencies that could be exploited to predict the next symbol.

Figure 2: Symbols were presented every ten timesteps, with sparsity-matched random filler patterns in between. — Symbols were presented every ten timesteps, with sparsity-matched random filler patterns in between.

After training, plasticity was frozen for evaluation. We examined three aspects of the network dynamics: how long stimulus information remained decodable, how population-level dimensionality developed over time, and how strongly perturbations altered subsequent activity.

Results

Memory: How long does stimulus information remain available?

To quantify memory, a separate linear decoder at each timestep predicted the identity of the most recently presented symbol from the excitatory population activity. Chance accuracy was 0.1 because the input consisted of ten symbol classes.

Figure 3: Decoding of the most recently presented stimulus from network activity. The dotted line indicates chance level. — Decoding of the most recently presented stimulus from network activity. The dotted line indicates chance level.

In the Base SORN, decoding accuracy decreased rapidly and soon approached chance. In the Delayed SORN, stimulus identity remained decodable for substantially longer. The delayed pathways therefore did more than sustain activity: they prolonged stimulus-specific information that could be accessed by a simple linear readout.

Dimensionality: How rich are the population dynamics?

If network activity collapses onto only a few dominant patterns, different inputs and input histories become difficult to distinguish. We therefore applied principal component analysis to consecutive windows of population activity and measured how many principal components were required to explain 80% of the variance.

Figure 4: Time-resolved PCA of population activity in the Base and Delayed SORN. — Time-resolved PCA of population activity in the Base and Delayed SORN.

The Delayed SORN consistently exhibited higher dimensionality than the Base SORN. Its activity was therefore distributed across more population-level directions, indicating richer and more distributed representations. PCA alone does not show whether those dimensions are task-relevant, but the decoding result demonstrates that the delayed network’s activity also carries more accessible information about recent stimuli.

Perturbations: Do trajectories collapse or remain distinct?

We created perturbed and unperturbed copies of the same frozen network state, supplied both with the same subsequent input, and flipped the activity of different numbers of excitatory neurons in the perturbed copy. We then followed the normalized Hamming distance between the two trajectories.

Figure 5: Normalized Hamming distance between perturbed and unperturbed trajectories following state perturbations of different strengths. — Normalized Hamming distance between perturbed and unperturbed trajectories following state perturbations of different strengths.

The Base SORN rapidly returned to the exact unperturbed trajectory. In the Delayed SORN, perturbations decayed strongly but left a bounded trace rather than disappearing completely. This intermediate response may be useful for temporal processing: activity does not diverge uncontrollably, but differences between recent trajectories are not erased immediately.

Conclusion

Delayed recurrent integration preserved stimulus information over longer timescales and increased the dimensionality of population activity. Perturbations also left bounded traces, suggesting flexible dynamics without immediate convergence to a fixed activity trajectory. Together, these results indicate that dendritic delays can serve as a computational resource by distributing recurrent feedback across time.

References

Maass, W., Natschläger, T., & Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531–2560. https://doi.org/10.1162/089976602760407955

Del Papa, B., Priesemann, V., & Triesch, J. (2017). Criticality meets learning: Criticality signatures in a self-organizing recurrent neural network. PLOS ONE, 12(5), e0178683. https://doi.org/10.1371/journal.pone.0178683

Lazar, A., Pipa, G., & Triesch, J. (2009). SORN: A self-organizing recurrent neural network. Frontiers in Computational Neuroscience, 3, 23. https://doi.org/10.3389/neuro.10.023.2009

London, M., & Häusser, M. (2005). Dendritic computation. Annual Review of Neuroscience, 28, 503–532. https://doi.org/10.1146/annurev.neuro.28.061604.135703

Pérez, T., Garcia, G. C., Eguíluz, V. M., Vicente, R., Pipa, G., & Mirasso, C. (2011). Effect of the topology and delayed interactions in neuronal networks synchronization. PLOS ONE, 6(5), e19900. https://doi.org/10.1371/journal.pone.0019900