A Brain-Inspired Reinforcement Learning Mechanism for Pattern Recognition

Milad MOZAFARI
PhD, Research Scientist at Torus AI
November 23rd, 2023

Abstract
The reinforcement learning (RL) mechanism in the mammalian brain, characterized by learning from experience, addresses decision-making challenges in various situations. Visual object recognition, involving decisions on visual stimuli categories, starts at the retina in the back of the eyes, where visual inputs become spike patterns. These patterns climb the visual cortex hierarchy, culminating in the Inferior Temporal cortex, rich enough to represent complete objects. We demonstrate that bio-inspired RL can be efficiently used to train spiking neural networks (SNNs) for object recognition in natural images without external supervised classifiers. We use a feed-forward convolutional SNN with a time-to-first-spike encoding scheme. Neurons in the decision layer are associated with object categories and the neuron with the earliest spike time or highest membrane potential signals the decision of the network. Correct decisions reinforce connections via spike-timing-dependent plasticity (STDP), while wrong decisions trigger anti-STDP to reset weights. The proposed reward-modulated STDP excels in extracting visual features and outperforms classic unsupervised STDP with supervised classifiers on various image datasets.

Supplementary Materials
Papers:
- https://ieeexplore.ieee.org/abstract/document/8356226
- https://www.sciencedirect.com/science/article/abs/pii/S0031320319301906
- https://www.frontiersin.org/articles/10.3389/fnins.2019.00625/full?ref=https://githubhelp.com
Code:
- https://github.com/miladmozafari/SpykeTorch