Enhancement of hippocampal spatial decoding using a dynamic Q-learning method with a relative reward using theta phase precession

Main Authors: Chen, Bo-Wei, Yang, Shih-Hung, Lo, Yu-Chun, Wang, Ching-Fu, Wang, Han-Lin, Hsu, Chen-Yang, Kuo, Yung-Ting, Chen, Jung-Chen, Lin, Sheng-Huang, Pan, Han-Chi, Lee, Sheng-Wei, Yu, Xiao, Qu, Boyi, Kuo, Chao-Hung, Chen, You-Yin, Lai, Hsin-Yi
Format: info software eJournal
Terbitan: , 2020
Online Access: https://zenodo.org/record/3724076
ctrlnum 3724076
fullrecord <?xml version="1.0"?> <dc schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd"><creator>Chen, Bo-Wei</creator><creator>Yang, Shih-Hung</creator><creator>Lo, Yu-Chun</creator><creator>Wang, Ching-Fu</creator><creator>Wang, Han-Lin</creator><creator>Hsu, Chen-Yang</creator><creator>Kuo, Yung-Ting</creator><creator>Chen, Jung-Chen</creator><creator>Lin, Sheng-Huang</creator><creator>Pan, Han-Chi</creator><creator>Lee, Sheng-Wei</creator><creator>Yu, Xiao</creator><creator>Qu, Boyi</creator><creator>Kuo, Chao-Hung</creator><creator>Chen, You-Yin</creator><creator>Lai, Hsin-Yi</creator><date>2020-03-23</date><description> Winners of the 2014 Nobel Prize in Physiology or Medicine, Professors John O&#x2019;Keefe, May&#x2010;Britt Moser and Edvard I. Moser found that the internal global positioning system (GPS) in the brain allows us to be able to flexibly navigate the world they live in &#x2013; exploring new areas, returning quickly to remembered places, and taking shortcuts and confirmed that place cells in hippocampus and grid cells in entorhinal cortex (EC) are responsible for higher-order cognitive map of the environment. Indeed, these abilities feel so easy and natural that it is not immediately obvious how complex the underlying processes really are. In contrast, spatial navigation remains a substantial challenge for artificial agents whose abilities are far outstripped by those of mammals. Hippocampal place cells and interneurons in mammals have proved that they own stable place fields and theta phase precession profiles to encode the spatial information from the environment. The hippocampal CA1 neurons can be represented as the location of the animal and the prospective information of goal location. Reinforcement learning algorithm, e.g., Q-learning, has been adopted to build a navigation model of place cells for the purpose of addressing goal direction navigation problems. In this study, we propose dynamical Q-learning (dQ-learning), because of its adaptive reward function based on theta phase precession, which has recently been associated with a rat&#x2019;s experiences at destinations, and use of information from both place cells and interneurons as inputs to predict the animal&#x2019;s trajectory. We evaluated the convergence rates and learning performances of tQ-learning and dQ-learning with different cell types. The results demonstrate that dQ-learning improves learning performance and convergence rate and place cells and interneurons with phase precession may provide valuable information to improve the prediction of trajectory. To investigate whether the enhancement of hippocampal spatial decoding with the dQ-learning method was effective in goal-direction navigation, experimental data were recorded from rats implanted with microelectrodes and trained in a water reward task. During the task electrophysiological recordings of spikes, LFPs, and movement trajectories were acquired. The proposed dQ-learning algorithm achieved better learning performance with good prediction accuracy and a high convergence rate. The adaptive reward function and cell types were found to be critical factors for hippocampal spatial decoding using the dQ-learning method. </description><identifier>https://zenodo.org/record/3724076</identifier><identifier>10.5281/zenodo.3724076</identifier><identifier>oai:zenodo.org:3724076</identifier><relation>doi:10.5281/zenodo.3724075</relation><rights>info:eu-repo/semantics/openAccess</rights><rights>https://creativecommons.org/licenses/by/4.0/legalcode</rights><title>Enhancement of hippocampal spatial decoding using a dynamic Q-learning method with a relative reward using theta phase precession</title><type>Other:info:eu-repo/semantics/other</type><type>Other:software</type><recordID>3724076</recordID></dc>
format Other:info:eu-repo/semantics/other
Other
Other:software
Journal:eJournal
Journal
author Chen, Bo-Wei
Yang, Shih-Hung
Lo, Yu-Chun
Wang, Ching-Fu
Wang, Han-Lin
Hsu, Chen-Yang
Kuo, Yung-Ting
Chen, Jung-Chen
Lin, Sheng-Huang
Pan, Han-Chi
Lee, Sheng-Wei
Yu, Xiao
Qu, Boyi
Kuo, Chao-Hung
Chen, You-Yin
Lai, Hsin-Yi
title Enhancement of hippocampal spatial decoding using a dynamic Q-learning method with a relative reward using theta phase precession
publishDate 2020
url https://zenodo.org/record/3724076
contents Winners of the 2014 Nobel Prize in Physiology or Medicine, Professors John O’Keefe, May‐Britt Moser and Edvard I. Moser found that the internal global positioning system (GPS) in the brain allows us to be able to flexibly navigate the world they live in – exploring new areas, returning quickly to remembered places, and taking shortcuts and confirmed that place cells in hippocampus and grid cells in entorhinal cortex (EC) are responsible for higher-order cognitive map of the environment. Indeed, these abilities feel so easy and natural that it is not immediately obvious how complex the underlying processes really are. In contrast, spatial navigation remains a substantial challenge for artificial agents whose abilities are far outstripped by those of mammals. Hippocampal place cells and interneurons in mammals have proved that they own stable place fields and theta phase precession profiles to encode the spatial information from the environment. The hippocampal CA1 neurons can be represented as the location of the animal and the prospective information of goal location. Reinforcement learning algorithm, e.g., Q-learning, has been adopted to build a navigation model of place cells for the purpose of addressing goal direction navigation problems. In this study, we propose dynamical Q-learning (dQ-learning), because of its adaptive reward function based on theta phase precession, which has recently been associated with a rat’s experiences at destinations, and use of information from both place cells and interneurons as inputs to predict the animal’s trajectory. We evaluated the convergence rates and learning performances of tQ-learning and dQ-learning with different cell types. The results demonstrate that dQ-learning improves learning performance and convergence rate and place cells and interneurons with phase precession may provide valuable information to improve the prediction of trajectory. To investigate whether the enhancement of hippocampal spatial decoding with the dQ-learning method was effective in goal-direction navigation, experimental data were recorded from rats implanted with microelectrodes and trained in a water reward task. During the task electrophysiological recordings of spikes, LFPs, and movement trajectories were acquired. The proposed dQ-learning algorithm achieved better learning performance with good prediction accuracy and a high convergence rate. The adaptive reward function and cell types were found to be critical factors for hippocampal spatial decoding using the dQ-learning method.
id IOS17403.3724076
institution Universitas PGRI Palembang
institution_id 189
institution_type library:university
library
library Perpustakaan Universitas PGRI Palembang
library_id 587
collection Marga Life in South Sumatra in the Past: Puyang Concept Sacrificed and Demythosized
repository_id 17403
city KOTA PALEMBANG
province SUMATERA SELATAN
repoId IOS17403
first_indexed 2022-07-26T02:18:49Z
last_indexed 2022-07-26T02:18:49Z
recordtype dc
_version_ 1739407857663606784
score 17.610285