Sannomiya Seitai

With the abnormal oscillation from basal ganglia, how motor cortex and thalamus are involved with the generation of tremor has not been made clear. To investigate this, we have been developing a spiking neural network model of motor cortex and  thalamus according to electrophysiological and anatomical data. We have confirmed reproduction of some basic neural behaviour observed in experiments, for instance, theta-alpha oscillation (4-12Hz) in thalamus, and lateral inhibition among the cortical columns in the motor cortex (Figure 3.1.2). The basal ganglia are the locus of Parkinson's disease through the loss of dopamine-neurons in the substantia nigra pars compacta, and the basal ganglia-thalamo-cortical loops are known to play a critical role in the development of tremor and other motor-related symptoms. However, the precise mechanisms by which the loss of dopaminergic neurons gives rise to parkinsonian tremor are still unknown. Aiming at understanding the parkinsonian pathology as a dynamical system, we attemp to construct an integrated set of realistic models of the basal ganglia-thalamo-cortical areas.
To  achieve this, we must coordinate with research institutes that can study themes closely related to the local community and a system that realizes communication among NPOs, residents and administration. Amagase Masahiro completed human behavioral science course at Graduate School of Literature and Human Sciences, Osaka City University. He studies environmental cognition and behaviors, and the psychological basis of ethics as our way of living. He chairs the meeting for promoting moral education at Nara since 2012, and the conference for the study of “Inochi” education at Nara Prefecture Uda Animal Park since 2013.

The Neural Computation Unit pursues the dual goals of developing robust and flexible learning algorithms and elucidating the brain’s mechanisms for robust and flexible learning. Our specific focus is on how the brain realizes reinforcement learning, in which an agent, biological or artificial, learns novel behaviors in uncertain environments by exploration and reward feedback. We combine top-down, computational approaches and bottom-up, neurobiological approaches to achieve these goals. The major achievements of the three subgroups in the fiscal year 2013 are the following.
We regularly held discussions and presented proposals beyond the boundary of project about how to apply the outcome of each project to the establishment of global environmentology, which is the target of the Research Institute for Humanity and Nature, and considered the required mechanisms. System in the Lake Biwa-Yodo River 三宮　美容鍼 watershed in the rice-growing areas. Todonavi is a website where you can find everything you want to know about Kagoshima and Japanese culture. We offer a large amount of content that includes tourist places, restaurants, gastronomy, culture, local products, accommodation and useful information about this wonderful city.

Jointly with the Center for Ecological Research of Kyoto University, one of our cooperating institutes, and introduced the Institute's projects at free gatherings. For publication, we prepared a final year result report on the Project and we plan to deliver copies to the relevant organizations in Shiga prefecture and to university libraries throughout the country. Next year, we are planning to publish a book on the results of the Project on a commercial basis through Kyoto University Press as part of the Institute's library.
To further investigate whether a timely activation of the DRN serotonergic neurons causes animals to be more patient for delayed rewards, we introduced transgenic mice that expressed the channelrhodopsin-2 variant ChR2 in the serotonin neurons. We confirmed that blue light stimulation of DRN effectively activate serotonin neurons by monitoring serotonin efflux in the medial prefrontal. We found that serotonin neuron stimulation prolonged the time animals spent for waiting in reward omission trials. This effect was observed specifically when the animal was engaged in deciding whether to keep waiting and not due to motor inhibition. Control experiments showed that the prolonged waiting times observed with optogenetic stimulation were not due to behavioral inhibition or the reinforcing effects of serotonergic activation (Miyazaki et al., Curr Biol, 2014). These results show that the timed activation of serotonin neurons during waiting promotes animals’ patience to wait for delayed rewards.
It is easy to design a sparse reward function which gives a positive reward when the task is accomplished and zero otherwise, but that makes it hard to find an optimal policy. In some situations, it is easier to prepare examples of a desired behavior than to handcraft an appropriate reward/cost function. Recently, Inverse Reinforcement Learning has been proposed in order to derive a reward/cost function from demonstrator's performance and to implement imitation learning. This study proposes  a novel inverse reinforcement learning method based on density estimation under the framework of Linearly solvable Markov Decision Process .

We found that post-inhibitory rebound potentiation of STN neurons and short-term synaptic plasticities are required for the pathological oscillatory burst activities (Figure 3.1.1). This suggests that strengthened connectivity between STN and GPe and reduced autonomous activity of GPe neurons, both of which are known to be caused by dopamine depletion, switch neuronal discharges in STN and GPe from normal to the parkinsonian states. Yukie Nagai, “From cognition to social interaction based on predictive learning,” IROS 2016 Workshop on Bio-inspired Social Robot Learning in Home Scenarios, Daejeon, Korea, October 10, 2016.
In model-based reinforcement learning, an optimal controller is derived from an optimal value (cost-to-go) function by solving the Bellman equation, which is often intractable due to its nonlinearity. Linearly solvable Markov decision process is a computational framework to efficiently solve the Bellman equation by exponential transformation of the value function under a constraint on the action cost function. The major drawback of the LMDP framework is, however, that an environmental model is given in advance. Model learning is integrated with LMDP to overcome the problem for continuous problems reported in FY2012, but the performance of the obtained controllers is critically affected by the accuracy of the environmental model. Value-based decision strategies, such as Q-learning, has been utilized to analyze the neuronal basis of decision making.

Yukie Nagai, “Computational models for cognitive development,” ISSA Summer School 2017, Osaka, May 22-June 2, 2017. Yukie Nagai, “Computational approach to understanding underlying neural mechanisms of autism spectrum disorder,” The 115th Annual Meeting of the Japanese Society of Psychiatry and Neurology, Niigata, June 20-22, 2019. Anja Philippsen, Sho Tsuji, and Yukie Nagai, “Quantifying developmental differences in drawing ability using a convolutional neural network,” International Symposium on Artificial Intelligence and Brain Science, October 10-12, 2020. Yukie Nagai, “Cognitive development based on predictive coding,” Cross Roads #16, Tokyo, September 25, 2020. & Doya, K. Scaled free-energy based reinforcement learning for robust and efficient learning in high-dimensional state spaces, in Neuro 2013, Kyoto International Conference Center . Uchibe, E. Inverse reinforcement learning for understanding human behaviors, in International Symposium on Past and Future Directions of Cognitive Developmental Robotics, Osaka University Nakanoshima Center 10F .
At the same time, rat’s movement was recorded by 3D motion tracking system (Figure 3.2.3). This system was able to measure IR-reflection marker positions by using two video cameras and IR lights. We attached IR-reflection markers on the rat’s  head, back, and tail.

On the other hand, when the cue tone B was represented, they should not have selected, and were not able to get a reward. If they selected, the cue tone B was represented repeatedly. We recorded the neuronal activities from the dorsomedial striatum , the prelimbic cortex (PL; a dorsal part of the prefrontal cortex), and the primary motor cortex of rats during the choice task.
These differences in how the problem develops depending on the hierarchy are a detrimental factor with respect to inter-hierarchical communication about watershed management. To preserve the environment of Lake Biwa, this Project specifically demonstrated the importance of fine-tuned environmental activities with local residents taking the initiative in a bottom-up approach now that it is known that smaller rivers, which cannot be handled by the administration, have a big impact. The Project was also engaged in the development of a method to support local citizens' voluntary environmental preservation activities based on the prerequisite of coordination with local individualities and problems unique to the specific local community. In other words, we were engaged in practical research that includes the development of a mechanism to realize governance.