FY2021 Annual Report

Cognitive Neurorobotics Research Unit
Professor Jun Tani

cnru FY2021 Annual Report 01
From left: Vsevolod, Jorge, Alexander, Tomoe, Takazumi, Nadine, Jun, Dongqi, Hiroki, Wataru, Prasanna, Fabien (on the monitor), Jeffrey, Federico (11/11/2021)

Abstract

CNRU started in September 2017. Currently CNRU consists of 2 staff scientists, 2 postdoc researchers, 1 technical staff, 7 Ph.D. students, and one administrator. The group is investigating the essential mechanisms on human embodied cognition by using frameworks of predictive coding, active inference, and model-based reinforcement learning while conducting various robotics experiments.

1. Staff

As of March 31, 2022

  • Prof. Jun Tani, Professor
  • Dr. Takazumi Matsumoto, Staff Scientist
  • Dr. Jeffrey Queißer, Staff Scientist
  • Dr. Jorge Gallego Perez, Postdoctoral Scholar
  • Dr. Fabien Benureau, Postdoctoral Scholar
  • Dr. Federico Sangati, Technical Staff
  • Mr. Dongqi Han, PhD Student
  • Ms. Nadine Wirkuttis, PhD Student
  • Mr. Wataru Ohata, PhD Student
  • Mr. Prasanna Vijayaraghavan, PhD Student
  • Mr. Vsevolod Nikulin, PhD Student
  • Mr. Hiroki Sawada, PhD Student
  • Mr. Alexander Baranski, PhD Student
  • Dr. Jeffrey White, Visiting researcher
  • Ms. Tomoe Furuya, Research Unit Administrator

2. Collaborations

2.1 Computational models on reinforcement learning

  • Description: Model studies on hierarchical, model-based, variational reinforcement learning.
  • Type of collaboration: Joint research
  • Researchers:
    • Prof. Kenji Doya, Neural Computation Unit, OIST

2.2 Computational models on the sense of self

  • Description: Model studies on the sense of primitive and narrative selves.
  • Type of collaboration: Joint research
  • Researchers:
    • Prof. Jeff White, Twente University

3. Activities and Findings

3.1 Morphological Development at the Evolutionary Timescale: Robotic Developmental Evolution.

Evolution and development operate at different timescales; generations for the one, a lifetime for the other. These two processes, the basis of much of life on earth, interact in many non-trivial ways, but their temporal hierarchy—evolution overarching development—is observed for most multicellular lifeforms. When designing robots however, this tenet lifts: it becomes—however natural—a design choice. We propose to inverse this temporal hierarchy and design a developmental process happening at the phylogenetic timescale. Over a classic evolutionary search aimed at finding good gaits for tentacle 2D robots, we add a developmental process over the robots’ morphologies. Within a generation, the morphology of the robots does not change. But from one generation to the next, the morphology develops. Much like we become bigger, stronger, and heavier as we age, our robots are bigger, stronger and heavier with each passing generation. Our robots start with baby morphologies, and a few thousand generations later, end-up with adult ones. We show that this produces better and qualitatively different gaits than an evolutionary search with only adult robots, and that it prevents premature convergence by fostering exploration. In addition, we validate our method on voxel lattice 3D robots from the literature and compare it to a recent evolutionary developmental approach. Our method is conceptually simple, and can be effective on small or large populations of robots, and intrinsic to the robot and its morphology, not the task or environment. Furthermore, by recasting the evolutionary search as a learning process, these results can be viewed in the context of developmental learning robotics.

cnru FY2021 Annual Report 02
Figure 1: Development has a notable impact on exploration. Principal component analysis of the genotype vectors of two runs: one with a developmental muscle model with a starting size 0.5 (a), and one with the adult evolution (b), with the same starting population. The genealogy shows the member with the best fitness of each generation and their ancestors. Edges show parental relationships.

3.2 Leading or Following? Dyadic Robot Imitative Interaction Using the Active Inference Framework.

This study investigated how social interaction among robotic agents changes dynamically depending on the individual belief of action intention. In a set of simulation studies, we examine dyadic imitative interactions of robots using a variational recurrent neural network model. The model is based on the free energy principle such that a pair of interacting robots find themselves in a loop, attempting to predict and infer each other’s actions using active inference. We examined how regulating the complexity term to minimize free energy determines the dynamic characteristics of networks and interactions. When one robot trained with tighter regulation and another trained with looser regulation interact, the latter tends to lead the interaction by exerting stronger action intention, while the former tends to follow by adapting to its observations. The study confirms that the dyadic imitative interaction becomes successful by achieving a high synchronization rate when a leader and a follower are determined by developing action intentions with strong belief and weak belief, respectively.

cnru FY2021 Annual Report 03
Figure 2: An architecture used in the experiments of imitative interaction.

3.3 Emergence of Content-Agnostic Information Processing by a Robot Using Active Inference, Visual Attention, Working Memory, and Planning.

Generalization by learning is an essential cognitive competency for humans. For example, we can manipulate even unfamiliar objects and we can generate mental images before enacting a preplan. How is this possible? The current study investigated this problem by revisiting our previous study (Jung et al., 2019), which examined the problem of vision-based, goal-directed planning by robots performing a task of block stacking. By extending the previous study, the current work introduces a large network comprised of dynamically interacting sub-modules, including visual working memory (VWMs), a visual attention module, and an executive network. The executive network predicts motor signals, visual images, and various controls for attention, as well as masking of visual information. The most significant difference from the previous study is that the current model contains an additional VWM. The whole network is trained by using predictive coding and an optimal visuo-motor plan to achieve a given goal state is inferred using active inference. Results indicate that the current model performs significantly better than that used in Jung et al. (2019), especially when manipulating blocks with unlearned colors and textures. Simulation results revealed that the observed generalization was achieved because content-agnostic information processing developed through synergistic interaction between the 2nd VWM and other modules during the course of learning, in which memorizing image contents and transforming them are dissociated. This paper verifies this claim by conducting both qualitative and quantitative analysis of simulation results.

cnru FY2021 Annual Report 04
Figure 3: Results of successful planning in a case with objects having learned colors (green and blue) and a novel color (orange) with the goal of building a stack of blocks, green on blue and orange on green. Visualization of the internal states of the proposed model and the discrepancy between the inferred plan and the ground truth. Animation available at https://youtu.be/pZBMEIjjh6Q.

4. Publications

4.1 Journals

  1. Benureau, F. C., & Tani, J. (2020). Morphological Development at the Evolutionary Timescale: Robotic Developmental Evolution. Alife, 00: 1–19.
    PDF
  2. 谷淳 (2021). 池上「生命理論としての認知科学：減算と縮約の哲学をめぐって」へのコメント. 認知科学. 28(2) 222-230. LINK
  3. Wirkuttis, N., & Tani, J. (2021). Leading or Following? Dyadic Robot Imitative Interaction Using the Active Inference Framework. IEEE Robotics and Automation Letters, 6(3) 6024-6031.
    PDF
  4. Queißer, J. F., Jung, M., Matsumoto, T., & Tani, J. (2021). Emergence of Content-Agnostic Information Processing by a Robot Using Active Inference, Visual Attention, Working Memory, and Planning. Neural Computation, 33(9), 2353–2407.
    PDF

4.2 Books and refereed conference proceedings

  1. Lanillos, P., Meo, C., Pezzato, C., Meera, A. A., Baioumy, M., Ohata, W., Tschantz, A., Millidge, B., Wisse, M., Buckley, C. L., & Tani, J. (2021). Active Inference in Robotics and Artificial Agents: Survey and Challenges. arXiv preprint arXiv:2112.01871. LINK
  2. Idei, H., Ohata, W. Yamashita, Y., Ogata, T., & Tani, J. (2021). Sensory attenuation develops as a result of sensorimotor experience. arXiv preprint arXiv:2111.02666. LINK
  3. Han, D., Doya, K., & Tani, J. (2020). Goal-Directed Planning by Reinforcement Learning and Active Inference. arXiv preprint arXiv:2106.09938v2. LINK

4.3 Invited seminars and talks

  1. Invited talk, Tani, J. Predictive coding and active inference framework for cognitive neurorobotics. ECoGs seminar organized by Tom Froese (Online), February 28, 2022.
  2. Plenary talk, Tani, J. Cognitive Neurorobotics Study Using the Free Energy Principle. FEP workshop(Online), December 14, 2021.
  3. Invited talk, Tani, J. Robotic Minds from Predictive Coding and Active Inference. Graduate School, University of Tokyo, Graduate Course: Brain Information Processing Systems, (Online), December 10, 2021.
  4. Tani, J. Studies on cognitive neurorobotics using the framework of predictive coding and active inference. RIKEN-OIST Joint Symposium, Series 2, Neurosciences and AI/Data science, Kinds of Minds - What is thinking? (Online), October 6, 2021.
  5. Invited talk, Tani, J. Cognitive neurorobotic self in the shared world. International Conference on Philosophy of Information ICPI 2021 (Online), September 18, 2021.
  6. Keynote talk, Tani, J. Exploring robotic minds using the concepts of predictive coding and active inference. IEEE ICDL 2021, Beijing, China (Online), August 28, 2021.
  7. Invited talk, Tani, J. An analysis of meta-level cognitive processes of a variational recurrent neural network model when acting with the environment. ICDL Workshop StEPP'21 (Online), August 22, 2021.
  8. Invited talk, Tani, J. Exploring robotic minds using predictive coding and active inference frameworks. Cognitive Informatics at Université du Québec à Montréal (Online), April 1, 2021.

5. Intellectual Property Rights and Other Specific Achievements

Nothing to report

6. Meetings and Events

6.1 Seminars

Nothing to report.

6.2 Events

Nothing to report.

7. Other

  • 2021 June, We received the Google Research Gift on the study of the application of the free energy principle.