MLDS Unit seminar-2, Makoto Yamada, OIST (PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis)

MLDS Unit seminar-2, Makoto Yamada, OIST (PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis)
Thursday June 6th, 2024 10:00 AM to 11:00 AM
Lab5 D23

Description

Title: PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis

Abstract: SimSiam is a prominent self-supervised learning method that achieves impressive results in various vision tasks under static environments. However, it has two critical issues: high sensitivity to hyperparameters, especially weight decay, and unsatisfactory performance in online and continual learning, where neuroscientists believe that powerful memory functions are necessary, as in brains. In this paper, we propose PhiNet, inspired by a hippocampal model based on the temporal prediction hypothesis. Unlike SimSiam, which aligns two augmented views of the original image, PhiNet integrates an additional predictor block that estimates the original image representation to imitate the CA1 region in the hippocampus. Moreover, we model the neocortex inspired by the Complementary Learning Systems theory with a momentum encoder block as a slow learner, which works as long-term memory. We demonstrate through analysing the learning dynamics that PhiNet benefits from the additional predictor to prevent the complete collapse of learned representations, a notorious challenge in non-contrastive learning. This dynamics analysis may partially corroborate why this hippocampal model is biologically plausible. Experimental results demonstrate that PhiNet is more robust to weight decay and performs better than SimSiam in memory-intensive tasks like online and continual learning.

Add Event to My Calendar

Subscribe to the OIST Calendar

See OIST events in your calendar app