BIU Learning Club, January 13, 2025: Distilling Foundation Models for 3D Generation and Understanding
January 13 @ 12:00 pm - 1:00 pm IST
On January 13, Dr. Sagie Benaim from the Hebrew University of Jerusalem will give a talk titled: Distilling Foundation Models for 3D Generation and Understanding
Abstract:
Visual foundation models have revolutionized 2D visual tasks, achieving remarkable success in both discriminative and generative domains by leveraging massive collections of 2D data through self-supervised learning. However, the transition to 3D remains a significant challenge due to the scarcity of 3D data, making such approaches less effective for 3D tasks.
In this talk, I will discuss an alternative approach of distilling pretrained 2D foundation models into 3D physical models. I will explore this approach in two key areas: 1. 3D generation, in which I will discuss static and dynamic object stylization, 2. 3D understanding, in which I will discuss the segmentation of static and dynamic 3D scenes. Additionally, I will discuss how integrating 3D physical models can, in turn, improve the 3D consistency of pretrained foundation models.that extends the classical bias-variance tradeoff. Moreover, empirical insights on the generalization and double descent phenomenon in transfer learning of deep neural networks will be presented.
BIO:
Sagie Benaim is an Assistant Professor (Senior Lecturer) at the School of Computer Science and Engineering at the Hebrew University of Jerusalem. Previously, he was a postdoc at Copenhagen University, working with Prof. Serge Belongie and as a member of the Pioneer Center for AI. Prior to that, he completed his PhD at Tel Aviv University in the Deep Learning Lab under the supervision of Prof. Lior Wolf. His research interests lie in computer vision, machine learning, and computer graphics, with a particular focus on generative models, neural-based signal representations, and inverse graphics.