BIU learning club – Ravid Shwartz-Ziv – Exploring the Successes and Limitations of Deep Learning

Location:

Engineering building (1103), room 329

Title:

Exploring the Successes and Limitations of Deep Learning 

Abstract:

In this talk, we will explore the successes and limitations of deep learning networks and highlight the need for more rigorous evaluation. Tree ensemble models often outperform deep learning models for tabular data. However, recently there have been claims that some new deep-learning models have better results. We will evaluate these claims and explore when deep learning may be a good solution. Another common challenge is dealing with class-imbalanced datasets. We will present our empirical investigation of this problem and discuss the learning behavior of different models and datasets. Based on this analysis, we will suggest simple yet effective solutions for handling class imbalance.

In the second part of the talk, we will focus on self-supervised models. We will analyze their construction and optimality, demonstrate how they can be (re)discovered based on first principles, and present new ones. We will also discuss the optimal transfer learning for these models and how to transfer the optimal amount of information between tasks.

Finally, we will propose a framework for making deep learning research more rigorous using tractable datasets and generative models. This will help us investigate models on various problems and control the true joint distribution of the data, allowing us to address issues of generalization and optimal representations.

Short Bio:

Ravid Shwartz-Ziv is currently a CDS Faculty Fellow at NYU Center for Data Science, working with Prof. Yann Lecun on neural networks, information theory, and self-supervised learning.