- This event has passed.
BIU Learning Club 4.4.2021 — Ronen Basri — On the Connection between Deep Neural Networks and Kernel Methods
April 4, 2021 @ 12:00 pm - 1:00 pm IDT
Hey everyone,After the holiday, on Sunday 4.4.2021 at 12:00, we will host Ronen Basri from Weizman Institute.Ronen will present his work on the connection between deep neural networks and kernel methods.Meeting ID: 880 0879 7212Passcode: 416934Title: On the Connection between Deep Neural Networks and Kernel MethodsAbstract: Recent theoretical work has shown that massively overparameterized neural networks are equivalent to kernel regressors that use Neural Tangent Kernels (NTKs). Experiments indicate that these kernel methods perform similarly to real neural networks. My work in this subject aims to better understand the properties of NTK and relate them to properties of real neural networks. In particular, I will argue that for input data distributed uniformly on the sphere NTK favors low frequency predictions over high frequency ones, potentially explaining why overparameterized networks do not overfit their training data. I will further discuss the behavior of NTK when data is distributed nonuniformly, and finally show that NTK is tightly related to the classical Laplace kernel, which has a simple closed form. Our results suggest that much insight about neural networks can be obtained from analysis of NTK.