- This event has passed.
BIU Learning Club, November 18 – Exploiting Symmetries for Learning in Deep Weight Spaces
November 18 @ 12:00 pm - 1:00 pm IST
On November 18, Dr. Haggai Maron from the Technion will give a talk titled: Exploiting Symmetries for Learning in Deep Weight Spaces
Abstract:
This talk explores the emerging research direction that studies neural network weights as a novel data modality. We’ll discuss recent advances in processing and analyzing raw weight matrices, which exhibit inherent symmetries reminiscent of other structured data types such as graphs. Our focus will be on designing deep architectures that effectively operate on weight spaces while respecting these symmetries. We’ll present our ICML 2023 work on equivariant architectures for processing multilayer perceptron weight spaces, and our ICLR 2024 paper on Graph Metanetworks (GMN), which generalizes this approach to diverse network architectures by representing them as graphs. Additionally, we’ll touch on our recent ICML 2024 works exploring data augmentations in weight spaces and data-driven network alignment. These developments open new possibilities for deep network analysis, editing, and manipulation, with applications ranging from Implicit Neural Representation editing to weight pruning and function manipulation.
BIO:
Haggai Maron is an assistant professor at the Faculty of Electrical and Computer Engineering at the Technion, Israel. He is also a Senior Research Scientist at NVIDIA Research. His research focuses on machine learning in structured domains, with a particular interest in applying deep learning to data with symmetries. He received his Ph.D. in 2019 from the Weizmann Institute of Science, where he worked under the supervision of Professor Yaron Lipman. Haggai received the ICML outstanding paper award (2020) and the Alon fellowship for outstanding young faculty members (2024).