- This event has passed.
Second-order Optimization Made Practical – Talk by Tomer Koren, TAU
December 1 @ 12:00 pm - 1:00 pm IST
Title: Second-order Optimization Made Practical
Abstract: Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent. Higher-order (preconditioned) optimization methods have become far less prevalent, despite compelling theoretical properties, due to their impractical computation, memory and communication costs. I will present some recent theoretical, algorithmic and infrastructural advances that allow for overcoming these challenges and obtaining significant gains from higher-order methods in practice, at very large scale and on highly-parallel computing architectures.