Tamir Hazan, Technion . “Direct Optimization through argmax for Discrete Variational Auto-Encoder” (DSI Learning Club)

Nov. 25th 2018, Sun. 12:00 , Tamir Hazan (webpage).

Technion – Israel Institute of Technology.

Location: Gonda Building (901), Room 101.

Direct Optimization through argmax for Discrete Variational Auto-Encoder

Abstract:

Reparameterization of variational auto-encoders is an ehttps://sites.google.com/view/biu-learning-clubffective method for reducing the variance of their gradient estimates. However, when the latent variables are discrete, a reparameterization is problematic due to discontinuities in the discrete space. In this work, we extend the direct loss minimization technique to discrete variational auto-encoders. We first reparameterize a discrete random variable using the arg max function of the Gumbel-Max perturbation model. We then use direct optimization to propagate gradients through the non-differentiable arg max using two perturbed arg max operations.