New Capabilities in Unsupervised Image to Image Translation by Sagie Benaim, TAU (DSI Learning Club talk)

Feb. 24th 2019, Sun. 12:00 , Sagie Benaim (webpage).

Tel-Aviv University (PhD Student).

Location: Gonda Building (901), Room 101.

New Capabilities in Unsupervised Image to Image Translation

Abstract:

In Unsupervised Image to Image Translation, we are given an unmatched set of images from domain A and domain B, and our task is to generate, given an image from domain A, its analogous image in domain B.

In the first part of the talk, I’ll describe a new capability which allows us to perform such translation, where only a single image is present in domain A. Specifically, given a single image x from domain A and a set of images from domain B, our task is to generate the analogous of x in B. We argue that this task could be a key AI capability that underlines the ability of cognitive agents to act in the world and present empirical evidence that the existing unsupervised domain translation methods fail on this task.

In the second part of the talk, I’ll describe a new capability which allows us to disentangle the “common” and “domain-specific” information of domains A and B. This allows us to generate, given a sample a in A and a sample b in B, an image in domain B which contains the “common” information of a and “domain-specific” information of b. For example, ignoring occlusions, B can be “people with glasses”, A can be “people without”. The “common” information is “faces” where the “domain-specific” information of B is “glasses”. At test time, we add the glasses of person in domain B to any person in domain A.

Lastly, time permitting, I’ll describe the application of these techniques in the context of Singing Voice Separation, where the training data contains a set of samples of mixed music (singing and instrumental) and an unmatched set of instrumental music.