Transforming and Projecting Images into Class-conditional Generative Networks


Minyoung Huh12
Richard Zhang2
Jun-Yan Zhu2
Sylvain Paris2
Aaron Hertzmann2

1MIT CSAIL
2Adobe Research


Code [GitHub]

arXiv preprint [Paper]







Abstract


We present a method for projecting an input image into the space of a class-conditional generative neural network. We propose a method that optimizes for transformation to counteract the model biases in a generative neural networks. Specifically, we demonstrate that one can solve for image translation, scale, and global color transformation, during the projection optimization to address the object-center bias of a Generative Adversarial Network. This projection process poses a difficult optimization problem, and purely gradient-based optimizations fail to find good solutions. We describe a hybrid optimization strategy that finds good projections by estimating transformations and class parameters. We show the effectiveness of our method on real images and further demonstrate how the corresponding projections lead to better edit-ability of these images.





Paper


M. Huh, R. Zhang, JY. Zhu, S. Paris, A. Hertzmann,
Aligning and Projecting Images to
Class-conditional Generative Networks

In arXiv preprint, 2020 (arXiv).
[Bibtex]




Acknowledgements

We would like to thank Phillip Isola and David Bau for helpful discussions.

Website template from Colorful Colorization.