Generative adversarial networks: architecture, training objective, optimality, and variants of GANs.

@Credits: EECS 498.007 | Video Lecture: UM-CV

Personal work for the assignments of the course: github repo.

Notice on Usage and Attribution

These are personal class notes based on the University of Michigan EECS 498.008 / 598.008 course. They are intended solely for personal learning and academic discussion, with no commercial use.

For detailed information, please refer to the complete notice at the end of this document

Generative Adversarial Networks

Setup: Assume we have data $x_i$ drawn from distribution $p_{data}(x)$. Want to sample from $p_{data}$

Architecture

Idea: Introduce a latent variable $z$ with simple prior $p(z)$

• Sample $z\sim p(z)$ and pass to a Generator Network $x=G(z)$
• Then $x$ is a sample from the generator distribution $p_G$. Want $p_G=p_{data}$
• The Discriminator Network $D$ is trained to classify data as real or fake

GAN

Training Objective

GAN: Training Objective

Problems:

• We are not minimizing any overall loss, and there is no training curves to look at.
• Vanishing gradients for G:

Optimality

Optimal Discriminator

This expression cannot be computed since it involves $p_{data}$。

GAN: Optimal Discriminator

Optimality

GAN: Jensen-Shannon Divergence

Summary

GAN: Jensen-Shannon Divergence

Variants of GAN

DC-GAN

Radford et al, ICLR 2016

Fun facts

Fun facts

Conditional GANs

A style-based generator architecture for generative adversarial networks, CVPR 2019

A learned representation for artistic style, ICLR 2017

Conditional batch normalization

BigGAN

Large Scale GAN Training for High Fidelity Natural Image Synthesis

Generating Videos with GANs

Adversarial Video Generation on Complex Datasets, Arxiv 2019

Text2Img

Zhang et al, “StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks.”, TPAMI 2018

Zhang et al, “StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks.”, ICCV 2017

Reed et al, “Generative Adversarial Text-to-Image Synthesis”, ICML 2016

Super-resolution

Ledig et al, “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”, CVPR 2017

Pix2Pix

Isola et al, “Image-to-Image Translation with Conditional Adversarial Nets”, CVPR 2017

Zhu et al, “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”, ICCV 2017

Label Map to Img

Park et al, “Semantic Image Synthesis with Spatially-Adaptive Normalization”, CVPR 2019

Trajectory Prediction

Gupta, Johnson, Li, Savarese, Alahi, “Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks”, CVPR 2018

Notice on Usage and Attribution

This note is based on the University of Michigan's publicly available course EECS 498.008 / 598.008 and is intended solely for personal learning and academic discussion.

• Nature of the Notes: These notes include extensive references and citations from course materials to ensure clarity and completeness. However, they are presented as personal interpretations and summaries, not as substitutes for the original course content. Please refer to the official University of Michigan website for complete and accurate course materials.
• Third-Party Open Access Content: This note may reference Open Access (OA) papers or resources cited within the course materials. These materials are used under their original Open Access licenses (e.g., CC BY, CC BY-SA). Every referenced OA resource is appropriately cited, including the author, publication title, source link, and license type.
• Copyright: All rights to third-party content remain with their respective authors or publishers. If you believe any content infringes on your copyright, please contact me, and I will promptly remove the content in question.

Thanks to the University of Michigan and the contributors to the course for their openness and dedication to accessible education.