Instantiating an adversarial model

• Build separate models for each component / player such as generator and discriminator.
• Build a combined model. For a GAN, this might have an input for images and an input for noise and an output for D(fake) and an output for D(real)
• Pass the combined model and the separate models to the AdversarialModel constructor

adversarial_model = AdversarialModel(base_model=gan,
  player_params=[generator.trainable_weights, discriminator.trainable_weights],
  player_names=["generator", "discriminator"])

The resulting model will have the same inputs as gan but separate targets and metrics for each player. This is accomplished by copying the model for each player. If each player has a different model, use player_models (see below regarding dropout).

adversarial_model = AdversarialModel(player_models=[gan_g, gan_d],
  player_params=[generator.trainable_weights, discriminator.trainable_weights],
  player_names=["generator", "discriminator"])

Compiling an adversarial model

Use adversarial_compile to compile the model. The parameters are an AdversarialOptimizer and a list of Optimizer objects for each player. The loss is passed to model.compile for each model, so may be a dictionary or other object. Use the same order for player_optimizers as you did for player_params and player_names.

model.adversarial_compile(adversarial_optimizer=adversarial_optimizer,
  player_optimizers=[Adam(1e-4, decay=1e-4), Adam(1e-3, decay=1e-4)],
  loss='binary_crossentropy')

Training a simple adversarial model

Adversarial models can be trained using fit and callbacks just like any other Keras model. Just make sure to provide the correct targets in the correct order.

For example, given simple GAN named gan: * Inputs: [x] * Targets: [y_fake, y_real] * Metrics: [loss, loss_y_fake, loss_y_real]

AdversarialModel(base_model=gan, player_names=['g', 'd']...) will have: * Inputs: [x] * Targets: [g_y_fake, g_y_real, d_y_fake, d_y_real] * Metrics: [loss, g_loss, g_loss_y_fake, g_loss_y_real, d_loss, d_loss_y_fake, d_loss_y_real]

MNIST Generative Adversarial Network (GAN)

example_gan.py shows how to create a GAN in Keras for the MNIST dataset.

Example GAN

CIFAR10 Generative Adversarial Network (GAN)

example_gan_cifar10.py shows how to create a GAN in Keras for the CIFAR10 dataset.

Example GAN

MNIST Bi-Directional Generative Adversarial Network (BiGAN)

example_bigan.py shows how to create a BiGAN in Keras.

Example BiGAN

MNIST Adversarial Autoencoder (AAE)

An AAE is like a cross between a GAN and a Variational Autoencoder (VAE). example_aae.py shows how to create an AAE in Keras.

Example AAE

Unrolled Generative Adversarial Network

example_gan_unrolled.py shows how to use the unrolled optimizer.

WARNING: Unrolling the discriminator 8 times takes about 6 hours to build the function on my computer, but only a few minutes for epoch of training. Be prepared to let it run a long time or turn the depth down to around 4.

Dropout

When training adversarial models using dropout, you may want to create separate models for each player.

If you want to train a discriminator with dropout, but train the generator against the discriminator without dropout, create two models. * GAN to train generator: D(G(z, dropout=0.5), dropout=0) * GAN to train discriminator: D(G(z, dropout=0), dropout=0.5)

If you create separate models, use player_models parameter of AdversarialModel constructor.

If you aren't using dropout, one model is sufficient, and use base_model parameter of AdversarialModel constructor, which will duplicate the base_model for each player.

Theano and Tensorflow

I do most of my development in theano but try to test tensorflow when I have extra time. The goal is to support both. Please let me know any issues you have with either backend.

Questions?

Feel free to start an issue or a PR here or in Keras if you are having any issues or think of something that might be useful.