Open in app Sign up Sign In

Search Medium

Published in Towards Data Science

You have 2 free member-only stories left this month.

Sign up for Medium and get an extra one

Nahid Alam Follow

Feb 7, 2022 · 6 min read · · Listen

Save

Building a CycleGAN model with Custom

Dataset using Tensorflow 2
And deploying in Hugging Face Space
 Image Credit: Author

Generative Adversarial Network (GAN) is a type of Generative modeling

technique in Machine Learning. In a GAN, two different neural networks
(Generator and Discriminator) compete with each other. The output of the
Generator — although synthetic, can be close to reality.

There are many different GAN architectures. Today, we will focus on

CycleGAN. What makes cycleGAN interesting is that — it is an unpaired
image-to-image translation technique.

In this work, we will look into an end-to-end example of training and

deploying a cycleGAN model on a custom dataset. You can find workable
CycleGAN code (using Tensorflow 2) here so I will not repeat what we
already have. Rather, I would like to focus on a few important missing
pieces that you will need in real life Deep Learning projects — working
with custom data. evaluating GAN models, using an already trained model
for prediction and finally creating a fun demo!!

Therefore the contribution of this work can be summarized as —

• Create a Tensorflow dataset with custom image data

• Calculate FID score to evaluate a GAN model

• Saving and loading the model

• Deploy the model at Hugging Face Space

And have fun !

Create a Tensorflow dataset with custom image data:

To train our model in Tensorflow, we need to have our training dataset in

Tensorflow Dataset format — exposed as tf.data.Datasets . Each element
in a tf.data.Datasets can be composed of one or more element. For
example, a single element in an image pipeline can be a pair of tensor
representing the image and its label. In our example, we will represent an
image in a TFRecord format — Tensorflow’s own binary record format.
There are a few advantages of using TFRecord format -

• It is a binary format, therefore takes less space in disk and less

read/write time in memory

• With TFRecord, you can combine multiple datasets into one dataset.
For example — in this work, we combine multiple images into one
TFRecord. This combined record is well integrated into the data
loading and preprocessing functionalities of tf.data.Datasets . This is
specially helpful when loading a really big dataset — instead of loading
 the entire dataset in the memory, the dataset library can load only the
necessary portion of the TFRecord for processing.

• You can store sequence data in TFRecord — for example word

embedding or TimeSeries data.

One downside of TFRecord is — creating a TFRecord is not straightforward

— not to me at least :)

First , we need to define a dictionary that describes the components of the

TFRecord. For example — for an image classification problem
(supervised), you might have an image and a label . Since, we are working
on a cycleGAN model, we do not need a label as it is unsupervised in
nature.

feature = {
‘image’: _bytes_feature(image),
}

Here, _bytes_feature is a private method that returns a bytes list from a

string/byte

def _bytes_feature(value):
“””Returns a bytes_list from a string / byte.”””
if isinstance(value, type(tf.constant(0))):
value = value.numpy()
return
tf.train.Feature(bytes_list=tf.train.BytesList(value=
[value]))

Once we have the TFRecord, how do we know that the record was actually
created correctly? We can count the number of items in each of our
TFRecords (remember — we have multiple images in one TFRecord). We
can also visualize the entire dataset.
 FILENAMES = tf.io.gfile.glob('cat*.tfrec')
print(f'TFRecords files: {FILENAMES}')
print(f'Created image samples:
{count_data_items(FILENAMES)}')

display_samples(load_dataset(FILENAMES,
*IMAGE_SIZE).batch(1), 2, 5)

Note those two numbers in 2 and 5 in display_samples function call.

Those numbers indicate number of rows and columns. Multiplying the
number of rows with number of columns gives you the total number of
images in your dataset. So it needs to match with your dataset size.

For the end-to-end code on custom dataset creation, please refer here

Evaluating a GAN model

There is no single metric to evaluate a GAN model. Depending on the

usecase, you might want to use a combination of quantitative and
qualitative metrics.

In our work, we will use FID score. Frechet Inception Distance (FID)
measures the distance between the features of generated image and real
image. The lower the FID, the better. If FID is 0, it means two images are
same

So how do we calculate FID score? tldr;

• Use the pretrained Inception V3 model, remove the last layer from it

• Generate feature vectors of the two images (generated and real). The
vector size will be 2,048

• FID score is then calculated using equation 1 as described in the paper

A few shortcomings of FID to remember

 • FID uses a pretrained Inception model whose feature vector might not
capture the necessary features for your usecase.

• To work well, FID requires a large sample size. The minimum

recommended sample size is 10,000

If you want to learn more on how to calculate FID score, please refer here

Saving and Loading a Model:

After training the model, we want to save it (assuming we are happy with
the train/validation loss and evaluation). While saving the model, we want
to make sure we save the entire model.

What do we mean by entire model?

• Model’s architecture/config, weight

• Model’s optimizer state and

• Model’s compilation information (if .compile() was called)

We want to save the entire model because —

• During inference, you don’t need the model architecture code. This
way, you will have a clean inference logic and faster development
cycle.

• If you want to convert the model for edge devices (TFLite) or want to
run the model in browser with Tensorflow.js, you need to have the
entire model

There are a couple of ways to save the entire model. For Tensorflow 2, I
prefer SavedModel format. You can use Keras’s model.save method or
Tensorflow’s tf.keras.models.save_model method. In the model.save()

function, if you use a string as a parameter, your model will be saved as

SavedModel format.
 model.save('art_generator')

Deploying in Hugging Face Space:

Now that we have our model saved, we can write the inference logic and
deploy it for people to use. For this project, I have deployed the model in
Hugging Face Space here. If you visit that link, you will see something like
this —

A Sample App on Hugging Face Space

There, you can upload your cat photo (or any other pet), press the submit

button and wait 10 sec to see the cat art something like below —

Photo to art

You may realize that I need a lot more iterations to improve the quality of
the art. But nonetheless, this is a fun app to play with!
Details on how to deploy a model in Hugging Face Space can be found
here.

References:
1. https://developers.google.com/machine-learning/gan

2. Details on TFRecord https://medium.com/mostly-ai/tensorflow-

records-what-they-are-and-how-to-use-them-c46bc4bbb564

3. How to evaluate a GAN model https://machinelearningmastery.com

/how-to-evaluate-generative-adversarial-networks/

4. Kaggle
Deep CycleGAN
Learning Example
Tensor Flow https://www.kaggle.com/amyjang/monet-
Generative Adversarial Hugging Face
cyclegan-tutorial
Cats
5. How to calculate FID score https://machinelearningmastery.com/how-
to-implement-the-frechet-inception-distance-fid-from-scratch/

6. How to save and load a Tensorflow Model https://medium.com/deep-

learning-with-keras/save-load-keras-models-with-custom-layers-
8f55ba9183d2
Sign up foronThe
7. Details Variable
evaluating GAN models https://wandb.ai/ayush-thakur/gan-
By Towards Data Science
evaluation/reports/How-to-Evaluate-GANs-using-Frechet-Inception-
Every Thursday, the Variable delivers the very best of Towards Data Science: from hands-on tutorials
Distance-FID---Vmlldzo0MTAxOTI
and cutting-edge research to original features you don't want to miss. Take a look.

8. Deploying your model at Hugging Face Space https://huggingface.co

By signing up, you will create a Medium account if you don’t already have one. Review
our Privacy Policy for more information about our privacy practices.
/docs/hub/spaces
Get this newsletter

About Help Terms Privacy

Get the Medium app