1.

Using TensorBoard Debugger

During model training using Tensorflow, events which involve NANs can affect the training
process leading to the non-improvement in model accuracy in subsequent steps. TensorBoard
2.3+ (together with TensorFlow 2.3+) provides a debugging tool known as Debugger 2. This
tool will help track NANs in a Neural Network written in Tensorflow.

Here’s an example provided in the TensorFlow Github which involves training the mnist
dataset and capturing NANs then analyzing our results on TensorBoard debugger 2. 

First, Navigate to project root on your terminal and run the below command.

python -m tensorflow.python.debug.examps.v2.debug_mnist_v2 --dump_dir

/tmp/tfdbg2_logdir --dump_tensor_debug_mode FULL_HEALTH
Copy
Once the above command runs, you will notice non-improvement in model accuracy displayed
on the terminal. This is caused by NANs, you can launch a TensorBoard debugger 2 by the
following command. 

tensorboard --logdir /tmp/tfdbg2_logdir

Copy
Output

To invoke the debugger on your model,

use tf.debugging.experimental.enable_dump_debug_info()  which is the API entry point of
Debugger V2. It takes parameters:

1. logdir which is log directory

 2. tensor_debug_mode – controls information debugger extract from each aeger or in-
graph tensor
3. circular_buffer_size – controls number of tensor events to be saved. Default is 1000 and
unsetting it, use value -1
tf.debugging.experimental.enable_dump_debug_info(
logdir,
tensor_debug_mode="FULL_HEALTH",
circular_buffer_size=-1)
Copy
If you don’t have NANs in your model, you will see a blank Debugger 2 on your TensorBoard
dashboard.

2.How to display image data in TensorBoard

If you want to visualize layer weights, generated tensors or input data, TensorFlow Image
Summary API helps you view them on TensorBoard Images.

Visualizing a single image in TensorBoard

The shape of a single image in our data set is (28,28) known as a rank-2 tensor, this represents
the height and width of the image. You can examine this by running the following code

print(X_train[0].shape)
Copy
Since we are going to use `tf.summary.image()` which expects  rank-4 tensor, we have to
reshape using the `numpy` reshape method. Our output should contain (batch_size, height,
width, channels) .

Since we are logging a single image and our image is grayscale, we are setting both batch_size
and ‘channels’ values as 1.

First, let’s delete old logs and create a file writer.

rm -rf ./logs/
logdir = "logs/single-image/"
file_writer = tf.summary.create_file_writer(logdir)
Copy
Next, log the image to TensorBoard

import numpy as np

with file_writer.as_default():
image = np.reshape(X_train[4], (-1, 28, 28, 1))
tf.summary.image("Single Image", image, step=0)
Copy
Launch tensorboard

tensorboard --logdir logs/single-image

Copy
Output 

Visualizing multiple images in TensorBoard

You can visualize multiple images, by setting a range as follows:

import numpy as np

with file_writer.as_default():
images = np.reshape(X_train[5:20], (-1, 28, 28, 1))
tf.summary.image("Multiple Digits", images, max_outputs=16,
step=0)
Copy
Visualizing actual images in TensorBoard
From the above two examples, you have been visualizing mnist tensors. However, using
`matplotlib`, you can visualize the actual images by logging them in TensorBoard.

Clear previous logs

rm -rf logs
Copy
Import `matplotlib` library and create class names and initiate ‘tf.summary.create_file_writer’.

import io
import matplotlib.pyplot as plt

class_names =
['Zero','One','Two','Three','Four','Five','Six','Seven','Eight','Nine
']
logdir = "logs/actual-images/"
file_writer = tf.summary.create_file_writer(logdir)
Copy
Write a function that will create grid of mnist images

def image_grid():
figure = plt.figure(figsize=(12,8))

for i in range(25):
 plt.subplot(5, 5, i + 1)
plt.xlabel(class_names[y_train[i]])
plt.xticks([])
plt.yticks([])
plt.grid(False)
plt.imshow(X_train[i], cmap=plt.cm.coolwarm)

return figure
Copy
Next, let’s write a function that will convert this 5X5 grid into a single image that will be
logged in to Tensorboard logs

def plot_to_image(figure):
buf = io.BytesIO()
plt.savefig(buf, format='png')
plt.close(figure)
buf.seek(0)
image = tf.image.decode_png(buf.getvalue(), channels=4)
image = tf.expand_dims(image, 0)
return image
Copy
Launch Tensorboard

tensorboard --logdir logs/actual-images

Copy