Excercise No.

4 :
Use Keras to build an Auto Encoder Model for dimensionality reduction.

Build an autoencoder to reduce the dimension of medical MNIST images into two dimensions.
After reducing the dimension.
a) Visualize how the images appear in the latent space.
b) Restore the original images from the latent space, as well as visualise the restored image.
MNIST Corpus Medical Corpus
https://medmnist.com/
MedMNIST v2: A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image
Classification.

Copy INPUT to OUTPUT

Encoder:
Compresses the input into a latent-space representation.
Encoding function h=f(x).
Decoder:
Reconstruct the input from the latent space representation.
Decoding function r=g(h).

Types :
Vanilla Autoencoder, Multilayer Autoencoder, Convolutional Autoencoder, Regularized
Autoencoder, Variational Autoencoder
Vanilla : only one hidden layer

Define : input_size, hidden_size, output_size # Input and output should be same

# input_size (784) > hidden (32) < output_size (784)
x = Input(shape=(input_size,))
# Encoder
h = Dense(hidden_size, activation='relu')(x)
# Decoder
r = Dense(output_size, activation='sigmoid')(h)
autoencoder = Model(input=x, output=r)
autoencoder.compile(optimizer='adam', loss='mse')

Multilayer : More hidden layers (3 hidden layers here)

Define : input_size, hidden_size, output_size # Input and output should be same

# input_size (784) < hidden (128) < output_size (64)
x = Input(shape=(input_size,))

# Encoder
hidden_1 = Dense(hidden_size, activation='relu')(x)
h = Dense(code_size, activation='relu')(hidden_1)
# Decoder
hiddent_2 = Dense(hidden_size, activation='relu')(h)
r = Dense (input_size, activation='sigmoid')(hidden_2)
autoencoder = Model(input=x, output=r)
autoencoder.compile(optimizer='adam', loss='mse')

Convolutional : Uses CNN instead of Dense

x = Input(shape=(28, 28,1))

# Encoder
conv1_1 = Conv2D(16, (3, 3), activation='relu', padding='same')(x)
pool1 = MaxPooling2D((2, 2), padding='same')(conv1_1)
conv1_2 = Conv2D(8, (3, 3), activation='relu', padding='same')(pool1)
pool2 = MaxPooling2D((2, 2), padding='same')(conv1_2)
conv1_3 = Conv2D(8, (3, 3), activation='relu', padding='same')(pool2)
h = MaxPooling2D((2, 2), padding='same')(conv1_3)

# Decoder
conv2_1 = Conv2D(8, (3, 3), activation='relu', padding='same')(h)
up1 = UpSampling2D((2, 2))(conv2_1)
conv2_2 = Conv2D(8, (3, 3), activation='relu', padding='same')(up1)
up2 = UpSampling2D((2, 2))(conv2_2)
conv2_3 = Conv2D(16, (3, 3), activation='relu')(up2)
up3 = UpSampling2D((2, 2))(conv2_3)
r = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(up3)

Regularized autoencoders : use a loss function that encourages the model to have other properties
besides the ability to copy its input to its output. Example, sparse
Adding (Penalty) a reguralization term in the loss function. Doing this will make our autoencoder
learn sparse representation of data.

x = Input(shape=(input_size,))

# Encoder
h = Dense(hidden_size, activation='relu', activity_regularizer=regularizers.l1(10e-5))(x)

# Decoder
r = Dense(output_size, activation='sigmoid')(h)
autoencoder = Model(input=x, output=r)
autoencoder.compile(optimizer='adam', loss='mse')

Here, “l1” activity regularizer, that will apply a penalty to the loss function during the optimization
phase. As a result, the representation is now sparser compared to the vanilla autoencoder.

Denoised autoencoders :
Changing the reconstruction error term of the loss function. This can be done by adding some noise
of the input image and make the autoencoder learn to remove it. By this means, the encoder will
extract the most important features and learn a robuster representation of the data.
def preprocess(array):
#Normalizes the supplied array and reshapes it into the appropriate format.
array = array.astype("float32") / 255.0
array = np.reshape(array, (len(array), 28, 28, 1))
return array

def noise(array):
#Adds random noise to each image in the supplied array.
noise_factor = 0.4
noisy_array=array+noise_factor*np.random.normal( loc=0.0, scale=1.0, size=array.shape )
return np.clip(noisy_array, 0.0, 1.0)
# Normalize and reshape the data
train_data = preprocess(train_data)
test_data = preprocess(test_data)

# Create a copy of the data with added noise

noisy_train_data = noise(train_data)
noisy_test_data = noise(test_data)

input = layers.Input(shape=(28, 28, 1))

# Encoder
x = layers.Conv2D(32, (3, 3), activation="relu", padding="same")(input)
x = layers.MaxPooling2D((2, 2), padding="same")(x)
x = layers.Conv2D(32, (3, 3), activation="relu", padding="same")(x)
x = layers.MaxPooling2D((2, 2), padding="same")(x)

# Decoder
x = layers.Conv2DTranspose(32, (3, 3), strides=2, activation="relu", padding="same")(x)
x = layers.Conv2DTranspose(32, (3, 3), strides=2, activation="relu", padding="same")(x)
x = layers.Conv2D(1, (3, 3), activation="sigmoid", padding="same")(x)

# Autoencoder
autoencoder = Model(input, x)
autoencoder.compile(optimizer="adam", loss="binary_crossentropy")
autoencoder.summary()