Assignment 1 - Introduction to Deep Learning 20231

1, Given the following implementation of ResNet-34

class ResNet(nn.Module):
def __init__(self, block, layers, planes, num_classes = 10):
super(ResNet, self).__init__()
self.inplanes = 64
self.conv1 = nn.Sequential(
nn.Conv2d(3, 64, kernel_size = 7, stride = 2, padding = 3),
nn.BatchNorm2d(64),
nn.ReLU())
self.maxpool = nn.MaxPool2d(kernel_size = 3, stride = 2, padding = 1)
self.layer0 = self._make_layer(block, planes[0], layers[0], stride = 1)
self.layer1 = self._make_layer(block, planes[1], layers[1], stride = 2)
self.layer2 = self._make_layer(block, planes[2], layers[2], stride = 2)
self.layer3 = self._make_layer(block, planes[3], layers[3], stride = 2)
self.avgpool = nn.AdaptiveAvgPool2d((1,1))
self.fc = nn.Linear(512, num_classes)

def _make_layer(self, block, planes, blocks, stride=1):

downsample = None
if stride != 1 or self.inplanes != planes:

downsample = nn.Sequential(
nn.Conv2d(self.inplanes, planes, kernel_size=1, stride=stride),
nn.BatchNorm2d(planes),
)
layers = []
layers.append(block(self.inplanes, planes, stride, downsample))
self.inplanes = planes
for i in range(1, blocks):
layers.append(block(self.inplanes, planes))

return nn.Sequential(*layers)

def forward(self, x):

x = self.conv1(x)
x = self.maxpool(x)
x = self.layer0(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.avgpool(x)
x = x.view(x.size(0), -1)
x = self.fc(x)

return x
ResNet34 = ResNet(ResidualBlock,[3,4,6,3],[64,128,256,512])
Modify 1 line of code to make a ResNet-18 model instead of ResNet-34

2, Data Augmentation
Refer to https://pytorch.org/vision/0.15/transforms.html
Given the following image (you can download it via this link). Firstly, resize it to 256x256.
List of augmentation methods used in this exercise
0 RandomCrop
1 Pad (zero)
2 Random rotation
3 Random affine
4 ElasticTransform
5 Horizontal Flip
6 Vertical Flip
7 Color Jitter
8 RandomInvert(p=1)
9 Gaussian Blur (kernel size = 7)
E1 Pad (edge)
E2 RandomAdjustSharpness
E3 Pad (symmetric)

Instruction: Visualize 4 augmentation methods as follows

- Take the last 4 digits your student ID and you have to choose 4 methods correspond
to the above table (e.g. 20202023 -> methods 2, 0, 2, 3)
- Digit 2 is repeated twice, so the methods for you to do are 2, 0, 3, and E1 (if you have
a digit repeated 3 times, e.g. 3133 do 3, 1, E1 and E2; etc.)
- You can use any parameters you want for the augmentation methods, but note it
down.
Example

3, Adjust learning rate

- Set up an Adam optimizer for a random network, initial learning rate = 0.01
- Plot the change of learning rate through 50 epochs using wandb
 - The following table lists the learning rate schedulers, refer to
https://pytorch.org/docs/stable/optim.html

0 ExponentialLR
1 StepLR (step size = 10)
2 CosineAnnealingLR (T max = 10)
3 CyclicLR (mode = “triangular2”)

- From the last 2 digits of your student ID, take the remainder of each by 4 and it will
be the 2 learning rates you will have to plot
- For example, 20237672 -> 72 -> 7 % 4 = 3 (CyclicLR) and 2 % 4 = 2
(CosineAnnealingLR) ; if you get the same number twice, do the n+1 one, e.g. 37 -> 3
and 3 -> 3 and 0
- Except from the mentioned parameters, everything else will be of your choice
- Note:get_last_lr() or optimizer.__dict__['param_groups'][0]['lr'] can be helpful
4,
a, Implement VGG 16 with a 224x224 input, adding BatchNorm2d after each conv+ReLU
layer except for the ones before max pool layers(as shown in the image)
b, How many extra learnable parameters are there compared to the one without
batchnorm?