Scribd Logo
Upload

Welcome to Scribd!

  • 2017 - Lecture 5 - Smaller Network - CNN - 1 (Ming Li) (10 Slides)

    Uploaded by

    rvasiliou
    4 views10 pages
    1. Convolutional neural networks (CNNs) apply multiple filters to input data to detect patterns, like detecting the presence or location of edges, beaks, etc. 2. CNNs apply filters in a sliding window manner over the input, performing dot products between the filter values and input region it is viewing to produce feature maps. 3. The filters are learned during training and shared across the input, which helps reduce parameters compared to fully connected networks while still detecting relevant patterns anywhere in the input.

    Original Description:

    Lecture 5 - Smaller Network - CNN [Ming Li]

    Original Title

    2017 - Lecture 5 - Smaller Network - CNN - 1 [Ming Li][10 Slides]

    Copyright

    © © All Rights Reserved

    Available Formats

    PPT, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    1. Convolutional neural networks (CNNs) apply multiple filters to input data to detect patterns, like detecting the presence or location of edges, beaks, etc. 2. CNNs apply filters in a sliding window manner over the input, performing dot products between the filter values and input region it is viewing to produce feature maps. 3. The filters are learned during training and shared across the input, which helps reduce parameters compared to fully connected networks while still detecting relevant patterns anywhere in the input.

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd
    Flag for inappropriate content
    4 views10 pages

    2017 - Lecture 5 - Smaller Network - CNN - 1 (Ming Li) (10 Slides)

    Uploaded by

    rvasiliou
    1. Convolutional neural networks (CNNs) apply multiple filters to input data to detect patterns, like detecting the presence or location of edges, beaks, etc. 2. CNNs apply filters in a sliding window manner over the input, performing dot products between the filter values and input region it is viewing to produce feature maps. 3. The filters are learned during training and shared across the input, which helps reduce parameters compared to fully connected networks while still detecting relevant patterns anywhere in the input.

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 10

    Lecture 5 Smaller Network: CNN

     We know it is good to learn a small model.


     From this fully connected model, do we really need all
    the edges?
     Can some of these be shared?
    Consider learning an image:

     Some patterns are much smaller than


    the whole image

    Can represent a small region with fewer parameters

    “beak” detector
    Same pattern appears in different places:
    They can be compressed!
    What about training a lot of such “small” detectors
    and each detector must “move around”.

    “upper-left beak”
    detector

    They can be compressed


    to the same parameters.

    “middle beak”
    detector
    A convolutional layer
    A CNN is a neural network with some convolutional layers
    (and some other layers). A convolutional layer has a number
    of filters that does convolutional operation.

    Beak detector

    A filter
    Convolution These are the network
    parameters to be learned.

    1 -1 -1
    1 0 0 0 0 1 -1 1 -1 Filter 1
    0 1 0 0 1 0 -1 -1 1
    0 0 1 1 0 0
    1 0 0 0 1 0 -1 1 -1
    0 1 0 0 1 0 -1 1 -1 Filter 2
    0 0 1 0 1 0 -1 1 -1

    …
    …
    6 x 6 image
    Each filter detects a
    small pattern (3 x 3).
    1 -1 -1
    Convolution -1 1 -1 Filter 1
    -1 -1 1
    stride=1

    1 0 0 0 0 1 Dot
    product
    0 1 0 0 1 0 3 -1
    0 0 1 1 0 0
    1 0 0 0 1 0
    0 1 0 0 1 0
    0 0 1 0 1 0

    6 x 6 image
    1 -1 -1
    Convolution -1 1 -1 Filter 1
    -1 -1 1
    If stride=2

    1 0 0 0 0 1
    0 1 0 0 1 0 3 -3
    0 0 1 1 0 0
    1 0 0 0 1 0
    0 1 0 0 1 0
    0 0 1 0 1 0

    6 x 6 image
    1 -1 -1
    Convolution -1 1 -1 Filter 1
    -1 -1 1
    stride=1

    1 0 0 0 0 1
    0 1 0 0 1 0 3 -1 -3 -1
    0 0 1 1 0 0
    1 0 0 0 1 0 -3 1 0 -3
    0 1 0 0 1 0
    0 0 1 0 1 0 -3 -3 0 1

    6 x 6 image 3 -2 -2 -1
    -1 1 -1
    Convolution -1 1 -1 Filter 2
    -1 1 -1
    stride=1
    Repeat this for each filter
    1 0 0 0 0 1
    0 1 0 0 1 0 3 -1 -3 -1
    -1 -1 -1 -1
    0 0 1 1 0 0
    1 0 0 0 1 0 -3 1 0 -3
    -1 -1 -2 1
    0 1 0 0 1 0 Feature
    0 0 1 0 1 0 -3 -3 Map
    0 1
    -1 -1 -2 1
    6 x 6 image 3 -2 -2 -1
    -1 0 -4 3
    Two 4 x 4 images
    Forming 2 x 4 x 4 matrix
    Color image: RGB 3 channels
    11 -1-1 -1-1 -1-1 11 -1-1
    1 -1 -1 -1 1 -1
    -1-1 11 -1-1 -1-1-1 111 -1-1-1 Filter 2
    -1 1 -1 Filter 1 -1 1 -1
    -1-1 -1-1 11 -1-1 11 -1-1
    -1 -1 1
    Color image
    1 0 0 0 0 1
    1 0 0 0 0 1
    0 11 00 00 01 00 1
    0 1 0 0 1 0
    0 00 11 01 00 10 0
    0 0 1 1 0 0
    1 00 00 10 11 00 0
    1 0 0 0 1 0
    0 11 00 00 01 10 0
    0 1 0 0 1 0
    0 00 11 00 01 10 0
    0 0 1 0 1 0
    0 0 1 0 1 0

    You might also like