Scribd Logo
Upload

Welcome to Scribd!

  • 5b - Word Embeddings

    Uploaded by

    Talha dar
    3 views17 pages
    This document discusses convolutional neural networks (CNNs) and their application to textual data. It explains that CNNs are multilayered neural networks designed to extract increasingly complex features from data at each layer. The document compares 1D and 2D convolutions, and describes how convolutions can be applied to text data by using word embeddings to represent words as vectors in a matrix. It provides an example of convolving a weight matrix with an input sentence matrix to generate a feature map, and explains how pooling layers are used to subsample outputs and reduce dimensionality while retaining salient information.

    Original Description:

    Machine Learning

    Original Title

    5b_Word Embeddings

    Copyright

    © © All Rights Reserved

    Available Formats

    PPTX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document discusses convolutional neural networks (CNNs) and their application to textual data. It explains that CNNs are multilayered neural networks designed to extract increasingly complex features from data at each layer. The document compares 1D and 2D convolutions, and describes how convolutions can be applied to text data by using word embeddings to represent words as vectors in a matrix. It provides an example of convolving a weight matrix with an input sentence matrix to generate a feature map, and explains how pooling layers are used to subsample outputs and reduce dimensionality while retaining salient information.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Flag for inappropriate content
    3 views17 pages

    5b - Word Embeddings

    Uploaded by

    Talha dar
    This document discusses convolutional neural networks (CNNs) and their application to textual data. It explains that CNNs are multilayered neural networks designed to extract increasingly complex features from data at each layer. The document compares 1D and 2D convolutions, and describes how convolutions can be applied to text data by using word embeddings to represent words as vectors in a matrix. It provides an example of convolving a weight matrix with an input sentence matrix to generate a feature map, and explains how pooling layers are used to subsample outputs and reduce dimensionality while retaining salient information.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 17

    MACHINE

    LEARNING

    WORD EMBEDDINGS-2

    1 Hammad Afzal
    Department of CSE
    hammad.afzal@mcs.edu.pk
    2
    CNN
     A multilayered neural network with a special architecture de
    signed to extract increasingly complex features of the data a
    t each layer to determine the output

     Has one or more convolutional layers and are used mainly


    for image processing, classification, segmentation and also
    for other auto-correlated data

    Artificial Intelligence - Online Workshop 3


    1D VS. 2D CONVOLUTIONS

    4
    CONVOLUTIONS IN TEXT

    5
    CONVOLUTIONS IN TEXT

    6
    CONVOLUTIONS IN TEXT

    7
    CONVOLUTIONS IN TEXT

    8
    CONVOLUTIONS IN TEXT

    Artificial Intelligence - Online Workshop 9


    CONVOLUTIONS IN TEXT

    Artificial Intelligence - Online Workshop 10


    CONVOLUTIONS IN TEXT

    Artificial Intelligence - Online Workshop 11


    CONVOLUTIONS IN TEXT
    A sliding window function applied to an input matrix. 

    Word Vectors

    ¿
    I 0.1 -0.7 0.5 1 0 0
    Like

    This
    0.2

    -0.1
    -0.5

    0.3
    0.4

    0.7
    × 1 1 0
    -
    0.2
    Weight Matrix
    Feature Map
    Movie 0.5 0.6 -0.1

    Input Sentence Matrix

    0.1*1 + -0.7*0 + 0.5*0 + 0.2*1 + -0.5*1 + 0.4*0 = -0.2

    12
    CONVOLUTIONS IN TEXT

    Word Vectors

    ¿
    I 0.1 -0.7 0.5 1 0 0
    Like

    This
    0.2

    -0.1
    -0.5

    0.3
    0.4

    0.7
    × 1 1 0
    0.2
    0.4
    Weight Matrix
    Movie 0.5 0.6 -0.1 Feature Map

    Input Sentence Matrix

    0.2*1 + -0.5*0 + 0.4*0 + -0.1*1 + 0.3*1 + 0.7*0 = 0.4


    CNN ON TEXTUAL DATA

    Word Vectors

    ¿
    I 0.1 -0.7 0.5 1 0 0
    Like

    This
    0.2

    -0.1
    -0.5

    0.3
    0.4

    0.7
    × 1 1 0
    -
    0.2
    Weight Matrix
    0.4
    Movie 0.5 0.6 -0.1
    1

    Input Sentence Matrix Feature Map

    -0.1*1 + 0.3*0 + 0.7*0 + 0.5*1 + 0.6*1 + -0.1*0 = 1


    CONVOLUTIONS IN TEXT

    Pooling
    • Pooling layer subsamples the input.
    • Applies a max operation to the result of each filter.
    • Reduces the output dimensionality but (hopefully) keeps
    the most salient information.

    0.2
    1 max pooling
    0.4
    1
    1
    CONVOLUTIONS IN TEXT
    ACKNOWLEDGEMENTS
     Prof Dr. Imran Siddiqi (Bahria University)
     Machine Intelligence, Dr M. Hanif, UET, Lahore

    17

    You might also like