Scribd Logo
Upload

Welcome to Scribd!

  • Word Embedding

    Uploaded by

    Pratik Kumar Pandey
    7 views12 pages
    The document discusses different types of hyperparameters used in transfer learning, including pre-processing hyperparameters like dynamic context window size and subsampling frequent words, association metric hyperparameters like shifted PMI and context distribution smoothing, and post-processing hyperparameters like adding context vectors and eigenvalue weighting. It also mentions subsampling frequent words can increase the context window size through a technique called dirty subsampling, and that adding context vectors leads to combining first-order and second-order similarity terms.

    Original Description:

    word embedding data

    Original Title

    Word embedding

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document discusses different types of hyperparameters used in transfer learning, including pre-processing hyperparameters like dynamic context window size and subsampling frequent words, association metric hyperparameters like shifted PMI and context distribution smoothing, and post-processing hyperparameters like adding context vectors and eigenvalue weighting. It also mentions subsampling frequent words can increase the context window size through a technique called dirty subsampling, and that adding context vectors leads to combining first-order and second-order similarity terms.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    7 views12 pages

    Word Embedding

    Uploaded by

    Pratik Kumar Pandey
    The document discusses different types of hyperparameters used in transfer learning, including pre-processing hyperparameters like dynamic context window size and subsampling frequent words, association metric hyperparameters like shifted PMI and context distribution smoothing, and post-processing hyperparameters like adding context vectors and eigenvalue weighting. It also mentions subsampling frequent words can increase the context window size through a technique called dirty subsampling, and that adding context vectors leads to combining first-order and second-order similarity terms.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 12

    Transferable Hyperparameters

    ● Pre-processing hyperparameters
    ● Association metric hyperparameters
    ● Post-processing hyperparameters
    Pre-processing- Dynamic Context Window
    Subsampling Frequent Words

    This technique of subsampling which leads to increase in context


    window is called as dirty subsampling
    Deleting Rare Words
    Association Metric- Shifted PMI
    Context Distribution Smoothing
    Post-processing- Adding Context Vectors

    This leads to adding up of first order(w* c*) and second order(wx.wy, cx.cy)
    similarity terms
    Eigenvalue Weighting
    Vector Normalisation
    Experiments

    You might also like