Scribd Logo
Upload

Welcome to Scribd!

  • AA - Lecture 4

    Uploaded by

    SAAD KHAN
    16 views30 pages
    This document discusses deep learning techniques for non-linear classification problems. It explains that continuous error functions and predictions are better than discrete ones for optimization. It introduces activation functions like sigmoid and softmax to convert scores into probabilities for classification. It also discusses representing text like emails as bags of words and using maximum likelihood estimation and Laplace smoothing for probability calculations in spam filtering.

    Original Description:

    Original Title

    AA_lecture 4

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document discusses deep learning techniques for non-linear classification problems. It explains that continuous error functions and predictions are better than discrete ones for optimization. It introduces activation functions like sigmoid and softmax to convert scores into probabilities for classification. It also discusses representing text like emails as bags of words and using maximum likelihood estimation and Laplace smoothing for probability calculations in spam filtering.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    16 views30 pages

    AA - Lecture 4

    Uploaded by

    SAAD KHAN
    This document discusses deep learning techniques for non-linear classification problems. It explains that continuous error functions and predictions are better than discrete ones for optimization. It introduces activation functions like sigmoid and softmax to convert scores into probabilities for classification. It also discusses representing text like emails as bags of words and using maximum likelihood estimation and Laplace smoothing for probability calculations in spam filtering.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 30

    Lecture 4

    Deep Learning
    Redefining Perceptron algorithm… for non
    linear regions
    Discrete vs Continuous Error
    • If we take two steps and still error is the same, we will have no idea
    where to move the line right or left.
    • But if we use continuous function we can move in any direction
    where error is less.
    • We learned that continuous error functions are better than discrete
    error functions, when it comes to optimizing.
    • For this, we need to switch from discrete to continuous predictions.
    Discrete Prediction to Continuous Predictions
    • Discrete Yes/No
    • Continuous 63.8%
    • Probabilities are function of distance from line.
    • Blue are give more probs 0.5 and red are less than 0.5
    Activation Functions
    Predictions
    • For 0.5
    • P(blue)=0.5
    • P(red)=0.5
    • For 0.8
    • P(blue)=0.8
    • P(red)=0.2
    • For 0.2
    • P(blue)=0.2
    • P(red)=0.8
    Sigmoid Function
    Step to Sigmoid
    Classification Problem
    • How can we convert these scores
    into probabilities?
    Two considerations
    • Sum of all probabilities=1
    • Probabilities should be higher for
    higher score animals
    Solution
    Classification Problem
    Softmax function
    Bag of Words

    • Any email is represented


    by bag of words
    • A dictionary is created to
    store the unique words
    present in an email
    What is the size of Vocabulary?

    SPAM HAM
    OFFER IS SECRET PLAY SPORTS TODAY
    CLICK SECRET LINK WENT PLAY SPORTS
    SECRET SPORTS LINK SECRET SPORTS EVENT
    SPORT IS TODAY
    SPORT COSTS MONEY
    Answer is 12
    1. OFFER
    2. IS
    3. SECRET
    4. CLICK
    5. LINK
    6. SPORTS
    7. PLAY
    8. TODAY
    9. WENT
    10. EVENT
    11. COSTS
    12. MONEY
    Maximum likelihood
    • P(Spam)=?
    • 3/8
    • This is called maximum likelihood of Spam message
    Maximum Likelihood
    SPAM HAM
    OFFER IS SECRET PLAY SPORTS TODAY
    CLICK SECRET LINK WENT PLAY SPORTS
    SECRET SPORTS LINK SECRET SPORTS EVENT
    SPORT IS TODAY
    SPORT COSTS MONEY

    P(“SECRET|SPAM”)=?

    P(“SECRET|HAM”)=?
    ANSWER
    P(“SECRET|SPAM”)=1/3

    P(“SECRET|HAM”)=1/15
    SPAM HAM
    OFFER IS SECRET PLAY SPORTS TODAY
    CLICK SECRET LINK WENT PLAY SPORTS
    SECRET SPORTS LINK SECRET SPORTS EVENT
    SPORT IS TODAY
    SPORT COSTS MONEY
    SPAM HAM
    OFFER IS SECRET PLAY SPORTS TODAY
    CLICK SECRET LINK WENT PLAY SPORTS
    SECRET SPORTS LINK SECRET SPORTS EVENT
    SPORT IS TODAY
    SPORT COSTS MONEY
    SPAM HAM
    OFFER IS SECRET PLAY SPORTS TODAY
    CLICK SECRET LINK WENT PLAY SPORTS
    SECRET SPORTS LINK SECRET SPORTS EVENT
    SPORT IS TODAY
    SPORT COSTS MONEY
    SPAM HAM
    OFFER IS SECRET PLAY SPORTS TODAY
    CLICK SECRET LINK WENT PLAY SPORTS
    SECRET SPORTS LINK SECRET SPORTS EVENT
    SPORT IS TODAY
    SPORT COSTS MONEY
    SPAM HAM
    OFFER IS SECRET PLAY SPORTS TODAY
    CLICK SECRET LINK WENT PLAY SPORTS
    SECRET SPORTS LINK SECRET SPORTS EVENT
    SPORT IS TODAY
    SPORT COSTS MONEY
    How can we overcome this problem?
    Laplace Smoothing…

    You might also like