Scribd Logo
Upload

Welcome to Scribd!

  • Abstraction Mechanisms Predict Generalization in Deep Neural Networks

    Uploaded by

    Thomas Lee
    10 views1 page
    The document summarizes a research paper that proposes a new metric called the Cognitive Neural Activation (CNA) metric to predict generalization in deep neural networks. The CNA metric measures the correlation between the complexity of input information and the concentration of higher activations in deeper layers of a network. The study finds that the CNA metric highly predicts generalization ability and outperforms other metrics, especially when there is noise in the data or labels are corrupted. This suggests that the CNA metric is useful for understanding generalization capabilities of DNNs and encourages further research on the connection between input complexity and deep layer representations.

    Original Description:

    icml 2

    Original Title

    Paper-2

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document summarizes a research paper that proposes a new metric called the Cognitive Neural Activation (CNA) metric to predict generalization in deep neural networks. The CNA metric measures the correlation between the complexity of input information and the concentration of higher activations in deeper layers of a network. The study finds that the CNA metric highly predicts generalization ability and outperforms other metrics, especially when there is noise in the data or labels are corrupted. This suggests that the CNA metric is useful for understanding generalization capabilities of DNNs and encourages further research on the connection between input complexity and deep layer representations.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    10 views1 page

    Abstraction Mechanisms Predict Generalization in Deep Neural Networks

    Uploaded by

    Thomas Lee
    The document summarizes a research paper that proposes a new metric called the Cognitive Neural Activation (CNA) metric to predict generalization in deep neural networks. The CNA metric measures the correlation between the complexity of input information and the concentration of higher activations in deeper layers of a network. The study finds that the CNA metric highly predicts generalization ability and outperforms other metrics, especially when there is noise in the data or labels are corrupted. This suggests that the CNA metric is useful for understanding generalization capabilities of DNNs and encourages further research on the connection between input complexity and deep layer representations.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 1

    8/19/2020 Paper

    Abstraction Mechanisms Predict Generalization in Deep Neural Networks


    Part of Proceedings of the International Conference on Machine Learning 119 pre-proceedings (ICML 2020)
    Bibtex » Metadata » Paper » Supplemental »

    Authors
    Alex Gain, Hava Siegelmann
    Abstract
    A longstanding problem for Deep Neural Networks (DNNs) is understanding their puzzling ability to generalize well. We approach this
    problem through the unconventional angle of \textit{cognitive abstraction mechanisms}, drawing inspiration from recent neuroscience
    work, allowing us to define the Cognitive Neural Activation metric (CNA) for DNNs, which is the correlation between information
    complexity (entropy) of given input and the concentration of higher activation values in deeper layers of the network. The CNA is highly
    predictive of generalization ability, outperforming norm-and-margin-based generalization metrics on an extensive evaluation of over 100
    dataset-and-network-architecture combinations, especially in cases where additive noise is present and/or training labels are corrupted.
    These strong empirical results show the usefulness of CNA as a generalization metric, and encourage further research on the
    connection between information complexity and representations in the deeper layers of networks in order to better understand the
    generalization capabilities of DNNs.

    This website is beta: Report an Issue »


    https://proceedings.icml.cc/book/2020/hash/27b09e189a405b6cca6ddd7ec869c143 1/1

    You might also like