Scribd Logo
Upload

Welcome to Scribd!

  • Hierarchical Sentence Level Attention Transformer

    Uploaded by

    Rupesh Nidhi
    11 views1 page
    This document proposes a hierarchical sentence level attention transformer model. It uses an encoder-decoder architecture with attention mechanisms at both the token and sentence levels. The encoder encodes input sentences into memory vectors, and the decoder generates output segments while attending to the encoder memories and input tokens.

    Original Description:

    Original Title

    transformer

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document proposes a hierarchical sentence level attention transformer model. It uses an encoder-decoder architecture with attention mechanisms at both the token and sentence levels. The encoder encodes input sentences into memory vectors, and the decoder generates output segments while attending to the encoder memories and input tokens.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    11 views1 page

    Hierarchical Sentence Level Attention Transformer

    Uploaded by

    Rupesh Nidhi
    This document proposes a hierarchical sentence level attention transformer model. It uses an encoder-decoder architecture with attention mechanisms at both the token and sentence levels. The encoder encodes input sentences into memory vectors, and the decoder generates output segments while attending to the encoder memories and input tokens.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 1

    Hierarchical Sentence Level Attention Transformer

    Attention for sentence selection


    att
    Q K V + encoder token out

    mem 1 mem 2 mem 3


    Encoder
    Decoder

    mem 1 mem 2 mem 3 token vectors token vectors token vectors


    mem 1 mem 2 mem 3 token vectors token vectors token vectors

    transformer layer transformer layer transformer layer transformer layer


    transformer layer transformer layer transformer layer transformer layer

    Q K V Q K V Q K V Q K V
    Q K V Q K V Q K V Q K V

    N layers
    N layers

    mem 1 mem 2 mem 3 token vectors token vectors token vectors


    mem 1 mem 2 mem 3 token vectors token vectors token vectors

    transformer layer transformer layer transformer layer transformer layer


    transformer layer transformer layer transformer layer transformer layer

    Q K V Q K V Q K V Q K V
    Q K V Q K V Q K V Q K V

    mem 1 mem 2 mem 3 input sentence 1 input sentence 2 input sentence 3


    mem 1 mem 2 mem 3 output segment 1 output segment 2 output segment 3

    You might also like