Learning generic sentence representations using convolutional neural networks

Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, Lawrence Carin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

42 Scopus citations

Abstract

We propose a new encoder-decoder approach to learn distributed sentence representations that are applicable to multiple purposes. The model is learned by using a convolutional neural network as an encoder to map an input sentence into a continuous vector, and using a long short-term memory recurrent neural network as a decoder. Several tasks are considered, including sentence reconstruction and future sentence prediction. Further, a hierarchical encoder-decoder model is proposed to encode a sentence to predict multiple future sentences. By training our models on a large collection of novels, we obtain a highly generic convolutional sentence encoder that performs well in practice. Experimental results on several benchmark datasets, and across a broad range of applications, demonstrate the superiority of the proposed model over competing methods.
Original languageEnglish (US)
Title of host publicationEMNLP 2017 - Conference on Empirical Methods in Natural Language Processing, Proceedings
PublisherAssociation for Computational Linguistics (ACL)acl@aclweb.org
Pages2390-2400
Number of pages11
ISBN (Print)9781945626838
DOIs
StatePublished - Jan 1 2017
Externally publishedYes

Fingerprint

Dive into the research topics of 'Learning generic sentence representations using convolutional neural networks'. Together they form a unique fingerprint.

Cite this