APo-VAE: Text Generation in Hyperbolic Space

Shuyang Dai, Zhe Gan, Yu Cheng, Chenyang Tao, Lawrence Carin

Research output: Contribution to journalArticlepeer-review

Abstract

Natural language often exhibits inherent hierarchical structure ingrained with complex syntax and semantics. However, most state-of-the-art deep generative models learn embeddings only in Euclidean vector space, without accounting for this structural property of language. In this paper, we investigate text generation in a hyperbolic latent space to learn continuous hierarchical representations. An Adversarial Poincare Variational Autoencoder (APo-VAE) is presented, where both the prior and variational posterior of latent variables are defined over a Poincare ball via wrapped normal distributions. By adopting the primal-dual formulation of KL divergence, an adversarial learning procedure is introduced to empower robust model training. Extensive experiments in language modeling and dialog-response generation tasks demonstrate the winning effectiveness of the proposed APo-VAE model over VAEs in Euclidean latent space, thanks to its superb capabilities in capturing latent language hierarchies in hyperbolic space.
Original languageEnglish (US)
JournalArxiv preprint
StatePublished - Apr 30 2020
Externally publishedYes

Keywords

  • cs.LG
  • stat.ML

Fingerprint Dive into the research topics of 'APo-VAE: Text Generation in Hyperbolic Space'. Together they form a unique fingerprint.

Cite this