Deep temporal sigmoid belief networks for sequence modeling

Zhe Gan, Chunyuan Li, Ricardo Henao, David Carlson, Lawrence Carin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

38 Scopus citations

Abstract

Deep dynamic generative models are developed to learn sequential dependencies in time-series data. The multi-layered model is designed by constructing a hierarchy of temporal sigmoid belief networks (TSBNs), defined as a sequential stack of sigmoid belief networks (SBNs). Each SBN has a contextual hidden state, inherited from the previous SBNs in the sequence, and is used to regulate its hidden bias. Scalable learning and inference algorithms are derived by introducing a recognition model that yields fast sampling from the variational posterior. This recognition model is trained jointly with the generative model, by maximizing its variational lower bound on the log-likelihood. Experimental results on bouncing balls, polyphonic music, motion capture, and text streams show that the proposed approach achieves state-of-the-art predictive performance, and has the capacity to synthesize various sequences.
Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems
PublisherNeural information processing systems foundation
Pages2467-2475
Number of pages9
StatePublished - Jan 1 2015
Externally publishedYes

Fingerprint

Dive into the research topics of 'Deep temporal sigmoid belief networks for sequence modeling'. Together they form a unique fingerprint.

Cite this