Scalable model selection for belief networks

Zhao Song, Yusuke Muraoka, Ryohei Fujimaki, Lawrence Carin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

We propose a scalable algorithm for model selection in sigmoid belief networks (SBNs), based on the factorized asymptotic Bayesian (FAB) framework. We derive the corresponding generalized factorized information criterion (gFIC) for the SBN, which is proven to be statistically consistent with the marginal log-likelihood. To capture the dependencies within hidden variables in SBNs, a recognition network is employed to model the variational distribution. The resulting algorithm, which we call FABIA, can simultaneously execute both model selection and inference by maximizing the lower bound of gFIC. On both synthetic and real data, our experiments suggest that FABIA, when compared to state-of-the-art algorithms for learning SBNs, (i) produces a more concise model, thus enabling faster testing; (ii) improves predictive performance; (iii) accelerates convergence; and (iv) prevents overfitting.
Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems
PublisherNeural information processing systems foundation
Pages4610-4620
Number of pages11
StatePublished - Jan 1 2017
Externally publishedYes

Fingerprint Dive into the research topics of 'Scalable model selection for belief networks'. Together they form a unique fingerprint.

Cite this