Computer Science > Machine Learning
[Submitted on 22 Jan 2016 (v1), revised 19 Mar 2017 (this version, v4), latest version 27 Mar 2018 (v6)]
Title:Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem
View PDFAbstract:In this paper, we develop a Bayesian evidence maximization framework to solve the sparse non-negative least squares problem (S-NNLS). We introduce a family of scale mixtures referred as to Rectified Gaussian Scale Mixture (R-GSM) to model the sparsity enforcing prior distribution for the signal of interest. Through proper choice of the mixing density, the R-GSM prior encompasses a wide variety of heavy-tailed distributions such as the rectified Laplacian and rectified Student-t distributions. Utilizing the hierarchical representation induced by the scale mixture prior, an evidence maximization or Type II estimation method based on the expectation-maximization (EM) framework is developed to estimate the hyper-parameters and to obtain a point estimate of the parameter of interest. In the proposed method, called rectified Sparse Bayesian Learning (R-SBL), we provide four alternative approaches that offer a range of options to trade-off computational complexity to quality of the E-step computation. The methods include the Markov Chain Monte Carlo EM, linear minimum mean square estimation, approximate message passing and a diagonal approximation. Through numerical experiments, we show that the proposed R-SBL method outperforms existing S-NNLS solvers in terms of both signal and support recovery.
Submission history
From: Alican Nalci [view email][v1] Fri, 22 Jan 2016 23:47:36 UTC (865 KB)
[v2] Tue, 8 Mar 2016 10:14:35 UTC (905 KB)
[v3] Wed, 9 Mar 2016 02:38:23 UTC (840 KB)
[v4] Sun, 19 Mar 2017 00:03:43 UTC (536 KB)
[v5] Fri, 16 Feb 2018 21:27:34 UTC (1,557 KB)
[v6] Tue, 27 Mar 2018 18:36:04 UTC (1,698 KB)
Current browse context:
cs.LG
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
IArxiv Recommender
(What is IArxiv?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.