Computer Science > Machine Learning
[Submitted on 22 Jan 2016 (v1), revised 8 Mar 2016 (this version, v2), latest version 27 Mar 2018 (v6)]
Title:Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem
View PDFAbstract:In this paper we introduce a hierarchical Bayesian framework to obtain sparse and non-negative solutions to the sparse non-negative least squares problem (S-NNLS). We introduce a new family of scale mixtures, the Rectified Gaussian Scale Mixture (R-GSM), to model the sparsity enforcing prior distribution for the signal of interest. One advantage of the R-GSM prior is that through proper choice of the mixing density it encompasses a wide variety of heavy tailed distributions, such as the rectified Laplacian and rectified Student's t distributions. Similar to the Gaussian Scale Mixture (GSM) approach, a Type II Expectation-Maximization framework is developed to estimate the hyper-parameters and obtain a point estimate of the parameter of interest. In the proposed method, called rectified Sparse Bayesian Learning (R-SBL), we provide two ways to perform the Expectation step; Markov-Chain Monte-Carlo (MCMC) simulations and a simple yet effective diagonal approximation approach (DA). Through numerical experiments we show that R-SBL outperforms existing S-NNLS solvers in terms of both signal and support recovery and that the proposed DA approach admits both computational efficiency and numerical accuracy.
Submission history
From: Alican Nalci [view email][v1] Fri, 22 Jan 2016 23:47:36 UTC (865 KB)
[v2] Tue, 8 Mar 2016 10:14:35 UTC (905 KB)
[v3] Wed, 9 Mar 2016 02:38:23 UTC (840 KB)
[v4] Sun, 19 Mar 2017 00:03:43 UTC (536 KB)
[v5] Fri, 16 Feb 2018 21:27:34 UTC (1,557 KB)
[v6] Tue, 27 Mar 2018 18:36:04 UTC (1,698 KB)
Current browse context:
cs.LG
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
IArxiv Recommender
(What is IArxiv?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.