Mathematics > Statistics Theory
[Submitted on 19 Dec 2016 (this version), latest version 18 Jan 2019 (v4)]
Title:Revisiting maximum-a-posteriori estimation in log-concave models: from differential geometry to decision theory
View PDFAbstract:Maximum-a-posteriori estimation has become the main Bayesian estimation methodology in many areas of modern data science such as mathematical imaging and machine learning, where high dimensionality is addressed by using models that are log-concave and where the posterior mode can be computed very efficiently by using convex optimisation algorithms. However, despite its success and rapid adoption, maximum-a-posteriori estimation is not theoretically well understood yet. This paper presents a new decision-theoretic derivation of maximum-a-posteriori estimation in Bayesian models that are log-concave. Our analysis is based on differential geometry and proceeds as follows. First, we exploit the log-concavity of the model to induce a Riemannian geometry on the parameter space, and use differential geometry to identify the natural or canonical loss function to perform Bayesian point estimation in that space. We then show that for log-concave models the canonical loss is the Bregman divergence associated with the negative log posterior density, and that the maximum-a-posteriori estimator is the Bayesian estimator that minimises the expected loss. We also show that the posterior mean or minimum mean square error estimator is the Bayesian estimator that minimises the dual canonical loss, and establish general performance guarantees for both maximum-a-posteriori and minimum mean square error estimation. These results provide a new understanding of these estimation methodologies under log-concavity, and reveal new insights about their good empirical performance and about the roles that log-concavity plays in high dimensional inference problems.
Submission history
From: Marcelo Pereyra [view email][v1] Mon, 19 Dec 2016 12:16:26 UTC (15 KB)
[v2] Wed, 21 Dec 2016 14:49:10 UTC (15 KB)
[v3] Mon, 23 Jan 2017 13:08:42 UTC (13 KB)
[v4] Fri, 18 Jan 2019 14:22:16 UTC (1,068 KB)
Current browse context:
math.ST
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.